Probing LMs for World KnowledgeThe most widely used task for probing LMs for world knowledge is the LAnguage Model Analysis (LAMA) (Petroni et al., 2019) task, which consists of cloze sentences created from a set of knowledge sources using manually defined templates. We define that an LM _knows_ a fact if it can successfully predict in a zero-shot manner the masked entity in the cloze sentence, such as "_Dante was born in_." as _Florence_. While there may be other alternatives for measuring the world knowledge encoded in LMs3, we construct our main datasets as LAMA tasks, while also additionally providing the corresponding question pairs to the cloze sentences for those who want to test on CBQA as well.