Measuring Update of Outdated World KnowledgeIn this work, we define _outdated_ world knowledge as information that is conflicting between \(D_{0}\) and \(D_{1}\). For example, the President of the US may be _Barack Obama_ in \(D_{0}\) and _Joe Biden_ in \(D_{1}\). In this case, the LM should update its internal knowledge as _Joe Biden_ as the US president. If an LM is pretrained on both \(D_{0}\) and \(D_{1}\) simultaneously, there is no guarantee that the LM will acquire the correct, recent information from \(D_{1}\), especially in cases where \(D_{0}\) is much bigger than \(D_{1}\), which is one of the biggest difference between the CKL and traditional CL setting. For measuring _update_ of outdated information, we construct UpdatedLAMA which is made up of cloze statements for which answers can be found in both \(D_{0}\) and \(D_{1}\), but are conflicting.