Measuring Retention of Time-invariant World KnowledgeWe define _time-invariant_ world knowledge as the information present in \(D_{0}\) that has no possibility of conflicting with information from \(D_{1}\). For example, if the information of the _birthplace of Barack Obama_ is present in \(D_{0}\), it is unlikely that \(D_{1}\) contains information that contradicts that fact. Also, we classify instances where the time-stamps are fixed such as "_Cristiano Ronaldo played for_. _in 2010_." as _time-invariant_. These _time-invariant_ instances should not be changed as LMs are continually pretrained on \(D_{1}\). In order to measure how much _time-invariant_ information is lost due to _catastrophic forgetting_ during continued pretraining, we create InvariantLAMA, a subset of LAMA (Petroni et al., 2019), consisting of only _time-invariant_ cloze sentences detailed in Appendix B.1.