Table 2 shows our main experimental result on the CKL benchmark. While only the exact match (EM) is reported in Table 2, we report the F1 score as well as the mean precision at k (_P@k_, k=1,5,10,20,50,100) in Appendix J. The T5 models are originally pretrained on C4 (about 1 trillion token updates) and Wikipedia, which is considered as \(D_{0}\).7, and then continually pretrained on CC-RecentNews (corpus \(D_{1}\)) for 4 epochs (25k global training steps, about 673 million token updates) using each of the CKL methods. Each of IL, UL, NL, NLE stands for InvariantLAMA, UpdatedLAMA, NewLAMA, and NewLAMA-Easy, respectively. Detailed descriptions about the setup for this experiment are included in the caption.