Figure 9 visualizes how the EM scores of each task change as T5-Kadapters, the CKL method with the most robust performance, and T5-Vanilla are continually pretrained on \(D_{1}\). In all of the tasks, the performance of T5-Initial can be considered as the upper-bound for IL and lower-bound for UL, NL, NLE. Corresponding with our main observations, CKL allows considerable retention of _time-invariant_ world knowledge while improving updating and gaining new world knowledge compared to T5-Vanilla, mitigating the overall trade-off.