Let \(T\) be an arbitrary task and \((D_{i})_{i=0}^{n}\) be a sequence of corpora used for LM pretraining, where \(D_{0}\) is the initial pretraining corpus. We define \(\text{Gap}(T,D_{a},D_{b})=Score(T)\) of \(LM_{a}-Score(T)\) of \(LM_{b}\), where \(LM_{a}\) represents the LM after being pretrained on \(D_{a}\). Then, we denote \(\mathbb{T}^{F}=(T_{i}^{F})_{i=0}^{n-1}\) as a sequence of tasks from \((D_{i})_{i=0}^{n-1}\) measuring the forgetting of invariant-knowledge from each corresponding corpus. If there is no such task from corpus \(D_{i}\), the value of \(T_{i}^{F}\) is set to \(n.d.\), which means _not defined_. Likewise, we denote \(T_{n}^{U}\) and \(T_{n}^{A}\) as tasks from \(D_{n}\) measuring the _update_ and _acquisition_ of new knowledge, respectively.