Figure 3: **The loss of four categories of tokens during pretraining.** (a) shows the loss of H\(\rightarrow\)H, L\(\rightarrow\)H, H\(\rightarrow\)L, and L\(\rightarrow\)L tokens during pretraining. (b) and (c) show three cases of fluctuating tokens’ loss in L\(\rightarrow\)L and H\(\rightarrow\)H during pretraining, respectively.