Figure 6: **The dynamics of pretraining loss and downstream loss.** (a) and (c) represent the loss of tokens selected/unselected by SLM during pretraining in both SLM and CLM methods, while (b) represents the loss of the SLM and CLM methods on downstream corpora. We tested the above results through the process of pretraining with a total of 4 billion tokens.