Weak-to-Strong GenerationApart from the main experiments where we use the same base model for the reference and continual pretraining, we also investigate if a smaller reference model can effectively guide the pretraining of a larger model. We use Tinyllma-1.1B as reference model and continual pretraining Llama-2-7B on math. Results presented in Table 3 indicate that, despite the considerable gap between the small and large models (Li et al., 2023), employing the small reference model to token selection can still yield benefits to the pre-training of the larger model. If reference and training models have different vocabularies, one can consider performing token alignment (Wan et al., 2024; Fu et al., 2023), which we leave for future work.