Reference Model TrainingTo train our mathematical reference model, we gathered a dataset of 0.5B high-quality, math-related tokens. This dataset is a blend of synthetic data from GPT (Yu et al., 2024; Huang et al., 2024) and manually curated data (Yue et al., 2024; Ni et al., 2024). For the general reference model, we compiled a corpus of 1.9B tokens from open-source datasets, such as Tulu-v2 (Ivison et al., 2023) and OpenHermes-2.5 (Teknium, 2023). We trained the reference models for 3 epochs. The maximum learning rate was set at 5e-5 for 1B models and 1e-5 for 7B models, applying a cosine decay schedule. We set the maximum sequence lengths to 2048 for 1B models and 4096 for 7B models, packing multiple samples into these lengths for model input. In all main experiments, we initialized the continual pretraining model and the reference model with the _same_ base model.