Our findings show that continual pre-training is an efficient and promising alternative to re-training when updating LLMs on new data. Equipped with our strategies, practitioners can efficiently update their existing models (Rae et al., 2021; Hoffmann et al., 2022; Touvron et al., 2023; Jiang et al., 2023; Gemma Team et al., 2024) on newly created higher-quality datasets. These strategies might also be relevant for pre-training curricula such as the ones used by Gemma Team et al. (2024). With the strong incentive for our community to continue creating datasets of increasing quality, we only expect the need for continual pre-training to increase.