Curating pretraining data for LLMs.Training massive transformer architecture on trillions of tokens, a.k.a. pretraining, is the primary step to building modern large language models that exhibit impressive human-like generalisation abilities (Brown et al., 2020; OpenAI, 2023; Jiang et al., 2023; Touvron et al., 2023b)). It takes up most of the computation resources for model training and researchers believe it endows almost all the knowledge in LLMs (Zhou et al., 2024). Such critical impact motivates the development of data curating strategies to reduce computation costs and enhance knowledge (Longpre et al., 2023). The efforts can be categorized into two steps. The first step focuses on obtaining a high-quality training dataset. A typical procedure includes selecting data sources to constitute different domains, deduplication, and the most intricate filtering (Wenzek et al., 2019; Penedo et al., 2023). A mass of endeavors in existing practice has involved multifarious filters, scoring the documents with from superficial features on characters (Rae et al., 2021; Xie et al., 2024; Raffel et al., 2020) to semantics including similarity to the high-quality reference corpus (Wenzek et al., 2019) and toxicity (Longpre et al., 2023; Friedl, 2023). With a dataset on hold, the second step aims to make the best use of it. This includes tuning the data mixture (Du et al., 2022; Touvron et al., 2023; Xie et al., 2024) and devising data schedules (Mindermann et al., 2022; Albalak et al., 2023; Chen et al., 2024; Fan et al., 2024). Our work is among those tune data mixtures and our extension to continue pretraining signifies our prospect of guiding the schedule design. Different from existing attempts that rely on intuition or qualitative targets, our study seeks a quantitative solution.