**Data Curriculum.** After preparing the data mixture, it is important to schedule the order that specific data is presented to LLMs for pre-training. It has been shown that, in some cases, to learn a certain skill, learning in a skill-set sequence (_e.g.,_ basic skills \(\rightarrow\) target skill) outperforms direct learning from a corpus focused solely on the target skill [234, 235]. Following the idea of curriculum learning [236], _data curriculum_ has been proposed and widely used in model pre-training [234, 235, 237, 238]. It aims to organize different parts of pre-training data for LLMs in a specific order, _e.g.,_ starting with easy/general examples and progressively introducing more challenging/specialized ones. More generally, it can broadly refer to the adaptive adjustment of data proportions for different sources during pre-training. Existing work about data curriculum mainly focuses on continual pre-training, such as specialized coding LLMs (_e.g.,_ CodeLLMa [235]) or long context LLMs (_e.g.,_ LongLLaMA [238]). However, it still lacks of more detailed report about data curriculum for general-purpose LLMs (_e.g.,_ LLaMA) in the literature. To determine data curriculum, a practical approach is to monitor the development of key abilities of LLMs based on specially constructed evaluation benchmarks, and then adaptively adjust the data mixture during pre-training. Next, we take three common abilities as examples to introduce how the concept of data curriculum23 applies in continual pre-training.