Footnote 23: We utilize the symbol “\(\rightarrow\)” to represent the data order in data curriculum. For example, “2T webpage tokens \(\rightarrow\) 500B code tokens” means that the LLM is firstly trained with 2T webpage tokens and subsequently with 500B code data tokens.