For the experiments with the customized mixed dataset, we manually curated a dataset containing 3T tokens comprising a mixture of English, Chinese, and code data. The data underwent a series of rigorous deduplication, filtering, and cleaning processes to ensure its quality. The context window was set to 4096, and each batch contained about 4M tokens. Similarly, we trained only 10 small models, each with a maximum of 60M parameters, to estimate the constant terms of the scaling-law formulas. The formulas are used to predict the test loss trajectory of models up to 33B (600 times larger). We test the accuracy of the predicted loss trajectory on both in-domain and out-of-domain test data.