Training Process and Comparative AnalysisThe training progress reveals a consistent trend of improvement across various datasets, with particular strides seen in language understanding, reasoning, and domain-specific knowledge. Notably, datasets such as HellaSwag, PIQA, and ARC show marked improvements, indicative of enhanced reasoning capabilities. The model shows notable progress in specialized fields such as mathematics (GSM8K and TheoremQA) and science (ARC-c and ARC-e), emphasizing its increasing ability to understand and produce content specific to these domains. The evaluation results of the intermediate checkpoints during our pre-training process are shown in Table.4.