In this work, we first investigate the dynamics of several open-sourced large language models (i.e., Baichuan-7B (Yang et al., 2023), DeepSeek-7B (DeepSeek-AI et al., 2024), Amber-7B (Liu et al., 2023), OpenLLaMA-7B (Geng and Liu, 2023; Touvron et al., 2023; Computer, 2023), Yi-34B (AI et al., 2024) and DeepSeek-67B), and analyze their performance results across diverse tasks using the corresponding intermediate checkpoints based on the number of pre-trained tokens.