Following the revelation of the scaling law Kaplan et al. (2020), there has been a vigorous pursuit in the field of Large Language Models (LLMs) Hoffmann et al. (2022); Bai et al. (2023); Gemini et al. (2023); Chowdhery et al. (2023); Achiam et al. (2023), encompassing models with up to an astonishing number of parameters in the trillions Fedus et al. (2022). These models have emerged as a pivotal driving force in the evolution of artificial intelligence.