**Small Language Models.** "Small Language Models" (SLMs) is an evolving concept that has undergone significant transformations over time. Presently, SLMs are generally construed as models that are smaller in scale compared to the well-known LLMs, typically not exceeding 7 billion parameters. These models are distinguished by their capacity for deployment on end-user devices, such as personal computers and smartphones, even in the absence of a GPU. Notable examples within the current landscape of SLMs include the Phi series (Gunasekar et al., 2023; Li et al., 2023; Javaheripi and Bubeck, 2023), TinyLlama (Zhang et al., 2024), MobileLLM (Liu et al., 2024), and Gemma (Banks and Warkentin, 2024), etc. A variety of methodologies have been explored to augment the efficacy of SLMs. These include the incorporation of high quality data (Gunasekar et al., 2023; Li et al., 2023; Javaheripi and Bubeck, 2023), the application of structure pruning techniques (Xia et al., 2023), and the reconfiguration of model architectures (Liu et al., 2024), among others. MiniCPMenhances the capabilities of SLMs through a meticulous amalgamation of hyper-parameter optimization, strategic training methodologies, architectural design, and high quality data.