Figure 12: Loss curve on C4 dataset for MiniCPM-1.2B (Left) and MiniCPM-2.4B (Right). The light green segment at the tail of the loss curve represents the remaining decay process, which is not utilized in the released version of MiniCPM.