In this section, we provide a brief analysis of the loss drop in the decay stage, examining it through the prisms of checkpoint updates and gradient information. We calculate the maximum weight element update \(max_{ij}(W_{ij}^{(t+1)}-W_{ij}^{(t)})\) across all weight matrices in the MiniCPM-2.4B (Introduced in Section 6). As depicted in Figure 7, the updates exhibit a robust correlation with the learning rate's magnitude. Notwithstanding the illustration of the two submodules (gate_proj and q_proj) module of the 25th layer), this pattern is prevalent across every layer and submodule within the network. This observation may not be trivial: the model checkpoints experience significant updates preceding the learning rate's decay, yet the loss exhibits minimal reduction. Conversely, during the decay stage, despite less pronounced weight alterations, there is an accelerated decrease in loss.