Dependency on Learning RateEq. 3.13 depends only on \(N,S\) and \(B\) but not other hyperparameters such as the learning rate. When training with a finite batch size, the learning rate will undoubtedly have a significant impact on the loss trajectory. Removing the influence of the learning rate from Eq. 3.13 is clearly impossible. Empirically, we observe that the prediction from Eq. 3.13 is accurate when the learning rate is _optimally set_, i.e., is adjusted to maximize the rate of decrease in curvature during the initial steps. As suggested in [16, BCC\({}^{+}\)24], the optimal learning rate should decrease when the model size increases. We could simply adopt a "trial and error" method to search for the optimal learning rate in the initial training steps. Once the learning rate is determined for a fixed-sized model, we can predict its precise test loss trajectory from Eq. 3.13 (after the initial warm-up stage.).