From \(S_{\min}\) to \(S\).Thankfully, there is a conversion between the training step \(S\) with any batch size \(B\) and the training step \(S_{\min}\) with sufficiently/infinitely large batch size. Let \(\theta\) be the model parameters at some point during optimizing the model, \(G_{\mathrm{est}}\) be the noisy gradients estimated by SGD at the point. Note that \(G_{\mathrm{est}}\) is a random variable whose expectation is the real gradients \(G\) with infinitely large batch size (_i.e._, \(\mathbb{E}[G_{\mathrm{est}}]=G\)).