The original scaling law paper by OpenAI presented the formulas of scaling laws and illustrated how they could aid in determining optimal training configurations [20]. Nonetheless, the presented formulas are based on static exponents estimated from their specific experiment setup. The full details on how to derive the constant terms in the scaling-law formulas for a new experiment setup (model architecture, tokenization, data distribution, etc) remain undisclosed. Furthermore, [20] only conducted experiments with models containing up to 1.5B parameters, a size significantly smaller than that of contemporary large language models. There have been subsequent works that study scaling laws on larger models [3, 23, 24]. Some have drawn different conclusions from the original scaling-law paper, casting doubt to the general applicability of scaling laws. For example, [23] claimed that the training data size should be scaled much more than the recommendation in [20]. [3] suggested that the optimal batch size depends only on the compute budget rather than the loss value.