Essentially, scaling laws [14] reveal how to predict the _validation/test_ loss3 of a given model, which can be the final loss when the model is trained to converge, or the loss at a certain step amid training. Scaling laws have been proven to be widely valid across a variety of likelihood-based tasks [15]. By adhering to scaling laws, researchers can uncover patterns in how changes in model parameters and training data impact the overall effectiveness of large language models before actually training them.