* Hyperparameters such as batch size, learning rate, and learning rate scheduler influence the rate of convergence, yet do not impact the final converged loss provided that (1) their values fall within a reasonable range and (2) the model is trained with sufficient steps on adequate amounts of data. * Adjusting the batch size involves a trade-off between time and computation. The critical batch size that strikes an optimal time/computation balance can be determined based solely on the loss value. Training with this critical batch size requires twice as many training steps to achieve a specific loss value compared to using an infinite batch size (minimum possible required steps). * The context length, tokenization, data distribution and model configurations have big impacts on the constants in scaling law formulas, but do not affect the form of scaling law itself. * When given a fixed context length, tokenization, data distribution, model configurations and learning rate scheduler, we observe precise and predictable power-law scalings for performance in relation to training step, batch size, and model size, provided that the learning rate is optimally configured. * By training models with fewer than 60 million parameters, we can accurately estimate the constants in scaling-law formulas. This allows us to predict various attributes for models with up to 33 billion parameters before their training, including (1) the minimum possible loss; (2) the minimum required training steps and processed tokens to achieve a specific loss; (3) the critical batch size at any loss value; and (4) the complete test loss trajectory with arbitrary batch size. * These predicted attributes have many intriguing features, assisting us in identifying crucial factors before training large models, such as the optimal model size and training steps within a fixed computational budget, the necessary amount of data, the ideal mix ratio of multiple datasets, and more.