Hoffmann et al. (2022) investigate the optimal model size and number of training tokens for training a transformer language model under a given compute budget. The authors train over 400 language models ranging from 70 million to over 16 billion parameters on 5 to 500 billion tokens and find that for compute-optimal training, the model size and number of training tokens should scale at equal rates: for every doubling of model size, the number of training tokens should also be doubled. They then train a model called Chinchilla that is compute-optimal according to their results. For this reason, the scaling laws they propose are often called "Chinchilla scaling laws".