Figure 2: **The performance-vs-loss curves of smaller models pre-trained with different numbers of training tokens**. Each data point is the loss (\(x\)-axis) and performance (\(y\)-axis) of the final checkpoint of one model, i.e., each point corresponds to one model trained from scratch. We mark the results of random guess in black dashed lines.