Figure 9: The rankings of the relative performance of 20 sample mixtures trained on RedPajama and validate on the Pile. **(A)** The rankings of models of different sizes all trained for 30k steps. **(B)** The rankings for 70M models trained for different steps.