One may wonder whether we can find the optimal data mixtures on small models and few numbers of steps, and then transfer the found mixture proportions to large-scale training. To answer this question, we compare the relative performance of models in different sizes and trained with different numbers of steps in Fig. 9.