Figure 17: **Empirical Hutter++ Model with Mixing.** The initial “clean” dataset comprises \(T_{0}=10,000\) samples. For future generations, the largest model is used to synthesize data. For \(T\leq 20,000\), training data is an equal mix of clean and generated data, for \(T>20,000\) all clean data is used; the remaining training data is synthetic (so the ratio of clean data diminishes). Top-p 0.9, no temperature scaling, \(\beta=3/2\).