We now study mixing of clean and synthesized data in the bigram setting. Figures 17 and 18 add top-p tail-cutting when synthesizing, and start with \(T_{0}=10,000\) original data samples, which are successively blended with synthesized data from the largest model. Note that in this setting we observe a reversion of scaling laws with increased AI data. This needs to be compared with the orange curve in Figure 20 in the deterministic Hutter setting. The probabilistic nature of the bigram models leads to a new effect here.