This result is empirically confirmed in Figure 21 and proved in Appendix E. It gives rise to similarly tapered-off scaling curves for synthetic data, as in the simpler models. The proofs for loss of scaling across generations in Section 2 and grokking phenomena in Section 3, carry over to this model as well, demonstrating their universality.