Figure 19: **S-shape “Smoothed Grokking”.** Bigram data with Hutter++ model, mixing clean data with AI generated data with ratio 50 to 50. The grokking line is smoothed in the probabilistic setting. Line 1, 2, 3 are generated by using 10,000, 1,000, and 100 data to train the generating model. Compared to Figure 17, we do not limit the number of accessible real data now. \(\beta=3/2\).