Figure 4: **Experimental Results (Details in Section 6). Left plot. The scaling law for finetuning Llama2–7B on the Wikitext-103 dataset. ’0-gen’ utilizes the original data, while subsequent generations use data generated by the previous model. Middle plot. Scaling law of the transformer model trained to predict GCD of two integers. Data is synthesized from a 0th generation model trained on \(300K\) samples. Note the tapered-off scaling of the model trained on synthesized data, as predicted by our theory. Right plot.” Skills” (bursts of new GCDs) learned by the GCD-transformer on original (bottom) and AI data (top).