Several empirical studies confirm the deleterious effect of training on self-generated data: In the context of image generation, (Martinez et al., 2023a;b) report degradation of models trained on AI-generated data. Specifically, they use a Denoising Diffusion Implicit Model and a few (relatively small) datasets (e.g. Orchids, MNIST) to demonstrate visual degradation when training in successive generations of AI-generated data. (Hataya et al., 2023) _"conclude that generated images negatively affect downstream performance, while the significance depends on tasks and the amount of generated images"_, (Bohacek & Farid, 2023) reports that the popular StableDiffusion model collapses when iteratively retrained on self-generated faces, even with as little as \(3\%\) synthetic data mixed into the original training set. For text, (Briesch et al., 2023) use _nanoGPT6_ on a curated 10K logical-expression dataset to demonstrate the iterative collapse of self-consuming loops - the model and dataset are sufficiently small to allow training from scratch. (Guo et al., 2023) observe a decline in linguistic diversity metrics across iteratively fine-tuned LLMs.