cluding GPT-4 (Achiam et al., 2023), were trained on predominantly human-generated text; similarly, diffusion models like DALL-E (Ramesh et al., 2021), Stable Diffusion (Rombach et al., 2022), Midjourney (Midjourney, 2023) are trained on web-scale image datasets. These training corpora already potentially exhaust all the available clean data on the internet. A growing number of synthetic data generated with these increasingly popular models starts to populate the web, often indistinguishable from "real" data. We have thus already raced into the future where our training corpora are irreversibly mixed with synthetic data and this situation stands to get worse. Recent works call attention to the potential dramatic deterioration in the resulting models, an effect referred to as _"model collapse"_(Shumailov et al., 2023). Facets of this phenomenon have been demonstrated _empirically_ in various settings (Hataya et al., 2023; Martinez et al., 2023; Bohacek and Farid, 2023; Briesch et al., 2023; Guo et al., 2023). Theoretically, a few works are emerging to analyze the effect of iterative training on self-generated (or mixed) data (see Related Work): (Shumailov et al., 2023) coin the term _"model collapse"_ to characterize complete reversion to the mean, Alemohammad et al. (2023) analyze _"self-consuming loops"_ and Bertrand et al. (2023) show that iterative synthetic training leads to a _"clueless generator"_.