Benefits of Synthesized DataThere is a range of results showing benefits of AI-synthesized data in training better models, though mostly these results pertain to image data, specifically in the context of diffusion models (Azizi et al., 2023; He et al., 2023; Shipard et al., 2023; Bansal and Grover, 2023; Lin et al., 2023), though not only (see Dai et al. (2023); Xu et al. (2023); Huang et al. (2022); Wang et al. (2023) for chat-related examples). One might argue that they either throw model-collapse caution to the winds or, possibly, settle in the protected corner where mild amounts of synthetic data (or larger amounts of "mildly synthetic" data, like in the case of data augmentation) helps. In particular, often benefits of synthetic data are observed when the synthetic data is generated by a model trained for a different use case than the downstream task (like images synthesized from diffusion models helping classification models) or generated by a stronger model (He et al., 2023; Shipard et al., 2023; Bansal and Grover, 2023; Lin et al., 2023). However, other works critically analyze the purported benefit of generated data. (Burg et al., 2023) find that while synthesized data from a diffusion model helps improving downstream tasks, such as classification, using the _pre-training data_ of the diffusion model alone gives even stronger performance (which we can interpret as evidence of mild first-generation model collapse). All in all it is fair to say that the impact of data augmentation using generative models is still not fully understood.