Such distributions are ubiquitous in natural datasets, from Zipf's law (Zipf, 1935) in distribution of word frequencies, to biological data, earthquake magnitudes, financial data etc. - this is the data being consumed by large models at scale, like LLMs. But what distribution do AI-models generate when trained on such data? Figure 2 provides an empirical answer for a large scale LLM (Llama2-7B) and a transformer model trained on an arithmetic task. Regenerating heavy-tailed data affects the distribution in two possible ways: (1) "Cutting off" the tail of the distribution and/or (2) "Narrowing" the tail (see Figure 1 for a cartoon illustration). The mechanisms leading to this, apart from finite sampling bias (as already proposed in Shumailov et al. (2023) - see Section 2 for a derivation in the Zipf-setting), stem from deliberate choices in the generation algorithms of the models: in LLMs via truncated next token prediction at inference (e.g. selecting more likely tokens via _top-_ or _top-_\(k\)_ truncation, concentrating the probability distribution by lowering the temperature); in vision models like GANs via truncation or in diffusion models through guidance.