Tail Narrowing.Figure 2 (for Llama2) shows that in addition to tail cutting, tail narrowing effects happen during AI-generation. One mechanism for this is lowered temperature during next-token prediction. Assume a softmax distribution on the logits \(z_{i}\) for the \(i\)th token: \(p_{i}=e^{z_{i}}/\sum_{j}e^{z_{j}}\). Define \(q_{i}^{T}=e^{z_{i}/T}/\sum_{j}e^{z_{j}/T}\) for general temperature \(T\). Then \(p_{i}\asymp i^{-\beta}\) morphs into \(q_{i}^{T}\asymp i^{-\beta/T}\) (to first order). We see that temperature scaling directly causes narrowing of tail for \(T>1\). Other mechanisms can come to play: for instance, for autoregressive models with perplexity, token-wise tail cutting can result in tail narrowing for sequence-perplexity (see Figure 35 and discussion in Appendix I).