decreases, as the LM distribution anneals to argmax, as well as when temperature increases, as the distribution collapses to uniform. For both top-p and top-k we note that the model requires almost all of the distribution to perform well, indicating that there is significant information in the tails.