where \(TV(a,b):=\sum_{j}|a_{j}-b_{j}|\) is the total-variation distance and the expectation is over the randomness in \(q_{T}\). An asset here is that (Berend and Kontorovich, 2012) can be used to control the quantities \(\operatorname{\mathbb{E}}\left[TV(q_{T}(\cdot\mid i),p(\cdot\mid i))\right]\). Note that TV is upper-bounded by the square-root of KL-divergence, thanks to _Pinker's inequality_. This gives indication that our results could also apply in the setting of autoregressive models with perplexity loss, like LLMs.