**Autoregressive Bigrams.** Similarly these results hold for autoregressive bigram model, where \(p(i_{1},i_{2},\ldots,i_{L})=p(i_{1})\prod_{\ell=1}^{L-1}p(i_{\ell+1}\mid i_{ \ell})\), and each \(p(j\mid i)\) is as in (16). The result is empirically confirmed in Figure 11 in Appendix B.