Here, we present the core of our theory for the simplest case of (i) _infinite memory_ and (ii) _a deterministic ground truth_ labeling function \(i\mapsto y_{i}\), studied by Hutter (2021) (the _"Hutter LLM"_). Both restrictions will be lifted in later sections, where we also analyze an _probabilistic autoregressive_ version (Section 4) and _limited memory_ models (Section 5). Token \(i\) is drawn according to the Zipf law in Equation (1), which e.g. models distribution of various metrics in language. Another interpretation of the appearance of a power-law is offered by the "Quantization Hypothesis" paper of Michaud et al. (2023): one may think of each \(i\) as some discrete skill, needed to solve a problem for example; thus, the skills occur at different rates \(p_{i}\). The shape parameter \(\beta>1\) controls the length of the tail of this distribution: bigger values of \(\beta\) correspond to longer tails.