Let's consider a set \(\mathcal{T}\) of sequences \((y_{i})_{i\in[1,|y|]}\) of elements taken from a vocabulary of size \(V\), representing the pretraining data. We consider a function \(\phi^{*}\) that _perfectly_ (e.g. in a bijective way) represents a given context \(y_{<i}\) as a single real vector of _infinite_ dimension. As we do not focus on \(\phi^{*}\), we can simplify the notations by introducing the contextual representations \(x_{i}^{*}=\phi^{*}(y_{<i})\).