Another limitation of this work is the loose nature of the mathematical connection that we establish between the dimensionality of the ideal language modeling head and the rank-constrained performance (cf. Theorem 5.3). Moreover, it can also be argued that considering _ideal_\(x_{i}^{*}\) representations is an ill-defined notion. We argue that the reasoning behind Theorem 5.3 could be applied to any contextual representations, as the _ideal_ nature of \(x_{i}^{*}\) is not necessary in the demonstrations. The word _ideal_ reflects that our observations hold for \(x_{i}^{*}\) representations obtained from _any underlying model_, to an extent that depends on the structure that these representations impose on the \(W^{*}\) matrix for a given training set \(\mathcal{T}\).