We now return to our main problem, understanding model collapse in the probabilistic setting and consider the Hutter++ presented above. Thus, suppose the learner only has access to at most a dataset of size \(T\) containing the \(k\)th head of the conditional distribution \(p(\cdot\mid i)\). That is, sampled from: \(i\sim p\), \(j\sim p(j\mid i)1[j\leq k]\) (normalized appropriately), where \(p(\cdot\mid i)\) is as in Equation (16).