Note that CPG is a much more challenging task than the well-studied next-token-prediction task [1]}, [2]}. The goal is to predict long sequences of tokens (e.g., an entire method body). Also, \(\mathsf {X}\)  is a (possibly imprecise) specification of the code to generate, not just a sequence of tokens we are trying to complete by, say, choosing the correct method to call for a variable.
