Because we do not expect thoughts to be useful in predicting every token, we would prefer the model's reward to depend less on the exact next word in the text following the thought and more on the following semantic content. There are two primary challenges here.