and train it via language modeling on the true prompt \(x\mid p(x)=v\). For training, we sample outputs from a checkpoint of our conditional LM (section 4) to use as hypotheses. We train the model on outputs from Llama-2 (7B) for \(100\) epochs using the same hyperparameters outlined in Section 6. This model is able to achieve a one-step BLEU score of \(58.7\) on Instructions-2M, essentially recovering the original model's BLEU performance of \(59.2\). However, we see no increase in BLEU score after applying multiple steps of correction; after 5 steps, our model achieves a BLEU score of \(56.43\).