our method disrupts zero-shot performance on common evaluation tasks: BoolQ (Clark et al., 2019), PIQA (Bisk et al., 2020), SIQA (Sap et al., 2019), HellaSwag (Zellers et al., 2019), WinoGrande (Sakaguchi et al., 2021), ARC easy and challenge (Clark et al., 2018), OpenBookQA (Mihaylov et al., 2018). We also report 5-shot performance on the aggregated MMLU benchmark (Hendrycks et al., 2020). Evaluation results are summarized in Table 7. We observe that our entity reversal trained model is slightly better than the standard data-matched 1.4B model trained in the standard forward direction, and is only 1.8 points behind the compute-matched model in accuracy on average, despite being trained on half of the tokens. We note that token reversal works slightly better than entity reversal on these standard benchmarks, and hence superior to the data-matched standard training as well.