Reversal Curse & MitigationsThe reversal curse was identified by the concurrent works Berglund et al. (2023); Allen-Zhu & Li (2023); its name was derived from the former. They demonstrated that the reversal curse occurs across model sizes and families, including very large models such as GPT-3.5 and GPT-4. They found that including auxiliary examples with both orders present in the finetuning or pre-training datasets (to promote meta-learning) does not aid generalization to examples where only one order is given, even if such data is rewritten in a question-answer format. Furthermore, including multiple paraphrases of each fact in a single direction does not facilitate learning in the reverse direction, despite aiding the given direction, as shown by Berglund et al. (2023); Allen-Zhu & Li (2023).