These are results for the largest GPT-3 model (175B). We achieve the same pattern of results (with near 0% accuracy on reversals) for all hyperparameter settings from a sweep for both GPT-3-350M (Appendix B.2) and for Llama-7b (Appendix B.4). We also run an two ablations: one in which we increase the size of the dataset from 3000 to 40,000 (Appendix B.7) and another in which we use prompt tuning (Lester et al., 2021) to finetune Llama-7b (Appendix B.8). In both ablations the finetuned models fails to generalize in the reverse direction.