Figure 7: **Test accuracy for GPT-3-350M using different hyperparameters.** Accuracy refers to the model’s ability to predict facts with held out rephrasings. **Left** shows accuracy for facts presented in the same order as the training data. **Right** shows accuracy for facts presented in the reverse order.