1. Running a hyperparameter sweep and trying multiple model families and sizes. 2. Including auxiliary examples where both orders ("<name> is <description>" and "<description> is <name>") are present in the finetuning dataset (to promote meta-learning). 3. Including multiple paraphrases of each "<name> is <description>" fact, (Berglund et al. (2023) showed this helps with generalization.)
4. Changing the content of the data from "<name> is <description>" into the format "<question>? <answer>" for synthetically generated questions and answers.