We next explore if our reverse training can be applied to the finetuning stage when the model is learning new, previously unseen knowledge from a small training dataset. We use the same pre-trained models described in Section 3.3 and an additional LLama-2 7B model, and further finetune them on a dataset made up of fictitious facts. These data are made up of statements of the form "[name] is [description]" (or the reverse) where the names and descriptions are randomly generated. The fictitious nature of this dataset guarantees that those facts are not seen during pre-training, and are only seen in the specified direction during finetuning. The model is then tested to see if it is able to learn these facts in the same or reverse direction that it has seen during training.