**Learning New Information** In Figure 3, we can see an interesting phenomenon observed throughout our experiments. After each epoch, i.e., completing another iteration over the entire dataset, the training loss drops significantly. This is consistent with what is known about LLMs memorizing the data during training and overfitting Tirumala et al. (2022).