This is well known for LLM pre-training Kandpal et al. (2023), and we see in this case that this holds for fine-tuning as well. The rationale for this hypothesis is that mere memorization of sentences does not entail knowledge of their content, as was already shown in Berglund et al. (2023). By providing the information in numerous forms (like the data augmentation process we used), the various relationships in the data (e.g., \(a\implies b,\ b\implies c\)) stand a higher chance of appearing naturally. We believe this can potentially both increase \(\mathcal{L_{M,Q}}\) in general, as well as ameliorate Berglund et al.'s _Reversal Curse_. While promising, this result still warrants further research.