Mitigating hallucination.Many widely-used language models utilize supervised finetuning (SFT) to learn to align their interactions with users (Wang et al., 2022b; Zhang et al., 2023b). In particular, there exist many methods of generating synthetic SFT data that can improve capabilities such as reasoning and alignment (Wei et al., 2023a,b). It has been shown, however, that these synthetic data can induce hallucinations into language models by containing nontrivial amounts of hallucinated answers or by forcing models to learn to answer questions that they do not know the answer to (Zhang et al., 2023c). These cases demonstrate that synthetic data, if not applied correctly, can actually increase hallucinations in language models.