Some works focus on mitigating factual mistakes. He et al. (2023) introduces external knowledge to evaluate reasoning chains and votes to filter out chains that contain factual mistakes but without correcting them Wang et al. (2023) adopts a similar way, with the difference that it additionally introduces a reflection mechanism to correct low-scoring reasoning. Zhao et al. (2023) filters out low-confidence reasoning by consistency and guides models to re-reasoning based on relevant external knowledge. While the aforementioned methods work well on knowledge-intensive tasks, they fall short in addressing the challenge of contextual inconsistencies. Zhang et al. (2023) explores the hallucination snowballing phenomena during the reasoning process. Others aim to address the inconsistency issues. Radhakrishnan et al. (2023) observes that models are more faithful when dealing with simple questions. Thus, it improves faithfulness through question decomposition. Faithful CoT (Lyu et al., 2023) initially generates symbolic reasoning chains and later deterministically executes symbolic functions, mitigating reasoning inconsistencies. Lanham et al. (2023) explores the factors that influence faithfulness, which provides an empirical perspective. It finds faithfulness varies on different tasks and decreases as the model size increases. CoNLI (Lei et al., 2023) proposes a post-editing strategy to diminish the hallucinations. SynTra (Jones et al., 2023) performs prefix-tuning on a synthetic dataset designed to elicit hallucination easily, and then transfers this capability to real tasks.