Despite the significant advancements Large Language Models (LLMs) have achieved across various tasks, they continue to encounter difficulties with multi-step reasoning problems (Lightman et al., 2023; Huang et al., 2023; Shi et al., 2023; Chen et al., 2024). A primary challenge in reasoning arises from the fact that even a minor error can disrupt the entire solution process (Lightman et al., 2023). This issue is compounded in state-of-the-art models, which often suffer from hallucinations that lead to such errors (Huang et al., 2023). Analyses of the error patterns in mathematical reasoning tasks performed by GPT-3 reveal that over 50% of errors stem from either a misunderstanding of the problem or from incoherent reasoning steps (Wei et al., 2022).