* _Reasoning inconsistency._ With improved reasoning strategies (_e.g.,_ CoT prompting), LLMs can solve some complex reasoning tasks, by performing step-by-step reasoning based on the supporting logic and evidence. Despite the effectiveness, the _reasoning inconsistency_ issue often occurs in the decomposed reasoning process. Concretely, LLMs may generate the correct answer following an invalid reasoning path, or produce a wrong answer after a correct reasoning process [33, 442], leading to inconsistency between the derived answer and the reasoning process. To alleviate this problem, existing work has proposed to guide the whole generation process of LLMs via external tools or models [636, 451, 437], to re-check the reasoning process and final answer for correcting the potential errors [685, 686, 687] or fine-tune LLMs with process-based feedback [688, 689]. For instance, _Tree of Thoughts (ToT)_[451] empowers LLMs to engage in the decision-making process by concurrently exploring and self-evaluating various reasoning paths. To refine the reasoning processes, Self-Refine [685] elicits feedback from LLMs on self-generated solutions, enabling the iterative refinement of solutions based on the feedback. Moreover, several studies improve the consistency in the reasoning chain of LLMs through the integration of process-based supervision during training [688, 689]. As a promising solution, recent approaches reformulate the complex reasoning tasks into code generation tasks, where the strict execution of the generated code ensures the consistency between the reasoning process and the outcome. Also, it has been revealed that there might exist inconsistency between tasks with similar inputs, where small changes in the task description may cause the model to produce different results [592, 49]. To mitigate this problem, self-consistency [436] adopts the ensemble of multiple reasoning paths to enhance the decoding process of LLMs.