**Knowledge Reasoning.** The knowledge reasoning tasks rely on logical relations and evidence about factual knowledge to answer the given question. Existing work mainly uses specific datasets to evaluate the reasoning capacity of the corresponding type of knowledge, _e.g.,_ CSQA [504]/StrategyQA [185] for commonsense knowledge reasoning and ScienceQA [565] for science knowledge reasoning. In addition to the accuracy of the predicted results, existing work [565] has also evaluated the quality of the generated reasoning process, via automatic metrics (_e.g., BLEU) or human evaluation. Typically, these tasks require LLMs to perform step-by-step reasoning based on factual knowledge, until reaching the answer to the given question. To elicit the step-by-step reasoning ability, chain-of-thought (CoT) prompting strategy [33] has been proposed for enhancing the complex reasoning capacity of LLMs. As discussed in Section 6.3, CoT involves the intermediate reasoning steps, which can be manually created [33] or automatically generated [674], into the prompts to guide LLMs to perform multi-step reasoning. Such a way largely improves the reasoning performance of LLMs, leading to new state-of-the-art results on several complex knowledge reasoning tasks [33, 56, 526]. Further, after reformulating knowledge reasoning tasks into code generation tasks, researchers have found that the performance of LLMs can be further improved [211], especially with the LLMs pre-trained on code. However, due to the complexity of knowledge reasoning tasks, the performance of current LLMs still lags behind human results on tasks such as commonsense reasoning [675, 56, 33]. As a common type of mistakes, LLMs might generate inaccurate intermediate steps, leading to a wrong final result. To address this issue, existing work has proposed special decoding or ensemble strategies to improve the accuracy of the whole reasoning chain [436, 437].