**Reasoning Structure Extension.** Despite the generality, the chain reasoning structure of basic CoT prompting limits its effectiveness in solving complex tasks, which require exploration like foresight and backtracking during inference. Therefore, many studies have been devoted to extending the reasoning structure by designing more intricate thought processes, _e.g._, tree- and graph-structured reasoning.