\(\bullet\)_Graph-structured reasoning._ Although the tree structure facilitates parallel reasoning, it also imposes restrictions on the reasoning process. With more complex topological structures, graphs offer greater flexibility in reasoning, enabling the characterization of more intricate relationships and interactions. For instance, Graph of Thoughts (GoT) [517, 518] conceptualizes the reasoning process as an arbitrary graph, where vertices denote intermediate thoughts and edges denote the interdependence between these thoughts. Compared with ToT, it can further utilize thoughts from other reasoning paths when generating new thoughts. However, such an approach requires a large number of interactions with LLMs, making the thought exploration process highly inefficient. To reduce potentially meaningless thought exploration, XoT [519] further proposes to guide the search of thoughts with pre-trained policy and value networks.