In our work, we use a math word problem (MWP) dataset. Solving MWPs has been an interesting area of research for the last couple of years [1]}, [2]}, [3]}, [4]}, [5]}, [6]}, [7]}, [8]}. However, the majority of the modern approaches are shifting towards the use of LLMs, often relying on approaches involving prompting or in-context learning [9]}, [10]}, [11]}, [12]}, [13]}, [14]}.
The most popular approach is the chain of thought prompting [11]}, which prompts the language model to generate a series of natural language-based intermediate steps that improve the reasoning capabilities in LLMs. [16]} took another step forward and sampled multiple reasoning paths and selected the most relevant output using majority voting. [17]} used the most voted outputs to further fine-tune the model for better performance. [18]} further improved the reasoning of LLM in a zero-shot manner by appending “Let's think step by step” to the prompt. Our work, however, not only trains the model rather than just prompting but also explicitly guides the reasoning path using sub-questions at each step which is missing in all the above reasoning chains. One work that is close to our work is by [19]} which decomposes the questions into multiple sub-questions and asks the language model to solve each sub-question sequentially. However, this work is also restricted to prompting and only works with LLMs with billions of parameters and cannot be directly transferred to smaller models.
