To demonstrate the effectiveness of our approach, we consider a multistep math word problems dataset, GSM8K [1]} due to its popularity and presence of step-wise reasoning solution steps. Our distilled models outperform our baseline models of the same size that were trained to reproduce the standard chain of thought reasoning without decomposed sub-questions by over 35%.
While the results provide a proof of concept of our proposed methodology, it is essential to note that the approach is not specifically tailored to the given domain and could, in the future, be applied to other reasoning tasks.
