**LM-guided CoT Performance.** Table 2 shows a comprehensive breakdown of our method's performance. Additionally, we explore an extension of our approach, which involves sampling multiple reasoning paths and subsequently ranking the most optimal rationales. Our method with only KD outperforms the original CoT prompting with 2% gain for HotpotQA and 10% for 2WikiMultiHopQA, respectively. Figure 2 illustrates the respective rationale qualities of all prompting techniques, reinforcing the effectiveness of our method in enhancing both answer prediction and rationale qualities. When employing the original CoT prompting for questions with lengthy contexts, models frequently recycle sentences from the provided context and struggle to deliver a conclusive answer to the question. This trend is mitigated by our approach, resulting in a significant decrease in error rates. It also surpasses the performance of CoT prompting + SC and is on par with standard prompting in terms of EM and F1. For the answer inclusion score, LM-guided CoT prompting is slightly higher (1-2%) than standard prompting. Furthermore, LM-guided CoT prompting + SC achieves the highest performance across all settings.