@inproceedings{lee-etal-2024-small,
    title = "Can Small Language Models Help Large Language Models Reason Better?: {LM}-Guided Chain-of-Thought",
    author = "Lee, Jooyoung  and
      Yang, Fan  and
      Tran, Thanh  and
      Hu, Qian  and
      Barut, Emre  and
      Chang, Kai-Wei",
    editor = "Calzolari, Nicoletta  and
      Kan, Min-Yen  and
      Hoste, Veronique  and
      Lenci, Alessandro  and
      Sakti, Sakriani  and
      Xue, Nianwen",
    booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
    month = may,
    year = "2024",
    address = "Torino, Italia",
    publisher = "ELRA and ICCL",
    url = "https://aclanthology.org/2024.lrec-main.252",
    pages = "2835--2843",
    abstract = "We introduce a novel framework, LM-Guided CoT, that leverages a lightweight (i.e., {\textless}1B) language model (LM) for guiding a black-box large (i.e., {\textgreater}10B) LM in reasoning tasks. Specifically, the lightweight LM first generates a rationale for each input instance. The Frozen large LM is then prompted to predict a task output based on the rationale generated by the lightweight LM. Our approach is resource-efficient in the sense that it only requires training the lightweight LM. We optimize the model through 1) knowledge distillation and 2) reinforcement learning from rationale-oriented and task-oriented reward signals. We assess our method with multi-hop extractive question answering (QA) benchmarks, HotpotQA, and 2WikiMultiHopQA. Experimental results show that our approach outperforms all baselines regarding answer prediction accuracy. We also find that reinforcement learning helps the model to produce higher-quality rationales with improved QA performance.",
}

We introduce a novel framework, LM-Guided CoT, that leverages a lightweight (i.e., {\textless}1B) language model (LM) for guiding a black-box large (i.e., {\textgreater}10B) LM in reasoning tasks. Specifically, the lightweight LM first generates a rationale for each input instance. The Frozen large LM is then prompted to predict a task output based on the rationale generated by the lightweight LM. Our approach is resource-efficient in the sense that it only requires training the lightweight LM. We optimize the model through 1) knowledge distillation and 2) reinforcement learning from rationale-oriented and task-oriented reward signals. We assess our method with multi-hop extractive question answering (QA) benchmarks, HotpotQA, and 2WikiMultiHopQA. Experimental results show that our approach outperforms all baselines regarding answer prediction accuracy. We also find that reinforcement learning helps the model to produce higher-quality rationales with improved QA performance.
Lee, Jooyoung Yang, Fan Tran, Thanh Hu, Qian Barut, Emre Chang, Kai-Wei
Can Small Language Models Help Large Language Models Reason Better?: LM-Guided Chain-of-Thought