In this paper, we introduce
Decompositional Distillation, a general framework that takes advantage of the reasoning abilities of LLMs and transfers these capabilities into smaller models.
Our methodology is motivated by the Knowledge distillation framework [1]}, where a larger teacher model transfers knowledge to a smaller student model.
