The studies above adopt a shared paradigm wherein reasoning chains are generated through LLMs possessing superior reasoning capabilities. These reasoning chains are then distilled into smaller models. The effectiveness of the distillation process is improved by augmenting the sampling strategy from the larger model, for example, through the utilization of multiple sampling paths, consistency, or contrastive decoding, which leads to improved diversity and accuracy in the generated reasoning chains, ultimately benefiting the distillation process to smaller models. It's notable that language models have intricate tradeoffs and complex balances associated with multidimensional capabilities. Fu et al. (2023) emphasizes that increasing task-specific chain-of-thought capabilities through distillation may also adversely impact the models' performance in solving generalized problems.