Knowledge distillation was described by Hinton, et al. [1]} using two networks for model compression: the teacher and student networks. Usually, the student network is smaller than the teacher network and the goal is network compression, in which the smaller student network achieves similar performance as the teacher.
