To address this, much progress has recently been made to obtain compact deep networks. Existing methods include pruning [1]}, [2]}, [3]}, [4]}, [5]} and quantization [6]}, [7]}, [8]}, both of which aim to reduce the size of an initial deep architecture, as well as knowledge distillation, whose goal is to exploit a deep teacher network to improve the training of a given compact student one. In this paper, we introduce a knowledge distillation approach for object detection.
