We conducted extensive experiments to verify the effectiveness of CTSL-MKT on image classification tasks using datasets including CIFAR-10 [1]}, CIFAR-100 [1]}, Tiny-ImageNet [3]} and Market-1501 [4]}. The peer network architectures were chosen from ResNet [5]}, MobileNet [6]}, ShuffleNet [7]}, VGG [8]}, AlexNet [9]} and SqueezeNet [10]}. CTSL-MKT was compared to the state-of-the-art KD methods, which are DML [11]}, Tf-KD [12]} and RKD [13]}. For a fair comparison, RKD uses online distillation with the peer networks. In all the experiments, the relation-based knowledge was modelled by the final feature embedding outputs by the peer networks.
