CIFAR-100.
A ResNet56 is adopted as the teacher network while a ResNet20 and a ResNet20-0.5\(\times \)  as student networks. The ResNet20-0.5\(\times \)  is obtained by reducing the number of channels of the ResNet20 by half.
Except for that Hinton  [1]} uses output predictions to calculate distillation loss, other methods use the output feature maps of the third residual block to calculate mimicking loss.
We show the top-1 accuracy of each method and our proposed results in Table REF .
From the results, we can find that our method significantly improves performance.
To be specific, with the ResNet56 as the teacher and the ResNet20-0.5\(\times \)  as the student, combined with RKD, our transformation achieves top-1 accuracy of 61.92%, improves 0.49% compared with original RKD performance.
For the ResNet20 as the student, the best performance achieves 70.13% applied with RCO, which improves 2.17% compared with the baseline.
