CIFAR-100 [1]} is commonly used to validate the performance of knowledge distillation methods, which composed of 50K training images and 10K testing images from 100 classes.
Networks of ResNet [2]} family are used on this dataset.
We set the weight decay to 5e-4 and use SGD with momentum 0.9.
The models are trained with 2 GPUs with batch size 64 on each of them.
We set the initial learning rate to be 0.1 and divide the learning rate by 10 at 32K, 48K iterations, terminating at 64K iterations.
