Three datasets are utilized including CIFAR-10, CIFAR-100 [1]} and ImageNet [2]}. AlexNet [3]} and ResNet [4]} are evaluated while ResNet include Res-{18, 34, 50, 101, 152}.
The last layer size of each model is changed in order to adapt them on CIFAR datasets. Additionally for AlexNet, the kernels in first two convolution layers are set as \(3 \times 3\)  with \(\text{padding} = 2\)  and \(\text{stride} = 1\) . For FC-1 and FC-2 layers in AlexNet, they are also resized to \(4096 \times 2048\)  and \(2048 \times 2048\) , respectively.
For ResNet, kernels in first layer are replaced by \(3 \times 3\)  kernels with \(\text{padding} = 1\)  and \(\text{stride} = 1\) . Meanwhile, the pooling layer before FC-1 in ResNet is set to Average-Pooling with the size of \(4 \times 4\) .
