WRN: We consider WRN [1]}, including 25 convolutional layers. In Figure REF , we show the architecture of WRN, where dropout is applied to the first convolutional layer in each network block; in total, dropout is applied to 12 convolutional layers.
We use CIFAR-10 and CIFAR-100 [2]} as benchmarks. All experiments are trained for 200 epochs with the Nesterov Momentum optimizer [3]}, whose base learning rate is set as \(0.1\) , with decay factor \(1/5\)  at epochs 60 and 120. All other hyperparameters are the same as MLP except for Gaussian dropout, where we use standard deviation equal to 0.8 for the CIFAR100 with no noise and 1 for all other cases.
