Following JEM [1]}, all our experiments are based on the Wide-ResNet architecture [2]}. To ensure a fair comparison, we follow the same configurations of JEM by removing batch normalization and dropout from the network. We use ADAM optimizer [3]} with the initial learning rate of 0.0001 and the decay rate of 0.3, and train all our models for 150 epochs. For CIFAR10 and SVHN, we reduce the learning rate at epoch [30, 50, 80], while for CIFAR100 we reduce the learning rate much later at epoch [120, 140]. The hyperparameters of our generative training of GMMC are listed in Table REF . Compared to the configurations of IGEBM [4]} and JEM, we use a 10x larger buffer size and set the reinitialization frequency to 2.5%. We note that with these settings GMMC generates images of higher quality. Comparing the two sampling approaches: Staged Sampling and Noise Injected Sampling, we find that their performances are very similar. Therefore, only the performances of Staged Sampling are reported.
<TABLE>