We consider an MLP with two hidden layers of size 300 and 100, respectively, with ReLU activations. Dropout is applied to the input layer and the outputs of first two full-connected layers. We use MNIST as the benchmark. We compare contextual dropout with MC dropout [1]}, concrete dropout [2]}, Gaussian dropout [3]}, and Bayes by Backprop [4]}.
Please see the detailed experimental setting in Appendix REF .
<TABLE>