Ba et al. [1]} proposed an adaptive Dropout for training deep neural networks, which is achieved by computing local expectations of binary dropout variables and by calculating derivatives using backpropagation and stochastic gradient descent. The experiments showed that the method achieved low misclassification rates in the MNIST and NORB datasets, highly competitive with CNNs.
