The major disadvantage of MCMC methods for deep learning is that they require a long simulation for constructing a good approximation of the posterior distribution, incurring heavy computational costs. However, recent progress in incorporating gradient-based proposals into MCMC sampling has led to the development of Hamiltonian and Langevin MCMC methods [1]}, [2]}. They have been demonstrated to be useful for Bayesian neural learning [3]}, [4]}. Combining them with parallel computing resulted in further improvements [3]}, perhaps allowing them to be used for deep learning methods such as autoencoders, and in particular deep autoencoders that feature thousands of weights or parameters.
