A Survey of Optimization Methods from a Machine Learning Perspective <s> E. Optimization in Markov Chain Monte Carlo <s> Abstract We present a new method for the numerical simulation of lattice field theory. A hybrid (molecular dynamics/Langevin) algorithm is used to guide a Monte Carlo simulation. There are no discretization errors even for large step sizes. The method is especially efficient for systems such as quantum chromodynamics which contain fermionic degrees of freedom. Detailed results are presented for four-dimensional compact quantum electrodynamics including the dynamical effects of electrons. <s> BIB001 </s> A Survey of Optimization Methods from a Machine Learning Perspective <s> E. Optimization in Markov Chain Monte Carlo <s> Hamiltonian Monte Carlo (HMC) sampling methods provide a mechanism for defining distant proposals with high acceptance probabilities in a Metropolis-Hastings framework, enabling more efficient exploration of the state space than standard random-walk proposals. The popularity of such methods has grown significantly in recent years. However, a limitation of HMC methods is the required gradient computation for simulation of the Hamiltonian dynamical system-such computation is infeasible in problems involving a large sample size or streaming data. Instead, we must rely on a noisy gradient estimate computed from a subset of the data. In this paper, we explore the properties of such a stochastic gradient HMC approach. Surprisingly, the natural implementation of the stochastic approximation can be arbitrarily bad. To address this problem we introduce a variant that uses second-order Langevin dynamics with a friction term that counteracts the effects of the noisy gradient, maintaining the desired target distribution as the invariant distribution. Results on simulated data validate our theory. We also provide an application of our methods to a classification task using neural networks and to online Bayesian matrix factorization. <s> BIB002 </s> A Survey of Optimization Methods from a Machine Learning Perspective <s> E. Optimization in Markov Chain Monte Carlo <s> Leveraging the coherent exploration of Hamiltonian flow, Hamiltonian Monte Carlo produces computationally efficient Monte Carlo estimators, even with respect to complex and highdimensional target distributions. When confronted with data-intensive applications, however, the algorithm may be too expensive to implement, leaving us to consider the utility of approximations such as data subsampling. In this paper I demonstrate how data subsampling fundamentally compromises the scalability of Hamiltonian Monte Carlo. <s> BIB003
Markov chain Monte Carlo (MCMC) is a class of sampling algorithms to simulate complex distributions that are difficult to sample directly. It is a practical tool for Bayesian posterior inference. The traditional and common MCMC algorithms include Gibbs sampling, slice sampling, Hamiltonian Monte Carlo (HMC) BIB001 ,  , Reimann manifold variants  , and so on. These sampling methods are limited by the computational cost and are difficult to extend to large-scale data. This section takes HMC as an example to introduce the optimization in MCMC. The bottleneck of the HMC is that the gradient calculation is costly on large datasets. We first introduce the derivation of HMC. Consider the random variable θ, which can be sampled from the posterior distribution, where D is the set of observations, and U is the potential energy function with the following formula: In HMC BIB001 , an independent auxiliary momentum variable r is introduced from Hamiltonian dynamic. The Hamiltonian function and the joint distribution of θ and r are described by where M denotes the mass matrix, and K(r) is the kinetic energy function. The process of HMC sampling is derived by simulating the Hamiltoonian dynamic system, Hamiltonian dynamic describes the continuous motion of a particle. Hamiltonian equations are numerically approximated by the discretized leapfrog integrator for practical simulating BIB001 . The update equations are as follows BIB001 : In the case of large datasets, the gradient of U (θ) needs to be calculated on the entire data set in each leapfrog iteration. In order to improve the efficiency, the stochastic gradient method was used to calculate ∇U (θ) with a minibatchD sampled uniformly from D, which reduces the cost of calculation BIB002 . However, the gradient calculated in a minibatch instead of the full dataset will cause noise. According to the central limit theorem, this noisy gradient can be approximated as where gradient noise obeys normal distribution whose covariance is V (θ). If we replace ∇U (θ) by ∇Ũ (θ) directly, the Hamiltonian dynamics will be changed as where B(θ) = 1 2 ǫV (θ) is the diffusion matrix BIB002 . Since the discretization of the dynamical system introduces noise, the Metropolis-Hastings (MH) correction step should be done after the leapfrog step. These MH steps require expensive calculations overall data in each iteration. Beyond that, there is an incorrect stationary distribution BIB003 in the stochastic gradient variant of HMC. Thus, Hamiltonian dynamic was further modified, which minimizes the effect of the additional noise, achieves the invariant distribution and eliminates MH steps BIB002 . Specifically, a friction term is added to the dynamical process of momentum update: The introduced friction term is helpful for decreasing total energy H(θ, r) and weakening the effects of noise in the momentum update phase. The dynamical system is also the type of second-order Langevin dynamics with friction in physics, which can explore efficiently and counteract the effect of the noisy gradients BIB002 and thus no MH correction is required. This second-order Langevin dynamic MCMC method, called SGHMC, is used to deal with sampling problems on large data sets BIB002 ,  . Moreover, HMC is highly sensitive to hyper-parameters, such as the path length (step number) L and the step size ǫ. If the hyper-parameters are not set properly, the efficiency of the HMC will drop dramatically. There are some methods to optimize these two hyper-parameters instead of manually setting them.