
where the expectation is over model distribution \(p_{\mathbf {\theta }}(\mathbf {x}^{\prime })\) , sampling from which is challenging due to the intractable \(Z(\mathbf {\theta })\) . Therefore, MCMC and Gibbs sampling [1]} have been proposed previously to estimate the expectation efficiently. To speed up the mixing for effective sampling, recenlty Stochastic Gradient Langevin Dynamics (SGLD) [2]} has been used to train EBMs by exploiting the gradient information [3]}, [4]}, [5]}. Specifically, to sample from \(p_{\mathbf {\theta }}(\mathbf {x})\) , SGLD follows
\(&x_0\sim p_0(\mathbf {x}), \;&x_{i+1}=x_i-\frac{\alpha }{2} \frac{\partial E_{\mathbf {\theta }}(\mathbf {x}_i)}{\partial \mathbf {x}_i} + \alpha \epsilon , \;\; \epsilon \sim \mathcal {N} (0,1),\) 
