Strategies for improving EBM learning beyond the standard framework (e.g. [1]}, [2]}, [3]}) can broadly be divided into methods that focus on the initialization of MCMC samples [1]}, [5]}, [6]} and methods that focus on the ML learning objective [7]}. Some works explore both [8]}, [9]}. Our novel learning methods focus on MCMC initialization, and we retain the standard ML objective and use conventional network architectures. We introduce three new MCMC initialization strategies which are tailored to the three different trajectories lengths we explore. During training we exclusively use shortrun MCMC to ensure computational feasibility. Learning models with midrun and longrun trajectories is accomplished by simulating longer trajectories via well-chosen initialization and optimizer annealing. Our initialization strategies are able to significantly improve the state-of-the-art across the tasks we investigate. We summarize our contributions below.
