There are a number of different methods of MCMC initialization for EBM learning. Early Restricted Boltzmann Machines use data samples as the initial states in every iteration [1]}. Persistent initialization [2]}, [3]} maintains a bank of MCMC samples from previous model updates as the source of MCMC states in the current iteration. Persistent states can be rejuvenated with a certain probability instead of returning to the image bank. Potential sources for rejuvenation include noise samples [4]}, data augmentation applied to persistent states [5]}, data samples, or samples from a generator network. Cooperative learning initializes MCMC states from a generator learned in tandem with an EBM [6]}. The Multi-grid model [7]} initializes samples from lower resolution images. Noise initialization is explored by [8]}. The VAEBM [9]} initializes samples from a pretrained VAE generator. EBMs can learn stable shortrun synthesis from virtually any initialization distribution as long as the initialization distribution remains fairly consistent across training updates. EBM sampling can produce realistic images after training when the testing initialization closely matches the training initialization, while synthesis can fail for test initializations unseen during training. [10]}.
