The strengths and limitations of these methods are complementary. Persistent initialization can reliably provide a diverse set of initial images that represent prior samples of several different model snapshots. Even if a single EBM update is biased to a certain image defect (e.g. MCMC samples are blue-tinged or too bright), samples across previous iterations have the correct diversity on average. On the other hand, the generator can quickly propose in-distribution states for rejuvenation instead of relying on out-of-distribution noise samples or aggressively augmented persistent samples. Furthermore, training the generator in tandem with the EBM provides an efficient way of generating new samples from scratch after training that match the samples used during training. In contrast, models trained with persistent initialization often cannot produce good samples after training without an in-distribution initialization [1]}. Learning a generator and EBM jointly enables self-contained and reproducible FID implementation to evaluate the learned model.
