Inferring the most likely configuration for a subset of variables of a joint distribution given the remaining ones – which we refer to as co-generation – is an important challenge that is computationally demanding for all but the simplest settings. This task has received a considerable amount of attention, particularly for classical ways of modeling distributions like structured prediction. In contrast, almost nothing is known about this task when considering recently proposed techniques for modeling high-dimensional distributions, particularly generative adversarial nets (GANs). Therefore, in this paper, we study the occurring challenges for co-generation with GANs. To address those challenges we develop an annealed importance sampling (AIS) based Hamiltonian Monte Carlo (HMC) co-generation algorithm. The presented approach significantly outperforms classical gradient-based methods on synthetic data and on CelebA. While generative adversarial nets (GANs) [6] and variational auto-encoders (VAEs) [8] model a joint probability distribution which implicitly captures the correlations between multiple parts of the output, e.g., pixels in an image, and while those methods permit easy sampling from the entire output space domain, it remains an open question how to sample from part of the domain given the remainder? We refer to this task as co-generation. To enable co-generation for a domain unknown at training time, for GANs, optimization based algorithms have been proposed [15, 10] . Intuitively, they aim at finding that latent sample which accurately matches the observed part. However, successful training of the GAN leads to an increasingly ragged energy landscape, making the search for an appropriate latent variable via backpropagation through the generator harder and harder until it eventually fails. To deal with this ragged energy landscape during co-generation, we develop a method using an annealed importance sampling (AIS) [11] based Hamiltonian Monte Carlo (HMC) algorithm [4, 12] , which is typically used to estimate (ratios of) the partition function [14, 13] . Rather than focus on the partition function, the proposed approach leverages the benefits of AIS, i.e., gradually annealing a complex probability distribution, and HMC, i.e., avoiding a localized random walk. We evaluate the proposed approach on synthetic data and imaging data (CelebA), showing compelling results via MSE and MSSIM metrics. For more details and results please see our main conference paper [5] . We propose a co-generation approach, i.e., we complete partially given input data, using annealed importance sampling (AIS) based on the Hamiltonian Monte Carlo (HMC). Different from classical optimization based methods, specifically GD, which get easily trapped in local optima when solving this task, the proposed approach is much more robust. Importantly, the method is able to traverse large energy barriers that occur when training generative adversarial nets. Its robustness is due to AIS gradually annealing a probability distribution and HMC avoiding localized walks. We show additional results for real data experiments. We observe our proposed algorithm to recover masked images more accurately than baselines and to generate better high-resolution images given low-resolution images. We show masked CelebA (Fig. 5) and LSUN (Fig. 6 ) recovery results for baselines and our method, given a Progressive GAN generator. Note that our algorithm is pretty robust to the position of the z initialization, since the generated results are consistent in Fig. 5 . (a) (b) (c)
Using annealed importance sampling on the co-generation problem. 
{"src": "scitldr-AIC_train", "paper_id": "rkeahX3qLr"}