Generative adversarial networks (GANs) provide a way to learn deep representations without extensively annotated training data. They achieve this by deriving backpropagation signals through a competitive process involving a pair of networks. The representations that can be learned by GANs may be used in a variety of applications, including image synthesis, semantic image editing, style transfer, image superresolution, and classification. The aim of this review article is to provide an overview of GANs for the signal processing community, drawing on familiar analogies and concepts where possible. In addition to identifying different methods for training and constructing GANs, we also point to remaining challenges in their theory and application.
What other comparisons can be made between GANs and the standard tools of signal processing? For PCA, ICA, Fourier and wavelet representations, the latent space of GANs is, by analogy, the coefficient space of what we commonly refer to as transform space. What sets GANs apart from these standard tools of signal processing is the level of complexity of the models that map vectors from latent space to image space. Because the generator networks contain non-linearities, and can be of almost arbitrary depth, this mapping -- as with many other deep learning approaches -- can be extraordinarily complex. With regard to deep image-based models, modern approaches to generative image modelling can be grouped into explicit density models and implicit density models. Explicit density models are either tractable (change of variables models, autoregressive models) or intractable (directed models trained with variational inference, undirected models trained using Markov chains). Implicit density models capture the statistical distribution of the data through a generative process which makes use of either ancestral sampling @cite_36 or Markov chain-based sampling. GANs fall into the directed implicit model category. A more detailed overview and relevant papers can be found in Ian Goodfellow's NIPS 2016 tutorial @cite_5 .
@cite_36 @cite_5
2953267151 2962775818
Recent work has shown how denoising and contractive autoencoders implicitly capture the structure of the data-generating density, in the case where the corruption noise is Gaussian, the reconstruction error is the squared error, and the data is continuous-valued. This has led to various proposals for sampling from this implicitly learned density function, using Langevin and Metropolis-Hastings MCMC. However, it remained unclear how to connect the training procedure of regularized auto-encoders to the implicit estimation of the underlying data-generating distribution when the data are discrete, or using other forms of corruption process and reconstruction errors. Another issue is the mathematical justification which is only valid in the limit of small corruption noise. We propose here a different attack on the problem, which deals with all these issues: arbitrary (but noisy enough) corruption, arbitrary reconstruction loss (seen as a log-likelihood), handling both discrete and continuous-valued variables, and removing the bias due to non-infinitesimal corruption noise (or non-infinitesimal contractive penalty). We introduce the "Energy-based Generative Adversarial Network" model (EBGAN) which views the discriminator as an energy function that attributes low energies to the regions near the data manifold and higher energies to other regions. Similar to the probabilistic GANs, a generator is seen as being trained to produce contrastive samples with minimal energies, while the discriminator is trained to assign high energies to these generated samples. Viewing the discriminator as an energy function allows to use a wide variety of architectures and loss functionals in addition to the usual binary classifier with logistic output. Among them, we show one instantiation of EBGAN framework as using an auto-encoder architecture, with the energy being the reconstruction error, in place of the discriminator. We show that this form of EBGAN exhibits more stable behavior than regular GANs during training. We also show that a single-scale architecture can be trained to generate high-resolution images.