Generative Adversarial Networks (GANs) have been shown to produce realistically looking synthetic images with remarkable success, yet their performance seems less impressive when the training set is highly diverse. In order to provide a better fit to the target data distribution when the dataset includes many different classes, we propose a variant of the basic GAN model, a Multi-Modal Gaussian-Mixture GAN (GM-GAN), where the probability distribution over the latent space is a mixture of Gaussians. We also propose a supervised variant which is capable of conditional sample synthesis. In order to evaluate the model's performance, we propose a new scoring method which separately takes into account two (typically conflicting) measures - diversity vs. quality of the generated data.   Through a series of experiments, using both synthetic and real-world datasets, we quantitatively show that GM-GANs outperform baselines, both when evaluated using the commonly used Inception Score, and when evaluated using our own alternative scoring method. In addition, we qualitatively demonstrate how the unsupervised variant of GM-GAN tends to map latent vectors sampled from different Gaussians in the latent space to samples of different classes in the data space. We show how this phenomenon can be exploited for the task of unsupervised clustering, and provide quantitative evaluation showing the superiority of our method for the unsupervised clustering of image datasets. Finally, we demonstrate a feature which further sets our model apart from other GAN models: the option to control the quality-diversity trade-off by altering, post-training, the probability distribution of the latent space. This allows one to sample higher quality and lower diversity samples, or vice versa, according to one's needs. Generative models have long been an important and active field of research in machine-learning. Generative Adversarial Networks BID6 include a family of methods for learning generative models where the computational approach is based on game theory. The goal of a GAN is to learn a Generator (G) capable of generating samples from the data distribution (p X ), by converting latent vectors from a lower-dimension latent space (Z) to samples in a higher-dimension data space (X ). Usually, latent vectors are sampled from Z using the uniform or the normal distribution.In order to train G, a Discriminator (D) is trained to distinguish real training samples from fake samples generated by G. Thus D returns a value D(x) ∈ [0, 1] which can be interpreted as the probability that the input sample (x) is a real sample from the data distribution. In this configuration, G is trained to obstruct D by generating samples which better resemble the real training samples, while D is continuously trained to tell apart real from fake samples. Crucially, G has no direct access to real samples from the training set, as it learns solely through its interaction with D. Both D and G are implemented by deep differentiable networks, typically consisting of multiple convolutional and fully-connected layers. They may be alternately trained using Stochastic Gradient Descent.In the short period of time since the introduction of the GAN model, many different enhancement methods and training variants have been suggested to improve their performance (see brief review below). Despite these efforts, often a large proportion of the generated samples is, arguably, not satisfactorily realistic. In some cases the generated sample does not resemble any of the real samples from the training set, and human observers find it difficult to classify synthetically generated samples to one of the classes which compose the training set (see illustration in FIG0 ).Figure 1: Images generated by different GANs trained on MNIST (top row), CelebA (middle row) and STL-10 (bottom row). Red square mark images of, arguably, low quality (best seen in color).This problem worsens with the increased complexity of the training set, and specifically when the training set is characterized by large inter-class and intra-class diversity. In this work we focus on this problem, aiming to improve the performance of GANs when the training dataset has large inter-class and intra-class diversity.Related Work. In an attempt to improve the performance of the original GAN model, many variants and extensions have been proposed in the past few years. These include architectural changes to G and D as in BID26 , modifications to the loss function as in BID20 ; BID7 , or the introduction of supervision into the training setting as in BID22 ; BID24 . Another branch of related work, which is perhaps more closely related to our work, involves the learning of a meaningfully structured latent space. Thus Info-GAN decomposes the input noise into an incompressible source and a "latent code", Adversarial Auto-Encoders BID19 employ GANs to perform variational inference, and BID16 combine a Variational Auto-Encoder with a Generative Adversarial Network (see Appendix A for a more comprehensive description).Our Approach. Although modifications to the structure of the latent space have been investigated before as described above, the significance of the probability distribution used for sampling latent vectors was rarely investigated. A common practice today is to use a standard normal (e.g. N (0, I)) or uniform (e.g. U [0, 1]) probability distribution when sampling latent vectors from the latent space. We wish to challenge this common practice, and investigate the beneficial effects of modifying the distribution used to sample latent vectors in accordance with properties of the target dataset.Specifically, many datasets, especially those of natural images, are quite diverse, with high interclass and intra-class variability. At the same time, the representations of these datasets usually span high dimensional spaces, which naturally makes them very sparse. Intuitively, this implies that the underlying data distribution, which we try to learn using a GAN, is also sparse, i.e. it mostly consists of low-density areas with relatively few areas of high-density.We propose to incorporate this prior-knowledge into the model, by sampling latent vectors using a multi-modal probability distribution which better matches these characteristics of the data space distribution. It is important to emphasize that this architectural modification is orthogonal to, and can be used in conjunction with, other architectural improvements such as those reviewed above (see for instance FIG6 in Appendix D.) Supervision can be incorporated into this model by adding correspondence (not necessarily injective) between labels and mixture components.The rest of this paper is organized as follows: In Section 2 we describe the family of GM-GAN models. In Section 3 we offer an alternative method which focuses on measuring the trade-off between sample quality and diversity of generative models. In Section 4 we empirically evaluate our proposed model using various diverse datasets, showing that GM-GANs outperform the corresponding baseline methods with uni-modal distribution in the latent space. In Section 5 we describe a method for clustering datasets using GM-GANs, and provide qualitative and quantitative evaluation using various datasets of real images.
Multi modal Guassian distribution of latent space in GAN models improves performance and allows to trade-off quality vs. diversity
{"src": "scitldr-AIC_train", "paper_id": "rkgWBi09Ym"}