Class labels have been empirically shown useful in improving the sample quality of generative adversarial nets (GANs). In this paper, we mathematically study the properties of the current variants of GANs that make use of class label information. With class aware gradient and cross-entropy decomposition, we reveal how class labels and associated losses influence GAN's training. Based on that, we propose Activation Maximization Generative Adversarial Networks (AM-GAN) as an advanced solution. Comprehensive experiments have been conducted to validate our analysis and evaluate the effectiveness of our solution, where AM-GAN outperforms other strong baselines and achieves state-of-the-art Inception Score (8.91) on CIFAR-10. In addition, we demonstrate that, with the Inception ImageNet classifier, Inception Score mainly tracks the diversity of the generator, and there is, however, no reliable evidence that it can reflect the true sample quality. We thus propose a new metric, called AM Score, to provide more accurate estimation on the sample quality. Our proposed model also outperforms the baseline methods in the new metric. Generative adversarial nets (GANs) BID7 as a new way for learning generative models, has recently shown promising results in various challenging tasks, such as realistic image generation BID17 BID26 BID9 , conditional image generation BID12 BID2 BID13 , image manipulation ) and text generation BID25 .Despite the great success, it is still challenging for the current GAN models to produce convincing samples when trained on datasets with high variability, even for image generation with low resolution, e.g., CIFAR-10. Meanwhile , people have empirically found taking advantages of class labels can significantly improve the sample quality.There are three typical GAN models that make use of the label information: CatGAN BID20 builds the discriminator as a multi-class classifier; LabelGAN BID19 extends the discriminator with one extra class for the generated samples; AC-GAN BID18 jointly trains the real-fake discriminator and an auxiliary classifier for the specific real classes. By taking the class labels into account, these GAN models show improved generation quality and stability. However, the mechanisms behind them have not been fully explored BID6 .In this paper , we mathematically study GAN models with the consideration of class labels. We derive the gradient of the generator's loss w.r.t. class logits in the discriminator, named as class-aware gradient, for LabelGAN BID19 and further show its gradient tends to guide each generated sample towards being one of the specific real classes. Moreover, we show that AC-GAN BID18 can be viewed as a GAN model with hierarchical class discriminator. Based on the analysis, we reveal some potential issues in the previous methods and accordingly propose a new method to resolve these issues.Specifically, we argue that a model with explicit target class would provide clearer gradient guidance to the generator than an implicit target class model like that in BID19 . Comparing with BID18 , we show that introducing the specific real class logits by replacing the overall real class logit in the discriminator usually works better than simply training an auxiliary classifier. We argue that, in BID18 , adversarial training is missing in the auxiliary classifier, which would make the model more likely to suffer mode collapse and produce low quality samples. We also experimentally find that predefined label tends to result in intra-class mode collapse and correspondingly propose dynamic labeling as a solution. The proposed model is named as Activation Maximization Generative Adversarial Networks (AM-GAN). We empirically study the effectiveness of AM-GAN with a set of controlled experiments and the results are consistent with our analysis and, note that, AM-GAN achieves the state-of-the-art Inception Score (8.91) on CIFAR-10.In addition, through the experiments, we find the commonly used metric needs further investigation. In our paper, we conduct a further study on the widely-used evaluation metric Inception Score BID19 and its extended metrics. We show that, with the Inception Model, Inception Score mainly tracks the diversity of generator, while there is no reliable evidence that it can measure the true sample quality. We thus propose a new metric, called AM Score, to provide more accurate estimation on the sample quality as its compensation. In terms of AM Score, our proposed method also outperforms other strong baseline methods.The rest of this paper is organized as follows. In Section 2, we introduce the notations and formulate the LabelGAN BID19 and AC-GAN * BID18 ) as our baselines. We then derive the class-aware gradient for LabelGAN, in Section 3, to reveal how class labels help its training. In Section 4, we reveal the overlaid-gradient problem of LabelGAN and propose AM-GAN as a new solution, where we also analyze the properties of AM-GAN and build its connections to related work. In Section 5, we introduce several important extensions, including the dynamic labeling as an alternative of predefined labeling (i.e., class condition), the activation maximization view and a technique for enhancing the AC-GAN * . We study Inception Score in Section 6 and accordingly propose a new metric AM Score. In Section 7, we empirically study AM-GAN and compare it to the baseline models with different metrics. Finally we conclude the paper and discuss the future work in Section 8. In this paper, we analyze current GAN models that incorporate class label information. Our analysis shows that: LabelGAN works as an implicit target class model, however it suffers from the overlaidgradient problem at the meantime, and explicit target class would solve this problem. We demonstrate that introducing the class logits in a non-hierarchical way, i.e., replacing the overall real class logit in the discriminator with the specific real class logits, usually works better than simply supplementing an auxiliary classifier, where we provide an activation maximization view for GAN training and highlight the importance of adversarial training. In addition, according to our experiments, predefined labeling tends to lead to intra-class mode collapsed, and we propose dynamic labeling as an alternative. Our extensive experiments on benchmarking datasets validate our analysis and demonstrate our proposed AM-GAN's superior performance against strong baselines. Moreover, we delve deep into the widelyused evaluation metric Inception Score, reveal that it mainly works as a diversity measurement. And we also propose AM Score as a compensation to more accurately estimate the sample quality.In this paper, we focus on the generator and its sample quality, while some related work focuses on the discriminator and semi-supervised learning. For future work, we would like to conduct empirical studies on discriminator learning and semi-supervised learning. We extend AM-GAN to unlabeled data in the Appendix C, where unsupervised and semi-supervised is accessible in the framework of AM-GAN. The classifier-based evaluation metric might encounter the problem related to adversarial samples, which requires further study. Combining AM-GAN with Integral Probability Metric based GAN models such as Wasserstein GAN could also be a promising direction since it is orthogonal to our work. DISPLAYFORM0 Label smoothing that avoiding extreme logits value was showed to be a good regularization BID21 . A general version of label smoothing could be: modifying the target probability of discriminator) BID19 proposed to use only one-side label smoothing. That is, to only apply label smoothing for real samples: λ 1 = 0 and λ 2 > 0. The reasoning of one-side label smoothing is applying label smoothing on fake samples will lead to fake mode on data distribution, which is too obscure. DISPLAYFORM1 We will next show the exact problems when applying label smoothing to fake samples along with the log(1−D r (x)) generator loss, in the view of gradient w.r.t. class logit, i.e., the class-aware gradient, and we will also show that the problem does not exist when using the − log(D r (x)) generator loss. DISPLAYFORM2 The log(1−D r (x)) generator loss with label smoothing in terms of cross-entropy is DISPLAYFORM3 with lemma 1, its negative gradient is DISPLAYFORM4 DISPLAYFORM5 Gradient vanishing is a well know training problem of GAN. Optimizing D r (x) towards 0 or 1 is also not what desired, because the discriminator is mapping real samples to the distribution with DISPLAYFORM6 The − log(D r (x)) generator loss with target [1−λ, λ] in terms of cross-entropy is DISPLAYFORM7 the negative gradient of which is DISPLAYFORM8 DISPLAYFORM9 Without label smooth λ, the − log(D r (x)) always * preserves the same gradient direction as log(1−D r (x)) though giving a difference gradient scale. We must note that non-zero gradient does not mean that the gradient is efficient or valid.The both-side label smoothed version has a strong connection to Least-Square GAN BID15 : with the fake logit fixed to zero, the discriminator maps real to α on the real logit and maps fake to β on the real logit, the generator in contrast tries to map fake sample to α. Their gradient on the logit are also similar.
Understand how class labels help GAN training. Propose a new evaluation metric for generative models. 
{"src": "scitldr-AIC_train", "paper_id": "HyyP33gAZ"}