We propose a novel, projection based way to incorporate the conditional information into the discriminator of GANs that respects the role of the conditional information in the underlining probabilistic model. 
 This approach is in contrast with most frameworks of conditional GANs used in application today, which use the conditional information by concatenating the (embedded) conditional vector to the feature vectors. 
 With this modification, we were able to significantly improve the quality of the class conditional image generation on ILSVRC2012 (ImageNet) dataset from the current state-of-the-art result, and we achieved this with a single pair of a discriminator and a generator. 
 We were also able to extend the application to super-resolution and succeeded in producing highly discriminative super-resolution images. 
 This new structure also enabled high quality category transformation based on parametric functional transformation of conditional batch normalization layers in the generator. Generative Adversarial Networks (GANs) BID5 are a framework to construct a generative model that can mimic the target distribution, and in recent years it has given birth to arrays of state-of-the-art algorithms of generative models on image domain BID23 Ledig et al., 2017; BID30 BID20 . The most distinctive feature of GANs is the discriminator D(x) that evaluates the divergence between the current generative distribution p G (x) and the target distribution q(x) BID5 BID16 . The algorithm of GANs trains the generator model by iteratively training the discriminator and generator in turn, with the discriminator acting as an increasingly meticulous critic of the current generator.Conditional GANs (cGANs) are a type of GANs that use conditional information BID13 for the discriminator and generator, and they have been drawing attention as a promising tool for class conditional image generation BID17 , the generation of the images from text BID20 BID30 , and image to image translation BID10 BID31 . Unlike in standard GANs, the discriminator of cGANs discriminates between the generator distribution and the target distribution on the set of the pairs of generated samples x and its intended conditional variable y. To the authors' knowledge, most frameworks of discriminators in cGANs at the time of writing feeds the pair the conditional information y into the discriminator by naively concatenating (embedded) y to the input or to the feature vector at some middle layer BID13 BID2 BID20 BID30 BID18 BID22 BID3 BID24 . We would like to however, take into account the structure of the assumed conditional probabilistic models underlined by the structure of the discriminator, which is a function that measures the information theoretic distance between the generative distribution and the target distribution.By construction, any assumption about the form of the distribution would act as a regularization on the choice of the discriminator. In this paper, we propose a specific form of the discriminator, a form motivated by a probabilistic model in which the distribution of the conditional variable y given x is discrete or uni-modal continuous distributions. This model assumption is in fact common in many real world applications, including class-conditional image generation and super-resolution.As we will explain in the next section, adhering to this assumption will give rise to a structure of the discriminator that requires us to take an inner product between the embedded condition vector y and the feature vector (Figure 1d ). With this modification, we were able to significantly improve the quality of the class conditional image generation on 1000-class ILSVRC2012 dataset BID21 with a single pair of a discriminator and generator (see the generated examples in Figure 2 ). Also, when we applied our model of cGANs to a super-resolution task, we were able to produce high quality super-resolution images that are more discriminative in terms of the accuracy of the label classifier than the cGANs based on concatenation, as well as the bilinear and the bicubic method. Any specification on the form of the discriminator imposes a regularity condition for the choice for the generator distribution and the target distribution. In this research, we proposed a model for the discriminator of cGANs that is motivated by a commonly occurring family of probabilistic models. This simple modification was able to significantly improve the performance of the trained generator on conditional image generation task and super-resolution task. The result presented in this paper is strongly suggestive of the importance of the choice of the form of the discriminator and the design A RESULTS OF CLASS CONDITIONAL IMAGE GENERATION ON CIFAR-10 AND CIFAR-100As a preliminary experiment, we compared the performance of conditional image generation on CIFAR-10 and CIFAR-100 3. For the discriminator and the generator, we reused the same architecture used in BID14 for the task on CIFAR-10. For the adversarial objective functions, we used (9), and trained both machine learners with the same optimizer with same hyper parameters we used in Section 5. For our projection model, we added the projection layer to the discriminator in the same way we did in the ImageNet experiment (before the last linear layer). Our projection model achieved better performance than other methods on both CIFAR-10 and CIFAR-100. Concatenation at hidden layer (hidden concat) was performed on the output of second ResBlock of the discriminator. We tested hidden concat as a comparative method in our main experiments on ImageNet, because the concatenation at hidden layer performed better than the concatenation at the input layer (input concat) when the number of classes was large (CIFAR-100).To explore how the hyper-parameters affect the performance of our proposed architecture, we conducted hyper-parameter search on CIFAR-100 about the Adam hyper-parameters (learning rate α and 1st order momentum β 1 ) for both our proposed architecture and the baselines. Namely , we varied each one of these parameters while keeping the other constant, and reported the inception scores for all methods including several versions of concat architectures to compare. We tested with concat module introduced at (a) input layer, (b) hidden layer, and at (c) output layer. As we can see in Figure 11 , our projection architecture excelled over all other architectures for all choice of the parameters, and achieved the inception score of 9.53. Meanwhile , concat architectures were able to achieve all 8.82 at most. The best concat model in term of the inception score on CIFAR-100 was the hidden concat with α = 0.0002 and β 1 = 0, which turns out to be the very choice of the parameters we picked for our ImageNet experiment. In this experiment, we followed the footsteps of Plug and Play Generative model (PPGNs) BID15 and augmented the original generator loss with an additional auxiliary classifier loss. In particular , we used the losses given by : DISPLAYFORM0 wherep pre (y|x) is the fixed model pretrained for ILSVRC2012 classification task. For the actual experiment, we trained the generator with the original adversarial loss for the first 400K updates, and used the augmented loss for the last 50K updates. For the learning rate hyper parameter, we adopted the same values as other experiments we described above. For the pretrained classifier, we used ResNet50 model used in BID7 . FIG0 compares the results generated by vanilla objective function and the results generated by the augmented objective function. As we can see in TAB2 , we were able to significantly outperform PPGNs in terms of inception score. However, note that the images generated here are images that are easy to classify. The method with auxiliary classifier loss seems effective in improving the visual appearance, but not in training faithful generative model.
We propose a novel, projection based way to incorporate the conditional information into the discriminator of GANs that respects the role of the conditional information in the underlining probabilistic model.
{"src": "scitldr-AIC_train", "paper_id": "ByS1VpgRZ"}