Generative Adversarial Networks in Computer Vision: A Survey and Taxonomy <s> Least Square GAN (LSGAN) <s> We propose a new framework for estimating generative models via an adversarial process, in which we simultaneously train two models: a generative model G that captures the data distribution, and a discriminative model D that estimates the probability that a sample came from the training data rather than G. The training procedure for G is to maximize the probability of D making a mistake. This framework corresponds to a minimax two-player game. In the space of arbitrary functions G and D, a unique solution exists, with G recovering the training data distribution and D equal to ½ everywhere. In the case where G and D are defined by multilayer perceptrons, the entire system can be trained with backpropagation. There is no need for any Markov chains or unrolled approximate inference networks during either training or generation of samples. Experiments demonstrate the potential of the framework through qualitative and quantitative evaluation of the generated samples. <s> BIB001 </s> Generative Adversarial Networks in Computer Vision: A Survey and Taxonomy <s> Least Square GAN (LSGAN) <s> Unsupervised learning with generative adversarial networks (GANs) has proven hugely successful. Regular GANs hypothesize the discriminator as a classifier with the sigmoid cross entropy loss function. However, we found that this loss function may lead to the vanishing gradients problem during the learning process. To overcome such a problem, we propose in this paper the Least Squares Generative Adversarial Networks (LSGANs) which adopt the least squares loss function for the discriminator. We show that minimizing the objective function of LSGAN yields minimizing the Pearson X2 divergence. There are two benefits of LSGANs over regular GANs. First, LSGANs are able to generate higher quality images than regular GANs. Second, LSGANs perform more stable during the learning process. We evaluate LSGANs on LSUN and CIFAR-10 datasets and the experimental results show that the images generated by LSGANs are of better quality than the ones generated by regular GANs. We also conduct two comparison experiments between LSGANs and regular GANs to illustrate the stability of LSGANs. <s> BIB002
The LSGAN is a new approach proposed in BIB002 to remedy the vanishing gradient problem for G from the perspective of the decision boundary determined by the discriminator. This work argues that the decision boundary for D of the original GAN penalizes very small error to update G for those generated samples that are far away from the decision boundary. The author proposes using a least square loss for D instead of sigmoid cross entropy loss stated in the original paper BIB001 . The proposed loss function is defined as where a is the label for the generated samples, b is the label for the real samples and c is the value that G wants D to believe for the generated samples. This modified change has two benefits: (1) The new decision boundary made by D penalizes large error arising from those generated samples that are far away from the decision boundary, which pushes those "bad" generated samples towards the decision boundary. This is beneficial in terms of generating improved image quality; (2) Penalizing the generated samples far away from the decision boundary can provide more gradient when updating the G, which remedies the vanishing gradient problems for training G. Figure 12 demonstrates the comparison of decision boundaries for LSGAN and the original GAN. The decision boundaries for D that have been trained by the original sigmoid cross entropy loss and the proposed least square loss are different. The work BIB002 has proven that the optimization of LSGAN is equivalent to minimizing the Pearson χ 2 divergence between p r + p g and 2p g when a,b and c satisfy the condition of b−c = 1 and b−a = 2. Similar to WGAN, D here behaves as regression and the sigmoid is also removed.