We propose studying GAN training dynamics as regret minimization, which is in contrast to the popular view that there is consistent minimization of a divergence between real and generated distributions. We analyze the convergence of GAN training from this new point of view to understand why mode collapse happens. We hypothesize the existence of undesirable local equilibria in this non-convex game to be responsible for mode collapse. We observe that these local equilibria often exhibit sharp gradients of the discriminator function around some real data points. We demonstrate that these degenerate local equilibria can be avoided with a gradient penalty scheme called DRAGAN. We show that DRAGAN enables faster training, achieves improved stability with fewer mode collapses, and leads to generator networks with better modeling performance across a variety of architectures and objective functions. Generative modeling involves taking a set of samples drawn from an unknown data generating distribution P real and finding an estimate P model that closely resembles it. Generative adversarial networks (GAN) BID6 ) is a powerful framework used for fitting implicit generative models. The basic setup consists of two networks, the generator and the discriminator, playing against each other in a repeated zero-sum game setting. The goal here is to reach an equilibrium where P real , P model are close, and the alternating gradient updates procedure (AGD) is used to achieve this. However, this process is highly unstable and often results in mode collapse BID7 . This calls for an deeper investigation into training dynamics of GANs.In this paper, we propose studying GAN training dynamics as a repeated game in which both the players are using no-regret algorithms BID2 and discuss how AGD 1 falls under this paradigm. In contrast, much of the theory BID6 BID0 and recent developments BID15 BID8 are based on the unrealistic assumption that the discriminator is playing optimally (in the function space) at each step and as a result, there is consistent minimization of a divergence between real and generated distributions. This corresponds to at least one player using the best-response algorithm (in the function space), and the resulting game dynamics can be completely different in both these cases BID14 . Thus, there is a clear disconnect between theoretical arguments used as motivation in recent literature and what actually happens in practice.We would like to point out that the latter view can still be useful for reasoning about the asymptotic equilibrium situation but we argue that regret minimization is the more appropriate way to think about GAN training dynamics. So, we analyze the convergence of GAN training from this new point of view to understand why mode collapse happens. We start with a short analysis of the artificial convex-concave case of the GAN game in section 2.2. This setting has a unique solution and guaranteed convergence (of averaged iterates) using no-regret algorithms can be shown with standard arguments from game theory literature. Here, we make explicit, the critical (previously not widely known) connection between AGD used in GAN training and regret minimization. This immediately yields a novel proof for the asymptotic convergence of GAN training, in the non-parametric limit. Prior to our work, such a result BID6 ) required a strong assumption that the discriminator is optimal at each step.However, these convergence results do not hold when the game objective function is non-convex, which is the practical case when deep neural networks are used. In non-convex games, global regret minimization and equilibrium computation are computationally hard in general. Recent gametheoretic literature indicates that AGD can end up cycling BID11 or converging to a (potentially bad) local equilibrium, under some conditions BID9 . We hypothesize these to be the reasons for cycling and mode collapse observed during GAN training, respectively (section 2.3). In this work, we do not explore the cycling issue but focus our attention on the mode collapse problem. In contrast to our hypothesis, the prevalent view of mode collapse and instability BID0 is that it results from attempting to minimize a strong divergence during training. However, as we argued earlier, GAN training with AGD does not consistently minimize a divergence and therefore, such a theory is not suitable to discuss convergence or to address the stability issue.Next, if mode collapse is indeed the result of an undesirable local equilibrium, a natural question then is how we can avoid it? We make a simple observation that, in the GAN game, mode collapse situations are often accompanied by sharp gradients of the discriminator function around some real data points (section 2.4). Therefore, a simple strategy to mitigate mode collapse is to regularize the discriminator so as to constrain its gradients in the ambient data space. We demonstrate that this improves the stability using a toy experiment with one hidden layer neural networks. This gives rise to a new explanation for why WGAN and gradient penalties might be improving the stability of GAN training -they are mitigating the mode collapse problem by keeping the gradients of the discriminator function small in data space. From this motivation, we propose a training algorithm involving a novel gradient penalty scheme called DRAGAN (Deep Regret Analytic Generative Adversarial Networks) which enables faster training, achieves improved stability and modeling performance (over WGAN-GP BID8 which is the state-of-the-art stable training procedure) across a variety of architectures and objective functions.Below, we provide a short literature review. Several recent works focus on stabilizing the training of GANs. While some solutions BID17 BID18 require the usage of specific architectures (or) modeling objectives, some BID4 BID20 significantly deviate from the original GAN framework. Other promising works in this direction BID12 BID16 BID8 ) impose a significant computational overhead. Thus, a fast and versatile method for consistent stable training of GANs is still missing in the literature. Our work is aimed at addressing this.To summarize, our contributions are as follows:• We propose a new way of reasoning about the GAN training dynamics -by viewing AGD as regret minimization.• We provide a novel proof for the asymptotic convergence of GAN training in the nonparametric limit and it does not require the discriminator to be optimal at each step.• We discuss how AGD can converge to a potentially bad local equilibrium in non-convex games and hypothesize this to be responsible for mode collapse during GAN training.• We characterize mode collapse situations with sharp gradients of the discriminator function around some real data points.• A novel gradient penalty scheme called DRAGAN is introduced based on this observation and we demonstrate that it mitigates the mode collapse issue. In this paper, we propose to study GAN training process as regret minimization, which is in contrast to the popular view that there is consistent minimization of a divergence between real and generated distributions. We analyze the convergence of GAN training from this new point of view and hypothesize that mode collapse occurs due to the existence of undesirable local equilibria. A simple observation is made about how the mode collapse situation often exhibits sharp gradients of the discriminator function around some real data points. This characterization partly explains the workings of previously proposed WGAN and gradient penalties, and motivates our novel penalty scheme. We show evidence of improved stability using DRAGAN and the resulting improvements in modeling performance across a variety of settings. We leave it to future works to explore our ideas in more depth and come up with improved training algorithms.
Analysis of convergence and mode collapse by studying GAN training process as regret minimization
{"src": "scitldr-AIC_train", "paper_id": "ryepFJbA-"}