Training generative adversarial networks requires balancing of delicate adversarial dynamics. Even with careful tuning, training may diverge or end up in a bad equilibrium with dropped modes. In this work, we introduce a new form of latent optimisation inspired by the CS-GAN and show that it improves adversarial dynamics by enhancing interactions between the discriminator and the generator. We develop supporting theoretical analysis from the perspectives of differentiable games and stochastic approximation. Our experiments demonstrate that latent optimisation can significantly improve GAN training, obtaining state-of-the-art performance for the ImageNet (128 x 128) dataset. Our model achieves an Inception Score (IS) of 148 and an Frechet Inception Distance (FID) of 3.4, an improvement of 17% and 32% in IS and FID respectively, compared with the baseline BigGAN-deep model with the same architecture and number of parameters. Generative Adversarial Nets (GANs) are implicit generative models that can be trained to match a given data distribution. GANs were originally proposed and demonstrated for images by Goodfellow et al. (2014) . As the field of generative modelling has advanced, GANs have remained at the frontier, generating high-fidelity images at large scale (Brock et al., 2018) . However, despite growing insights into the dynamics of GAN training, most recent advances in large-scale image generation come from architectural improvements (Radford et al., 2015; Zhang et al., 2019) , or regularisation focusing on particular parts of the model (Miyato et al., 2018; Miyato & Koyama, 2018) . Inspired by the compressed sensing GAN (CS-GAN; Wu et al., 2019) , we further exploit the benefit of latent optimisation in adversarial games using natural gradient descent to optimise the latent variable z at each step of training, presenting a scalable and easy to implement approach to improve the dynamical interaction between the discriminator and the generator. For clarity, we unify these approaches as latent optimised GANs (LOGAN). To summarise our contributions: 1. We present a novel analysis of latent optimisation in GANs from the perspective of differentiable games and stochastic approximation (Balduzzi et al., 2018; Heusel et al., 2017) , arguing that latent optimisation can improve the dynamics of adversarial training. 2. Motivated by this analysis, we improve latent optimisation by taking advantage of efficient second-order updates. 3. Our algorithm improves the state-of-the-art BigGAN-deep model (Brock et al., 2018) by a significant margin, without introducing any architectural change or additional parameters, resulting in higher quality images and more diverse samples (Figure 1 and 2). In this work we present the LOGAN model which significantly improves the state-of-the-art on large scale GAN training for image generation by online optimising the latent source z. Our results illustrate improvements in quantitative evaluation and samples with higher quality and diversity. Moreover, our analysis suggests that LOGAN fundamentally improves adversarial training dynamics. We therefore expect our method to be useful in other tasks that involve adversarial training, including representation learning and inference (Donahue et al., 2017; Dumoulin et al., 2017 ), text generation (Zhang et al., 2019) , style learning (Zhu et al., 2017; Karras et al., 2019) , audio generation and video generation (Vondrick et al., 2016; Clark et al., 2019 A ADDITIONAL SAMPLES AND RESULTS Figure 6 and 7 provide additional samples, organised similarly as in Figure 1 and 2. Figure 8 shows additional truncation curves.
Latent optimisation improves adversarial training dynamics. We present both theoretical analysis and state-of-the-art image generation with ImageNet 128x128.
{"src": "scitldr-AIC_train", "paper_id": "rJeU_1SFvr"}