A Survey on the New Generation of Deep Learning in Image Processing <s> B. GAN SERIES MODELS FOR IMAGE PROCESSING 1) GAN <s> Recently, generative adversarial networks U+0028 GANs U+0029 have become a research focus of artificial intelligence. Inspired by two-player zero-sum game, GANs comprise a generator and a discriminator, both trained under the adversarial learning idea. The goal of GANs is to estimate the potential distribution of real data samples and generate new samples from that distribution. Since their initiation, GANs have been widely studied due to their enormous prospect for applications, including image and vision computing, speech and language processing, etc. In this review paper, we summarize the state of the art of GANs and look into the future. Firstly, we survey GANs U+02BC proposal background, theoretic and implementation models, and application fields. Then, we discuss GANs U+02BC advantages and disadvantages, and their development trends. In particular, we investigate the relation between GANs and parallel intelligence, with the conclusion that GANs have a great potential in parallel systems research in terms of virtual-real interaction and integration. Clearly, GANs can provide substantial algorithmic support for parallel intelligence. <s> BIB001 </s> A Survey on the New Generation of Deep Learning in Image Processing <s> B. GAN SERIES MODELS FOR IMAGE PROCESSING 1) GAN <s> Although Generative Adversarial Networks (GANs) have shown remarkable success in various tasks, they still face challenges in generating high quality images. In this paper, we propose Stacked Generative Adversarial Networks (StackGAN) aiming at generating high-resolution photo-realistic images. First, we propose a two-stage generative adversarial network architecture, StackGAN-v1, for text-to-image synthesis. The Stage-I GAN sketches the primitive shape and colors of the object based on given text description, yielding low-resolution images. The Stage-II GAN takes Stage-I results and text descriptions as inputs, and generates high-resolution images with photo-realistic details. Second, an advanced multi-stage generative adversarial network architecture, StackGAN-v2, is proposed for both conditional and unconditional generative tasks. Our StackGAN-v2 consists of multiple generators and discriminators in a tree-like structure; images at multiple scales corresponding to the same scene are generated from different branches of the tree. StackGAN-v2 shows more stable training behavior than StackGAN-v1 by jointly approximating multiple distributions. Extensive experiments demonstrate that the proposed stacked generative adversarial networks significantly outperform other state-of-the-art methods in generating photo-realistic images. <s> BIB002
The GANs framework (see in Fig.7 ) includes a generative model that captures the sample distribution and a discriminative model that estimates the probability that a sample came from the training (real) sample rather than generating (fake) sample  , BIB001 . Based on game theory, one can training GANs requires finding a Nash equilibrium of a non-convex game with continuous, high-dimensional parameters. Generally, training GANs is a problematic issue in practice because of the instability of GANs learning. Several welldesigned networks have proposed to overcoming the problem of instability. In Fig.7 , We can use differentiable functions D and G to represent the discriminator and the generator, and their inputs are real sample x and random variables z, respectively. The purpose of D is to achieve the correct classification of sample source, while the purpose of G is to make the performance of generated sample G(z) consistent with the performance of the real sample. Therefore, the optimization of GAN can formulate as the following minimax problem: The adversarial optimization process improves the performance of D and G gradually. Eventually, when the discrimination ability of D has been improved to an excellent level but cannot discriminate against the sample source correctly, it is thought that the generator G has captured the distribution of a real sample. Since Ian J Goodfellow proposed GAN in 2014, many GAN variants have produced so far. Undoubtedly, GANs have solved many problems in generative models and inspired other artifical intelligence (AI) methods, but there are still some limitations BIB002 . For example, from a mathematical point of view, GANs adopt the adversarial learning idea, but the convergence of the model and existence of the equilibrium point have not been proved yet. Besides, the same as generative models based on neural networks, GANs have the common disadvantages of neural networks, such as poor interpretability. Furthermore, although the samples generated by GANs are very different in style, there exists the mode collapse problem (that is, GANs cannot generate continuously changing samples with the change of input noise z). Scientific researchers have put forward many research directions to focus on better solving those drawbacks of GANs. From the perspective of combining GANs with other methods, how to integrate GANs with feature learning, imitation learning, and reinforcement learning to develop new AI applications and promote the development of these methods is very meaningful and hopefully. Below, we mainly introduce the performance of several classic GAN variant models in image processing.