In this paper, we present a novel localized Generative Adversarial Net (GAN) to learn on the manifold of real data. Compared with the classic GAN that globally parameterizes a manifold, the Localized GAN (LGAN) uses local coordinate charts to parameterize distinct local geometry of how data points can transform at different locations on the manifold. Specifically, around each point there exists a local generator that can produce data following diverse patterns of transformations on the manifold. The locality nature of LGAN enables local generators to adapt to and directly access the local geometry without need to invert the generator in a global GAN. Furthermore, it can prevent the manifold from being locally collapsed to a dimensionally deficient tangent subspace by imposing an orthonormality prior between tangents. This provides a geometric approach to alleviating mode collapse at least locally on the manifold by imposing independence between data transformations in different tangent directions. We will also demonstrate the LGAN can be applied to train a robust classifier that prefers locally consistent classification decisions on the manifold, and the resultant regularizer is closely related with the Laplace-Beltrami operator. Our experiments show that the proposed LGANs can not only produce diverse image transformations, but also deliver superior classification performances.
The distinction between global and local coordinate systems results in conceptual and algorithmic differences between global and local GANs. Conceptually, the global GANs assume that the manifolds formed by their generators could be globally parameterized in a way that the manifolds are topologically similar to an Euclidean coordinate space. In contrast, the localized paradigm abandons the global parameterizability assumption, allowing us to use multiple local coordinate charts to cover an entire manifold. Algorithmically, if a global GAN needs to access local geometric information underlying its generated manifold, it has to invert the generator function in order to find the global coordinates corresponding to a given data point. This is usually performed by learning an auto-encoder network along with the GAN models, e.g., BiGAN @cite_16 and ALI @cite_4 . On the contrary, the localized GAN enables direct access to local geometry without having to invert a global generator, since the local geometry around a point is directly available from the corresponding local generator. Moreover, the orthormornality between local tangents could also maximize the capability of a local GAN in exploring independent local transformations along different coordinate directions, thereby preventing the manifold of generated data from being locally collapsed with deficient dimensions.
@cite_16 @cite_4
2412320034 2411541852
The ability of the Generative Adversarial Networks (GANs) framework to learn generative models mapping from simple latent distributions to arbitrarily complex data distributions has been demonstrated empirically, with compelling results showing that the latent space of such generators captures semantic variation in the data distribution. Intuitively, models trained to predict these semantic latent representations given data may serve as useful feature representations for auxiliary problems where semantics are relevant. However, in their existing form, GANs have no means of learning the inverse mapping -- projecting data back into the latent space. We propose Bidirectional Generative Adversarial Networks (BiGANs) as a means of learning this inverse mapping, and demonstrate that the resulting learned feature representation is useful for auxiliary supervised discrimination tasks, competitive with contemporary approaches to unsupervised and self-supervised feature learning. We introduce the adversarially learned inference (ALI) model, which jointly learns a generation network and an inference network using an adversarial process. The generation network maps samples from stochastic latent variables to the data space while the inference network maps training examples in data space to the space of latent variables. An adversarial game is cast between these two networks and a discriminative network is trained to distinguish between joint latent data-space samples from the generative network and joint samples from the inference network. We illustrate the ability of the model to learn mutually coherent inference and generation networks through the inspections of model samples and reconstructions and confirm the usefulness of the learned representations by obtaining a performance competitive with state-of-the-art on the semi-supervised SVHN and CIFAR10 tasks.