TECHNICAL FIELD

The present disclosure relates generally to generative adversarial networks (GANs) and, in particular, to methods and systems for autoencoder-based GANs for text generation.

BACKGROUND

Deep learning has shown great success in various domains such as natural language processing (NLP), autonomous driving, gaming, and unsupervised learning. Generative adversarial networks (GANs) have been developed to generate realistic-looking synthetic images. GANs correspond to a mini-max two-player game where two models (e.g., two artificial neural networks) are trained simultaneously: a generative model G that captures data distribution, and a discriminative model 1) that computes the probability that a sample comes from the training data rather than from the generator. GAN solutions can be useful when there is a scarcity of training samples.

GANs have achieved substantial success in the field of computer vision for generating realistic-looking images. However, applying a GAN to NLP applications can be technically challenging because of the discrete nature of natural languages (e.g., text in a language does not map to real numbers with an inherent mapping function). For example, one technical problem relates to backpropagation. In NLP applications, text is a sequence of discrete words, and the output of the generator would be a discrete representation of the sequence of words. The discrete nature of the representation of the sequence of words output by the generator makes the backpropagation procedure, which is used in training the GAN, difficult.

Accordingly, more efficient and robust techniques for training a GAN for NLP applications are desirable.

SUMMARY OF THE INVENTION

Technical advantages are generally achieved, by embodiments of this disclosure which describe methods and systems for training an autoencoder-based generative adversarial network (GAN) for text generation.

In accordance to embodiments for training an autoencoder-based generative adversarial network (GAN) executing on one or more processing units for text generation, the autoencoder-based GAN may train an encoder neural network of the autoencoder-based GAN. The encoder neural network may be configured to receive a one-hot representation of real text. The encoder neural network may further be configured to output a latent representation of the real text generated from the one-hot representation of the real text. The real text may comprise a sequence of words.

The autoencoder-based GAN may train a decoder neural network of the autoencoder-based GAN. The decoder neural network may be configured to receive the latent representation of the real text. The decoder neural network may further be configured to output a reconstructed softmax representation of the real text generated from the latent representation of the real text. The reconstructed softmax representation of the real text may comprise a soft-text that is a continuous representation of the real text.

The autoencoder-based GAN may train a GAN of the autoencoder-based GAN. The GAN may comprise a generator neural network and a discriminator neural network. The generator neural network may be configured to generate artificial text based on random noise data. The discriminator neural network may be configured to receive the soft-text from the decoder neural network. The discriminator neural network may further be configured to receive a softmax representation of the artificial text generated by the generator neural network. The discriminator neural network may additionally output a probability indicating whether the softmax representation of the artificial text received by the discriminator neural network is not from the generator neural network.

Apparatuses, as well as computer program products, for performing the methods are also provided.

