In a generative process, we want to produce new examples with the same statistical distribution as the training data. There are different techniques to model the data for generation. Among these techniques, autoencoders, generative adversarial networks, and normalizing flows are very popular. Autoencoders (AE) have two networks: an encoder that projects the inputs into a vector, usually with smaller dimension, and a decoder that reconstructs the input from this vector. The compressed vector has a high level representation of the model, in which each neuron contributes to properties beyond the data level at the input layer [1]}. Autoencoders are commonly trained in a unsupervised fashion, nevertheless there are variants that include labeled information to further increase training for a specific objective. Variational autoencoders (VAE) [2]}, [3]}, [4]} improve the sampling ability by projecting the data into a probabilistic latent vector.
Generative adversarial networks (GANs) [5]}, [6]}, [7]} are another example of a generative method. By having two competing networks, a generative which takes a random low-dimensional input and outputs an image, and a discriminator network that compares the images from the training data set and the sampled ones from the generator. The competition arises by training the generator to fool the discriminator by generating images as closest to the training data set as possible. Normalizing flows [8]}, [9]}, [10]} generate images by transforming a simple distribution to a more complex one by a series of reversible transformations.
