In this paper, we explore new approaches to combining information encoded within the learned representations of autoencoders. We explore models that are capable of combining the attributes of multiple inputs such that a resynthesised output is trained to fool an adversarial discriminator for real versus synthesised data. Furthermore, we explore the use of such an architecture in the context of semi-supervised learning, where we learn a mixing function whose objective is to produce interpolations of hidden states, or masked combinations of latent representations that are consistent with a conditioned class label. We show quantitative and qualitative evidence that such a formulation is an interesting avenue of research. The autoencoder is a fundamental building block in unsupervised learning. Autoencoders are trained to reconstruct their inputs after being processed by two neural networks: an encoder which encodes the input to a high-level representation or bottleneck, and a decoder which performs the reconstruction using the representation as input. One primary goal of the autoencoder is to learn representations of the input data which are useful BID1 , which may help in downstream tasks such as classification BID27 BID9 or reinforcement learning BID20 BID5 . The representations of autoencoders can be encouraged to contain more 'useful' information by restricting the size of the bottleneck, through the use of input noise (e.g., in denoising autoencoders, BID23 , through regularisation of the encoder function BID17 , or by introducing a prior BID11 . Another goal is in learning interpretable representations BID3 BID10 . In unsupervised learning, learning often involves qualitative objectives on the representation itself, such as disentanglement of latent variables BID12 or maximisation of mutual information BID3 BID0 BID8 .Mixup BID26 and manifold mixup BID21 are regularisation techniques that encourage deep neural networks to behave linearly between two data samples. These methods artificially augment the training set by producing random convex combinations between pairs of examples and their corresponding labels and training the network on these combinations. This has the effect of creating smoother decision boundaries, which can have a positive effect on generalisation performance. In BID21 , the random convex combinations are computed in the hidden space of the network. This procedure can be viewed as using the high-level representation of the network to produce novel training examples and provides improvements over strong baselines in the supervised learning. Furthermore, BID22 propose a simple and efficient method for semi-supervised classification based on random convex combinations between unlabeled samples and their predicted labels.In this paper we explore the use of a wider class of mixing functions for unsupervised learning, mixing in the bottleneck layer of an autoencoder. These mixing functions could consist of continuous interpolations between latent vectors such as in BID21 , to binary masking operations to even a deep neural network which learns the mixing operation. In order to ensure that the output of the decoder given the mixed representation resembles the data distribution at the pixel level, we leverage adversarial learning BID4 , where here we train a discriminator to distinguish between decoded mixed and unmixed representations. This technique affords a model the ability to simulate novel data points (such as those corresponding to combinations of annotations not present in the training set). Furthermore, we explore our approach in the context of semi-supervised learning, where we learn a mixing function whose objective is to produce interpolations of hidden states consistent with a conditioned class label. In this paper, we proposed the adversarial mixup resynthesiser and showed that it can be used to produce realistic-looking combinations of examples by performing mixing in the bottleneck of an autoencoder. We proposed several mixing functions, including one based on sampling from a uniform distribution and the other a Bernoulli distribution. Furthermore, we presented a semisupervised version of the Bernoulli variant in which one can leverage class labels to learn a mixing function which can determine what parts of the latent code should be mixed to produce an image consistent with a desired class label. While our technique can be used to leverage an autoencoder as a generative model, we conjecture that our technique may have positive effects on the latent representation and therefore downstream tasks, though this is yet to be substantiated. Future work will involve more comparisons to existing literature and experiments to determine the effects of mixing on the latent space itself and downstream tasks.
We leverage deterministic autoencoders as generative models by proposing mixing functions which combine hidden states from pairs of images. These mixes are made to look realistic through an adversarial framework.
{"src": "scitldr-AIC_train", "paper_id": "Syx9EIIKdN"}