Influencing the Properties of Latent Spaces
Machine learning models rely on learned parameters adapted to a given set of data to perform a task, such as classifying images or generating sentences. These learned parameters form latent spaces which can adapt various properties, to impact how well the model performs. Enabling a model to better fit properties of the data in its latent space can improve the performance of the model. One criteria for quality is the set of properties expressed by the latent space - that is, for example, topological properties of the learned representation. We develop a model which leverages a variational autoencoder’s generative ability and augments it with the ladder network’s lateral connections for discrimination. We propose a method to decouple two tasks perfomed by convolutional layers (that of learning useful filters for feature extraction, and that of arranging the learned filters such that the next layer may train eff ectively) by using interspersed fully-connected layers. Finally, we apply batch normalization to the recurrent state of the pixel-rnn layer and show that it significantly improves convergence speed as well as slightly improving overall performance. We show results applied in unsupervised and supervised settings, and augment models with various inter-layer interractions, such as encoder-to-decoder connections, affine post-layer transformations, and side-network connections. The effects of the proposed methods are assessed by measuring supervised performance or the quality of samples produced by the model and training curves. Models and methods are tested on popular image datasets such as MNIST and CIFAR10 and are compared to the state-of-the-art on the task they are applied to.
Non-supervise Machine learning Deep learning Computer vision Unsupervised Autoencoder Neural network