In this paper we study generative modeling via autoencoders while using the elegant geometric properties of the optimal transport (OT) problem and the Wasserstein distances. We introduce Sliced-Wasserstein Autoencoders (SWAE), which are generative models that enable one to shape the distribution of the latent space into any samplable probability distribution without the need for training an adversarial network or defining a closed-form for the distribution. In short, we regularize the autoencoder loss with the sliced-Wasserstein distance between the distribution of the encoded training samples and a predefined samplable distribution. We show that the proposed formulation has an efficient numerical solution that provides similar capabilities to Wasserstein Autoencoders (WAE) and Variational Autoencoders (VAE), while benefiting from an embarrassingly simple implementation. 
 Scalable generative models that capture the rich and often nonlinear distribution of highdimensional data, (i.e., image, video, and audio), play a central role in various applications of machine learning, including transfer learning BID13 BID24 , super-resolution BID15 BID20 , image inpainting and completion BID34 , and image retrieval BID6 , among many others. The recent generative models, including Generative Adversarial Networks (GANs) BID0 BID1 BID10 BID29 and Variational Autoencoders (VAE) BID4 BID14 BID23 enable an unsupervised and end-to-end modeling of the high-dimensional distribution of the training data.Learning such generative models boils down to minimizing a dissimilarity measure between the data distribution and the output distribution of the generative model. To this end, and following the work of Arjovsky et al. BID0 and Bousquet et al. BID4 we approach the problem of generative modeling from the optimal transport point of view. The optimal transport problem BID17 BID33 provides a way to measure the distances between probability distributions by transporting (i.e., morphing) one distribution into another. Moreover, and as opposed to the common information theoretic dissimilarity measures (e.g., f -divergences), the p-Wasserstein dissimilarity measures that arise from the optimal transport problem: 1) are true distances, and 2) metrize a weak convergence of probability measures (at least on compact spaces). Wasserstein distances have recently attracted a lot of interest in the learning community BID0 BID4 BID8 BID11 BID17 due to their exquisite geometric characteristics BID30 . See the supplementary material for an intuitive example showing the benefit of the Wasserstein distance over commonly used f -divergences.In this paper, we introduce a new type of autoencoders for generative modeling (Algorithm 1), which we call Sliced-Wasserstein Autoencoders (SWAE), that minimize the sliced-Wasserstein distance between the distribution of the encoded samples and a predefined samplable distribution. Our work is most closely related to the recent work by Bousquet et al. BID4 and the followup work by Tolstikhin et al. BID32 . However, our approach avoids the need to perform costly adversarial training in the encoding space and is not restricted to closed-form distributions, while still benefiting from a Wasserstein-like distance measure in the encoding space that permits a simple numerical solution to the problem.In what follows we first provide an extensive review of the preliminary concepts that are needed for our formulation. In Section 3 we formulate our proposed method. The proposed numerical scheme to solve the problem is presented in Section 4. Our experiments are summarized in Section 5. Finally, our work is Concluded in Section 6. We introduced Sliced Wasserstein Autoencoders (SWAE), which enable one to shape the distribution of the encoded samples to any samplable distribution. We theoretically showed that utilizing the sliced Wasserstein distance as a dissimilarity measure between the distribution of the encoded samples and a predefined distribution ameliorates the need for training an adversarial network in the embedding space. In addition, we provided a simple and efficient numerical scheme for this problem, which only relies on few inner products and sorting operations in each SGD iteration. We further demonstrated the capability of our method on two mid-size image datasets, namely the MNIST dataset and the CelebA face dataset and showed results comparable to the techniques that rely on additional adversarial trainings. Our implementation is publicly available BID0 .
"Generative modeling with no need for adversarial training"
{"src": "scitldr-AIC_train", "paper_id": "B1VPIA7iM"}