A standard Variational Autoencoder, with a Euclidean latent space, is structurally incapable of capturing topological properties of certain datasets. To remove topological obstructions, we introduce Diffusion Variational Autoencoders with arbitrary manifolds as a latent space. A Diffusion Variational Autoencoder uses transition kernels of Brownian motion on the manifold. In particular, it uses properties of the Brownian motion to implement the reparametrization trick and fast approximations to the KL divergence. We show that the Diffusion Variational Autoencoder is capable of capturing topological properties of synthetic datasets. Additionally, we train MNIST on spheres, tori, projective spaces, SO(3), and a torus embedded in R3. Although a natural dataset like MNIST does not have latent variables with a clear-cut topological structure, training it on a manifold can still highlight topological and geometrical properties.
If a manifold has the additional structure of a Lie group, this structure allows for a more straightforward implementation of the reparametrization trick @cite_22 . In our work, we do not assume the additional structure of a Lie group, but develop a reparametrization trick that works for general submanifolds of Euclidean space, and therefore by the Whitney (respectively Nash) embedding theorem, for general closed (Riemannian) manifolds.
@cite_22
2128878012 2817444259 2030605635 2963747906
For a sufficiently dense set of points in any closed Riemannian manifold, we prove that a unique Delannay triangulation exists. This triangulation has the same properties as in Euclidean space. Algorithms for constructing these triangulations will also be described. The manifold hypothesis states that many kinds of high-dimensional data are concentrated near a low-dimensional manifold. If the topology of this data manifold is non-trivial, a continuous en-coder network cannot embed it in a one-to-one manner without creating holes of low density in the latent space. This is at odds with the Gaussian prior assumption typically made in Variational Auto-Encoders (VAEs), because the density of a Gaussian concentrates near a blob-like manifold. In this paper we investigate the use of manifold-valued latent variables. Specifically, we focus on the important case of continuously differen-tiable symmetry groups (Lie groups), such as the group of 3D rotations SO(3). We show how a VAE with SO(3)-valued latent variables can be constructed, by extending the reparameterization trick to compact connected Lie groups. Our exper-iments show that choosing manifold-valued latent variables that match the topology of the latent data manifold, is crucial to preserve the topological structure and learn a well-behaved latent space. A convenient way of analysing Riemannian manifolds is to embed them in Euclidean spaces, with the embedding typically obtained by flattening the manifold via tangent spaces. This general approach is not free of drawbacks. For example, only distances between points to the tangent pole are equal to true geodesic distances. This is restrictive and may lead to inaccurate modelling. Instead of using tangent spaces, we propose embedding into the Reproducing Kernel Hilbert Space by introducing a Riemannian pseudo kernel. We furthermore propose to recast a locality preserving projection technique from Euclidean spaces to Riemannian manifolds, in order to demonstrate the benefits of the embedding. Experiments on several visual classification tasks (gesture recognition, person re-identification and texture classification) show that in comparison to tangent-based processing and state-of-the-art methods (such as tensor canonical correlation analysis), the proposed approach obtains considerable improvements in discrimination accuracy. By using optimal mass transportation and a quantitative Holder inequality, we provide estimates for the Borell–Brascamp–Lieb deficit on complete Riemannian manifolds. Accordingly, equality cases in Borell–Brascamp–Lieb inequalities (including Brunn–Minkowski and Prekopa–Leindler inequalities) are characterized in terms of the optimal transport map between suitable marginal probability measures. These results provide several qualitative applications both in the flat and non-flat frameworks. In particular, by using Caffarelli's regularity result for the Monge–Ampere equation, we give a new proof of Dubuc's characterization of the equality in Borell–Brascamp–Lieb inequalities in the Euclidean setting. When the n-dimensional Riemannian manifold has Ricci curvature Ric(M) ≥ (n-1)k for some k ⋲ ℝ, it turns out that equality in the Borell–Brascamp–Lieb inequality is expected only when a particular region of the manifold between the marginal supports has constant sectional curvature k. A precise characterization is provided for the equality in the Lott–Sturm–Villani-type distorted Brunn–Minkowski inequality on Riemannian manifolds. Related results for (not necessarily reversible) Finsler manifolds are also presented.