3D geometric contents are becoming increasingly popular. In this paper, we study the problem of analyzing deforming 3D meshes using deep neural networks. Deforming 3D meshes are flexible to represent 3D animation sequences as well as collections of objects of the same category, allowing diverse shapes with large-scale non-linear deformations. We propose a novel framework which we call mesh variational autoencoders (mesh VAE), to explore the probabilistic latent space of 3D surfaces. The framework is easy to train, and requires very few training examples. We also propose an extended model which allows flexibly adjusting the significance of different latent variables by altering the prior distribution. Extensive experiments demonstrate that our general framework is able to learn a reasonable representation for a collection of deformable shapes, and produce competitive results for a variety of applications, including shape generation, shape interpolation, shape space embedding and shape exploration, outperforming state-of-the-art methods.
To effectively represent a collection of shapes with the same connectivity, a naive solution is to take the vertex positions as the representation. Such representations however are not translation or rotation invariant. Among various representations, the RIMD representation @cite_8 is translation and rotation invariant, and suitable for data-driven shape analysis. Therefore, we use it to represent shapes in our framework.
@cite_8
2963706553
We aim to obtain an interpretable, expressive, and disentangled scene representation that contains comprehensive structural and textural information for each object. Previous representations learned by neural networks are often uninterpretable, limited to a single object, or lacking 3D knowledge. In this work, we propose 3D scene de-rendering networks (3D-SDN) to address the above issues by integrating disentangled representations for object semantics, appearance, and geometry into a deep generative model. Our scene encoder performs inverse graphics, translating a scene into a structured object representation. Our decoder has two components: a differentiable shape renderer and a neural texture generator. The disentanglement of semantics, appearance, and geometry supports various 3D-aware scene manipulation applications, , rotating and moving objects freely while maintaining consistent shape and texture, and changing the object appearance without affecting its shape. We systematically evaluate our model and demonstrate that our editing scheme based on 3D-SDN is superior to its 2D counterpart.