Mesh-based formulation encodes the underlying structure of 3D models, such as edges, vertices and faces. It is a powerful representation that naturally captures the surface of 3D shape.
Several works considered the problem of learning mesh generation from images [1]}, [2]} or point clouds data [3]}, [4]}. However, these approaches are only able to reconstruct single objects, and limited to generating models with simple structures or from familiar classes.
To tackle the problem of scene reconstruction in mesh representation, [5]} integrates the sparse features from monocular SLAM with dense depth maps from ConvNet for the update of 3D mesh representation. The depth predictions are fused into the monocular SLAM system to recover the absolute scale of pose and scene features estimation. To allow efficient computation and flexible information fusion, [6]} utilizes 2.5D mesh to represent scene geometry. In their approach, the image plane coordinates of mesh vertices are learned by deep neural networks, while the depth maps are optimized as free variables.
