Textured Mesh Generation. 3D object generation approaches that use voxels [1]}, [2]}, [3]}, [4]}, [5]} or point clouds [6]}, [7]} typically require some form of 3D supervision and are unfriendly for modeling texture.
Chen . [8]} propose DIB-R, a GAN framework for textured mesh generation, where 3D meshes are differentiably rendered into 2D images and discriminated with multi-view images of synthetic objects. Later on, Henderson . [9]} relax the multi-view restriction and propose a VAE framework [10]} that leverages a collection of single-view natural images. The appearance is parameterized by face colors instead of texture maps, limiting the visual detail of generated objects.
Under the same setting, ConvMesh [11]} achieves more realistic 3D generations by generating 3D objects in the form of topology-aligned texture maps and deformation maps in the UV space, where discrimination directly takes place in the UV space against pseudo ground truths. Pseudo deformation maps are obtained by overfitting a mesh reconstruction baseline on the training set. Subsequently, the associated pseudo texture maps can then be obtained by projecting natural images on the UV space. Our proposed method is built upon a pre-trained ConvMesh model to incorporate its generative prior in 3D reconstruction.
