An efficient, fully automatic method for 3D face shape and pose estimation in unconstrained 2D imagery is presented. The proposed method jointly estimates a dense set of 3D landmarks and facial geometry using a single pass of a modified version of the popular "U-Net" neural network architecture. Additionally, we propose a method for directly estimating a set of 3D Morphable Model (3DMM) parameters, using the estimated 3D landmarks and geometry as constraints in a simple linear system. Qualitative modeling results are presented, as well as quantitative evaluation of predicted 3D face landmarks in unconstrained video sequences.
In order to avoid the problem of viewpoint-dependent 2D landmarks, @cite_15 introduced the projected normalized coordinate code (PNCC), which acts as a fully-3D set of dense facial landmarks. They leverage the PNCC as an intermediate output in a cascade of convolutional neural networks, each of which produce updates to the current 3DMM coefficient estimates given the input image and a predicted PNCC, which is rendered using the current coefficient estimates at each stage. The proposed method also leverages the idea of the PNCC image, but directly estimates the PNCC (in conjunction with an image of dense 3D offsets) using a single network given only the original image as input. 3DMM coefficients are then fit (if desired) directly to the predicted 3D output.
@cite_15
2964014798
Face alignment, which fits a face model to an image and extracts the semantic meanings of facial pixels, has been an important topic in CV community. However, most algorithms are designed for faces in small to medium poses (below 45), lacking the ability to align faces in large poses up to 90. The challenges are three-fold: Firstly, the commonly used landmark-based face model assumes that all the landmarks are visible and is therefore not suitable for profile views. Secondly, the face appearance varies more dramatically across large poses, ranging from frontal view to profile view. Thirdly, labelling landmarks in large poses is extremely challenging since the invisible landmarks have to be guessed. In this paper, we propose a solution to the three problems in an new alignment framework, called 3D Dense Face Alignment (3DDFA), in which a dense 3D face model is fitted to the image via convolutional neutral network (CNN). We also propose a method to synthesize large-scale training samples in profile views to solve the third problem of data labelling. Experiments on the challenging AFLW database show that our approach achieves significant improvements over state-of-the-art methods.