In this paper we explore the problem of fitting a 3D morphable model to single face images using only sparse geometric features (edges and landmark points). Previous approaches to this problem are based on nonlinear optimisation of an edge-derived cost that can be viewed as forming soft correspondences between model and image edges. We propose a novel approach, that explicitly computes hard correspondences. The resulting objective function is non-convex but we show that a good initialisation can be obtained efficiently using alternating linear least squares in a manner similar to the iterated closest point algorithm. We present experimental results on both synthetic and real images and show that our approach outperforms methods that use soft correspondence and other recent methods that rely solely on geometric features.
Edge fitting An early example of using image edges for face model fitting is the Active Shape Model (ASM) @cite_20 where a 2D boundary model is aligned to image edges. In 3D, contours have been used directly for 3D face shape estimation @cite_26 and indirectly as a feature for fitting a 3DMM. The earliest work in this direction was due to Moghaddam al @cite_4 who fitted a 3DMM to silhouettes extracted from multiple views. From a theoretical standpoint, L "u thi al @cite_15 explored to what degree face shape is constrained when contours are fixed.
@cite_15 @cite_26 @cite_4 @cite_20
1711351391 1526895196 2141661257 2038952578
Statistical shape models, and in particular morphable models, have gained widespread use in computer vision, computer graphics and medical imaging. Researchers have started to build models of almost any anatomical structure in the human body. While these models provide a useful prior for many image analysis task, relatively little information about the shape represented by the morphable model is exploited. We propose a method for computing and visualizing the remaining flexibility, when a part of the shape is fixed. Our method, which is based on Probabilistic PCA, not only leads to an approach for reconstructing the full shape from partial information, but also allows us to investigate and visualize the uncertainty of a reconstruction. To show the feasibility of our approach we performed experiments on a statistical model of the human face and the femur bone. The visualization of the remaining flexibility allows for greater insight into the statistical properties of the shape. This paper presents a novel method for estimating the three-dimensional shape of faces, facilitating the possibility of enhanced face recognition. The method involves a combined use of photometric stereo and profile view information. It can be divided into three principal stages: (1) An initial estimate of the face is obtained using four-source high-speed photometric stereo. (2) The profile is determined from a side-view camera. (3) The facial shape estimation is iteratively refined using the profile until an energy functional is minimised. This final stage, which is the most important contribution of the paper, works by continually deforming the shape estimate so that its profile is exact. An energy is then calculated based on the difference between the raw images and synthetic images generated using the new shape estimate. The surface normals are then adjusted according to energy until convergence. Several real face reconstructions are presented and compared to ground truth. The results clearly demonstrate a significant improvement in accuracy compared to standard photometric stereo. We present a method for 3D face acquisition using a set or sequence of 2D binary silhouettes. Since silhouette images depend only on the shape and pose of an object, they are immune to lighting and or texture variations (unlike feature or texture-based shape-from-correspondence). Our prior 3D face model is a linear combination of "eigenheads" obtained by applying PCA to a training set of laser-scanned 3D faces. These shape coefficients are the parameters for a near-automatic system for capturing the 3D shape as well as the 2D texture-map of a novel input face. Specifically, we use back-projection and a boundary-weighted XOR-based cost function for binary silhouette matching, coupled with a probabilistic "downhill-simplex" optimization for shape estimation and refinement. Experiments with a multicamera rig as well as monocular video sequences demonstrate the advantages of our 3D modeling framework and ultimately, its utility for robust face recognition with built-in invariance to pose and illumination. !, Model-based vision is firmly established as a robust approach to recognizing and locating known rigid objects in the presence of noise, clutter, and occlusion. It is more problematic to apply modelbased methods to images of objects whose appearance can vary, though a number of approaches based on the use of flexible templates have been proposed. The problem with existing methods is that they sacrifice model specificity in order to accommodate variability, thereby compromising robustness during image interpretation. We argue that a model should only be able to deform in ways characteristic of the class of objects it represents. We describe a method for building models by learning patterns of variability from a training set of correctly annotated images. These models can be used for image search in an iterative refinement algorithm analogous to that employed by Active Contour Models (Snakes). The key difference is that our Active Shape Models can only deform to fit the data in ways consistent with the training set. We show several practical examples where we have built such models and used them to locate partially occluded objects in noisy, cluttered images. Q 199s A&& prrss, IN.