Estimating the head pose of a person is a crucial problem that has a large amount of applications such as aiding in gaze estimation, modeling attention, fitting 3D models to video and performing face alignment. Traditionally head pose is computed by estimating some keypoints from the target face and solving the 2D to 3D correspondence problem with a mean human head model. We argue that this is a fragile method because it relies entirely on landmark detection performance, the extraneous head model and an ad-hoc fitting step. We present an elegant and robust way to determine pose by training a multi-loss convolutional neural network on 300W-LP, a large synthetically expanded dataset, to predict intrinsic Euler angles (yaw, pitch and roll) directly from image intensities through joint binned pose classification and regression. We present empirical tests on common in-the-wild pose benchmark datasets which show state-of-the-art results. Additionally we test our method on a dataset usually used for pose estimation using depth and start to close the gap with state-of-the-art depth pose methods. We open-source our training and testing code as well as release our pre-trained models.
This paper investigates how far a very deep neural network is from attaining close to saturating performance on existing 2D and 3D face alignment datasets. To this end, we make the following 5 contributions: (a) we construct, for the first time, a very strong baseline by combining a state-of-the-art architecture for landmark localization with a state-of-the-art residual block, train it on a very large yet synthetically expanded 2D facial landmark dataset and finally evaluate it on all other 2D facial landmark datasets. (b)We create a guided by 2D landmarks network which converts 2D landmark annotations to 3D and unifies all existing datasets, leading to the creation of LS3D-W, the largest and most challenging 3D facial landmark dataset to date ( 230,000 images). (c) Following that, we train a neural network for 3D face alignment and evaluate it on the newly introduced LS3D-W. (d) We further look into the effect of all “traditional” factors affecting face alignment performance like large pose, initialization and resolution, and introduce a “new” one, namely the size of the network. (e) We show that both 2D and 3D face alignment networks achieve performance of remarkable accuracy which is probably close to saturating the datasets used. Training and testing code as well as the dataset can be downloaded from https: www.adrianbulat.com face-alignment Face alignment, which fits a face model to an image and extracts the semantic meanings of facial pixels, has been an important topic in CV community. However, most algorithms are designed for faces in small to medium poses (below 45), lacking the ability to align faces in large poses up to 90. The challenges are three-fold: Firstly, the commonly used landmark-based face model assumes that all the landmarks are visible and is therefore not suitable for profile views. Secondly, the face appearance varies more dramatically across large poses, ranging from frontal view to profile view. Thirdly, labelling landmarks in large poses is extremely challenging since the invisible landmarks have to be guessed. In this paper, we propose a solution to the three problems in an new alignment framework, called 3D Dense Face Alignment (3DDFA), in which a dense 3D face model is fitted to the image via convolutional neutral network (CNN). We also propose a method to synthesize large-scale training samples in profile views to solve the third problem of data labelling. Experiments on the challenging AFLW database show that our approach achieves significant improvements over state-of-the-art methods. Keypoint detection is one of the most importantpre-processing steps in tasks such as face modeling, recognitionand verification. In this paper, we present an iterative methodfor Keypoint Estimation and Pose prediction of unconstrainedfaces by Learning Efficient H-CNN Regressors (KEPLER) foraddressing the face alignment problem. Recent state of the artmethods have shown improvements in face keypoint detectionby employing Convolution Neural Networks (CNNs). Althougha simple feed forward neural network can learn the mappingbetween input and output spaces, it cannot learn the inherentstructural dependencies. We present a novel architecture calledH-CNN (Heatmap-CNN) which captures structured global andlocal features and thus favors accurate keypoint detecion. H-CNNis jointly trained on the visibility, fiducials and 3D-pose of theface. As the iterations proceed, the error decreases making thegradients small and thus requiring efficient training of DCNNs tomitigate this. KEPLER performs global corrections in pose andfiducials for the first four iterations followed by local correctionsin a subsequent stage. As a by-product, KEPLER also provides3D pose (pitch, yaw and roll) of the face accurately. In thispaper, we show that without using any 3D information, KEPLERoutperforms state of the art methods for alignment on challengingdatasets such as AFW [38] and AFLW [17].