In this paper, we present a novel approach to automatic 3D Facial Expression Recognition (FER) based on deep representation of facial 3D geometric and 2D photometric attributes. A 3D face is firstly represented by its geometric and photometric attributes, including the geometry map, normal maps, normalized curvature map and texture map. These maps are then fed into a pre-trained deep convolutional neural network to generate the deep representation. Then the facial expression prediction is simplyachieved by training linear SVMs over the deep representation for different maps and fusing these SVM scores. The visualizations show that the deep representation provides a complete and highly discriminative coding scheme for 3D faces. Comprehensive experiments on the BU-3DFE database demonstrate that the proposed deep representation can outperform the widely used hand-crafted descriptors (i.e., LBP, SIFT, HOG, Gabor) and the state-of-art approaches under the same experimental protocols.
3D face models accurately capture facial surfaces, making it possible for precise description of facial activities. In this paper, we present a novel mesh-based method for 3D facial expression recognition using two local shape descriptors. To characterize shape information of the local neighborhood of facial landmarks, we calculate the weighted statistical distributions of surface differential quantities, including histogram of mesh gradient (HoG) and histogram of shape index (HoS). Normal cycle theory based curvature estimation method is employed on 3D face models along with the common cubic fitting curvature estimation method for the purpose of comparison. Based on the basic fact that different expressions involve different local shape deformations, the SVM classifier with both linear and RBF kernels outperforms the state of the art results on the subset of the BU-3DFE database with the same experimental setting. This paper describes a pose invariant three-dimensional (3D) facial expression recognition method using distance vectors retrieved from 3D distributions of facial feature points to classify universal facial expressions. Probabilistic Neural Network architecture is employed as a classifier to recognize the facial expressions from a distance vector obtained from 3D facial feature locations. Facial expressions such as anger, sadness, surprise, joy, disgust, fear and neutral are successfully recognized with an average recognition rate of 87.8 . The creation of facial range models by 3D imaging systems has led to extensive work on 3D face recognition [19]. However, little work has been done to study the usefulness of such data for recognizing and understanding facial expressions. Psychological research shows that the shape of a human face, a highly mobile facial surface, is critical to facial expression perception. In this paper, we investigate the importance and usefulness of 3D facial geometric shapes to represent and recognize facial expressions using 3D facial expression range data. We propose a novel approach to extract primitive 3D facial expression features, and then apply the feature distribution to classify the prototypic facial expressions. In order to validate our proposed approach, we have conducted experiments for person-independent facial expression recognition using our newly created 3D facial expression database. We also demonstrate the advantages of our 3D geometric based approach over 2D texture based approaches in terms of various head poses. In this paper, we propose a novel approach for facial expression analysis and recognition. The proposed approach relies on the distance vectors retrieved from 3D distribution of facial feature points to classify universal facial expressions. Neural network architecture is employed as a classifier to recognize the facial expressions from a distance vector obtained from 3D facial feature locations. Facial expressions such as anger, sadness, surprise, joy, disgust, fear and neutral are successfully recognized with an average recognition rate of 91.3 . The highest recognition rate reaches to 98.3 in the recognition of surprise. We investigate the problem of facial expression recognition using 3D face data. Our approach is based on local shape analysis of several relevant regions of a given face scan. These regions or patches from facial surfaces are extracted and represented by sets of closed curves. A Riemannian framework is used to derive the shape analysis of the extracted patches. The applied framework permits to calculate a similarity (or dissimilarity) distances between patches, and to compute the optimal deformation between them. Once calculated, these measures are employed as inputs to a commonly used classification techniques such as AdaBoost and Support Vector Machines (SVM). A quantitative evaluation of our novel approach is conducted on a subset of the publicly available BU-3DFE database. We present a semi-automatic 3D Facial Expression Recognition system based on geometric facial information. In this approach, the 3D facial meshes are first fitted to an Annotated Face Model (AFM). Then, the Expressive Maps are computed, which indicate the parts of the face that are most expressive according to a particular geometric feature (e.g., vertex coordinates, normals, and local curvature). The Expressive Maps provide a way to analyze the geometric features in terms of their discriminative information and their distribution along the face and allow the reduction of the dimensionality of the input space to 2:5 of the original size. Using the selected features a simple linear classifier was trained and yielded a very competitive average recognition rate of 90:4 when evaluated using ten-fold cross validation on the publicly available BU-3DFE database. We propose a general and fully automatic framework for 3D facial expression recognition by modeling sparse representation of conformal images. According to Riemann Geometry theory, a 3D facial surface S embedded in ℝ3, which is a topological disk, can be conformally mapped to a 2D unit disk D through the discrete surface Ricci Flow algorithm. Such a conformal mapping induces a unique and intrinsic surface conformal representation denoted by a pair of functions defined on D, called conformal factor image (CFI) and mean curvature image (MCI). As facial expression features, CFI captures the local area distortion of S induced by the conformal mapping; MCI characterizes the geometry information of S. To model sparse representation of conformal images for expression classification, both CFI and MCI are further normalized by a Mobius transformation. This transformation is defined by the three main facial landmarks (i.e. nose tip, left and right inner eye corners) which can be detected automatically and precisely. Expression recognition is carried out by the minimal sparse expression-class-dependent reconstruction error over the conformal image based expression dictionary. Extensive experimental results on the BU-3DFER dataset demonstrate the effectiveness and generalization of the proposed framework. The 3D facial geometry contains ample information about human facial expressions. Such information is invariant to pose and lighting conditions, which have imposed serious hurdles on many 2D facial analysis problems. In this paper, we perform person and gender independent facial expression recognition based on properties of the line segments connecting certain 3D facial feature points. The normalized distances and slopes of these line segments comprise a set of 96 distinguishing features for recognizing six universal facial expressions, namely anger, disgust, fear, happiness, sadness, and surprise. Using a multi-class support vector machine (SVM) classifier, an 87.1 average recognition rate is achieved on the publicly available 3D facial expression database BU-3DFE. The highest average recognition rate obtained in our experiments is 99.2 for the recognition of surprise. Our result outperforms the result reported in the prior work, which uses elaborately extracted primitive facial surface features and an LDA classifier and which yields an average recognition rate of 83.6 on the same database. This survey focuses on discrete expression classification and facial action unit recognition performed using 3D face data, possibly including a corresponding 2D texture image. Research trends to date are summarized and the limitations of current methods are discussed. The challenges towards the development of more accurate and automated 3D facial expression recognition methods are identified. We also call for standardized experimental protocols in order to draw fair and meaningful comparisons between different systems. In this paper, the problem of person-independent facial expression recognition from 3D facial shapes is investigated. We propose a novel automatic feature selection method based on maximizing the average relative entropy of marginalized class-conditional feature distributions and apply it to a complete pool of candidate features composed of normalized Euclidean distances between 83 facial feature points in the 3D space. Using a regularized multi-class AdaBoost classification algorithm, we achieve a 95.1 average recognition rate for six universal facial expressions on the publicly available 3D facial expression database BU-3DFE [1], with a highest average recognition rate of 99.2 for the recognition of surprise. We compare these results with the results based on a set of manually devised features and demonstrate that the auto features yield better results than the manual features. Our results outperform the results presented in the previous work [2] and [3], namely average recognition rates of 83.6 and 91.3 on the same database, respectively.