This paper proposes a novel 4D Facial Expression Recognition (FER) method using Collaborative Cross-domain Dynamic Image Network (CCDN). Given a 4D data of face scans, we first compute its geometrical images, and then combine their correlated information in the proposed cross-domain image representations. The acquired set is then used to generate cross-domain dynamic images (CDI) via rank pooling that encapsulates facial deformations over time in terms of a single image. For the training phase, these CDIs are fed into an end-to-end deep learning model, and the resultant predictions collaborate over multi-views for performance gain in expression classification. Furthermore, we propose a 4D augmentation scheme that not only expands the training data scale but also introduces significant facial muscle movement patterns to improve the FER performance. Results from extensive experiments on the commonly used BU-4DFE dataset under widely adopted settings show that our proposed method outperforms the state-of-the-art 4D FER methods by achieving an accuracy of 96.5 indicating its effectiveness.
Yao al @cite_3 applied the scattering operator @cite_4 over 4D data, producing geometrical and textual scattering representations. Multiple Kernel Learning (MKL) was then applied to learn from this information for FER. Authors in @cite_31 presented a statistical shape model with global and local constraints in an attempt to recognize FEs. They claimed that the combination of global face shape and local shape index-based information can be handy for FER. Li al @cite_17 introduced a Dynamic Geometrical Image Network (DGIN) for automatically recognizing expressions. Given 4D data, the differential geometry quantities are estimated first, followed by generating geometrical images. These images are then fed into the DGIN for an end-to-end training. The prediction results are based on fusing the predicted scores of different types of geometrical images.
@cite_31 @cite_4 @cite_3 @cite_17
2891410536 2072072671 2794377003 1603155784
In this paper, we propose a novel method for 3D facial expression recognition based on a statistical shape model with global and local constraints. We show that the combination of the global shape of the face, along with local shape index-based information can be used to recognize a range of expressions. These expressions include happiness, sadness, surprise, embarrassment, fear, nervousness, anger, disgust, and pain. We give insights into which features are important for facial expression recognition through statistical analysis. We also show that our proposed method outperforms the current state-of-the-art methods on spontaneous and non-spontaneous facial data. A wavelet scattering network computes a translation invariant image representation which is stable to deformations and preserves high-frequency information for classification. It cascades wavelet transform convolutions with nonlinear modulus and averaging operators. The first network layer outputs SIFT-type descriptors, whereas the next layers provide complementary invariant information that improves classification. The mathematical analysis of wavelet scattering networks explains important properties of deep convolution networks for classification. A scattering representation of stationary processes incorporates higher order moments and can thus discriminate textures having the same Fourier power spectrum. State-of-the-art classification results are obtained for handwritten digits and texture discrimination, with a Gaussian kernel SVM and a generative PCA classifier. Facial Expression Recognition (FER) is one of the most important topics in the domain of computer vision and pattern recognition, and it has attracted increasing attention for its scientific challenges and application potentials. In this article, we propose a novel and effective approach to FER using multi-model two-dimensional (2D) and 3D videos, which encodes both static and dynamic clues by scattering convolution network. First, a shape-based detection method is introduced to locate the start and the end of an expression in videos; segment its onset, apex, and offset states; and sample the important frames for emotion analysis. Second, the frames in Apex of 2D videos are represented by scattering, conveying static texture details. Those of 3D videos are processed in a similar way, but to highlight static shape details, several geometric maps in terms of multiple order differential quantities, i.e., Normal Maps and Shape Index Maps, are generated as the input of scattering, instead of original smooth facial surfaces. Third, the average of neighboring samples centred at each key texture frame or shape map in Onset is computed, and the scattering features extracted from all the average samples of 2D and 3D videos are then concatenated to capture dynamic texture and shape cues, respectively. Finally, Multiple Kernel Learning is adopted to combine the features in the 2D and 3D modalities and compute similarities to predict the expression label. Thanks to the scattering descriptor, the proposed approach not only encodes distinct local texture and shape variations of different expressions as by several milestone operators, such as SIFT, HOG, and so on, but also captures subtle information hidden in high frequencies in both channels, which is quite crucial to better distinguish expressions that are easily confused. The validation is conducted on the BU-4DFE and BP-4D databa ses, and the accuracies reached are very competitive, indicating its competency for this issue. Facial Expression Recognition (FER) is one of the most active topics in the domain of computer vision and pattern recognition, and it has received increasing attention for its wide application potentials as well as attractive scientific challenges. In this paper, we present a novel method to automatic 3D FER based on geometric scattering representation. A set of maps of shape features in terms of multiple order differential quantities, i.e. the Normal Maps (NOM) and the Shape Index Maps (SIM), are first jointly adopted to comprehensively describe geometry attributes of the facial surface. The scattering operator is then introduced to further highlight expression related cues on these maps, thereby constructing geometric scattering representations of 3D faces for classification. The scattering descriptor not only encodes distinct local shape changes of various expressions as by several milestone descriptors, such as SIFT, HOG, etc., but also captures subtle information hidden in high frequencies, which is quite crucial to better distinguish expressions that are easily confused. We evaluate the proposed approach on the BU-3DFE database, and the performance is up to 84.8 and 82.7 with two commonly used protocols respectively which is superior to the state of the art ones.