Facial expressions are combinations of basic components called Action Units (AU). Recognizing AUs is key for developing general facial expression analysis. In recent years, most efforts in automatic AU recognition have been dedicated to learning combinations of local features and to exploiting correlations between Action Units. In this paper, we propose a deep neural architecture that tackles both problems by combining learned local and global features in its initial stages and replicating a message passing algorithm between classes similar to a graphical model inference approach in later stages. We show that by training the model end-to-end with increased supervision we improve state-of-the-art by 5.3 and 8.2 performance on BP4D and DISFA datasets, respectively.
Alternative methods have looked into fusing multi-scale temporal decisions or learning separable representations. @cite_24 proposed the fusion of complementary classifiers on frame-level detection, segment-level detection (detecting AU segments from contiguous frames) and transition detection (recognizing transitions between AU and non-AU frames). @cite_32 proposed an ensemble of classifiers that makes decisions in an easy-to-hard fashion using predefined shallow features extracted around facial landmarks. @cite_25 proposed a feature disentangling machine that is capable of selecting features into non-overlapped groups. They focus on common features that are shared across different expressions and expression-specific features that are discriminative only for a target expression.
@cite_24 @cite_25 @cite_32
2162772395 1594998705
Automatic facial Action Unit (AU) detection from video is a long-standing problem in facial expression analysis. AU detection is typically posed as a classification problem between frames or segments of positive examples and negative ones, where existing work emphasizes the use of different features or classifiers. In this paper, we propose a method called Cascade of Tasks (CoT) that combines the use of different tasks (i.e., frame, segment and transition) for AU event detection. We train CoT in a sequential manner embracing diversity, which ensures robustness and generalization to unseen data. In addition to conventional frame based metrics that evaluate frames independently, we propose a new event-based metric to evaluate detection performance at event-level. We show how the CoT method consistently outperforms state-of-the-art approaches in both frame-based and event-based metrics, across three public datasets that differ in complexity: CK+, FERA and RU-FACS. We present a novel Bayesian framework for facial action unit recognition. The first key observation behind this work is sparsity: out of possible 45 (and more) facial action units, only very few are active at any moment. The second is the strong statistical co-occurrence structure: most facial expressions are made by common combinations of facial action units, so knowing the presence of one can act as a strong prior for inferring the presence of others. We developed a novel Bayesian graphical model that encodes these two natural aspects of facial action units via compressed sensing and group-wise sparsity inducing priors. One crucial aspect of our approach is the allowance of overlapping group structures, which proves useful in dealing with action units that occur frequently across multiple groups. We derive an efficient inference scheme and show how such sparsity and co-occurrence can be automatically learned from data. Experiments on three standard benchmark datasets show superiority over the state-of-the-art.