Limited annotated data available for the recognition of facial expression and action units embarrasses the training of deep networks, which can learn disentangled invariant features. However, a linear model with just several parameters normally is not demanding in terms of training data. In this paper, we propose an elegant linear model to untangle confounding factors in challenging realistic multichannel signals such as 2D face videos. The simple yet powerful model does not rely on huge training data and is natural for recognizing facial actions without explicitly disentangling the identity. Base on well-understood intuitive linear models such as Sparse Representation based Classification (SRC), previous attempts require a prepossessing of explicit decoupling which is practically inexact. Instead, we exploit the low-rank property across frames to subtract the underlying neutral faces which are modeled jointly with sparse representation on the action components with group sparsity enforced. On the extended Cohn-Kanade dataset (CK+), our one-shot automatic method on raw face videos performs as competitive as SRC applied on manually prepared action components and performs even better than SRC in terms of true positive rate. We apply the model to the even more challenging task of facial action unit recognition, verified on the MPI Face Video Database (MPI-VDB) achieving a decent performance. All the programs and data have been made publicly available.
Particularly for facial actions, we treat videos as multichannel signals @cite_12 @cite_14 , different from image-based methods @cite_0 @cite_28 . @cite_0 explicitly separates the neutral face and action component, and then exploits the class-wise sparsity separately for the recognition of identity from neutral faces and expression from action components. Differently, with the focus of facial actions we exploit the low-rank property for disentangling identity as well as structured sparsity by inter-channel observation. Furthermore, there is tradeoff between simplicity and performance. As videos are sequential signals, the above appearance-based methods including ours cannot model the dynamics given by a temporal model @cite_27 or spatio-temporal models @cite_7 @cite_11 @cite_19 . Other linear models include ordinal regression @cite_15 @cite_9 @cite_21 and boosting @cite_18 .
@cite_18 @cite_14 @cite_7 @cite_28 @cite_9 @cite_21 @cite_0 @cite_19 @cite_27 @cite_15 @cite_12 @cite_11
2101965618 2487442924 1985820353 2104563967
In this work we report on the progress of building a system that enables fully automated fast and robust facial expression recognition from face video. We analyse subtle changes in facial expression by recognizing facial muscle action units (AUs) and analysing their temporal behavior. By detecting AUs from face video we enable the analysis of various facial communicative signals including facial expressions of emotion, attitude and mood. For an input video picturing a facial expression we detect per frame whether any of 15 different AUs is activated, whether that facial action is in the onset, apex, or offset phase, and what the total duration of the activation in question is. We base this process upon a set of spatio-temporal features calculated from tracking data for 20 facial fiducial points. To detect these 20 points of interest in the first frame of an input face video, we utilize a fully automatic, facial point localization method that uses individual feature GentleBoost templates built from Gabor wavelet features. Then, we exploit a particle filtering scheme that uses factorized likelihoods and a novel observation model that combines a rigid and a morphological model to track the facial points. The AUs displayed in the input video and their temporal segments are recognized finally by Support Vector Machines trained on a subset of most informative spatio-temporal features selected by AdaBoost. For Cohn-Kanade andMMI databases, the proposed system classifies 15 AUs occurring alone or in combination with other AUs with a mean agreement rate of 90.2 with human FACS coders. In this paper, we present an approach for learning a visual representation from the raw spatiotemporal signals in videos. Our representation is learned without supervision from semantic labels. We formulate our method as an unsupervised sequential verification task, i.e., we determine whether a sequence of frames from a video is in the correct temporal order. With this simple task and no semantic labels, we learn a powerful visual representation using a Convolutional Neural Network (CNN). The representation contains complementary information to that learned from supervised image datasets like ImageNet. Qualitative results show that our method captures information that is temporally varying, such as human pose. When used as pre-training for action recognition, our method gives significant gains over learning without external data on benchmark datasets like UCF101 and HMDB51. To demonstrate its sensitivity to human pose, we show results for pose estimation on the FLIC and MPII datasets that are competitive, or better than approaches using significantly more supervision. Our method can be combined with supervised representations to provide an additional boost in accuracy. In this paper we propose an adaptive part-based spatio-temporal model that characterizes person's appearance using color and facial features. Face image selection based on low level cues is used to select usable face images to build a face model. Color features that capture the distribution of colors as well as the representative colors are used to build the color model. The model is built over a sequence of frames of an individual and hence captures the characteristic appearance as well as its variations over time. We also address the problem of multiple person re-identification in the absence of calibration data or prior knowledge about the camera layout. Multiple person re-identification is a open set matching problem with a dynamically evolving and open gallery set and an open probe set. Re-identification is posed as a rectangular assignment problem and is solved to find a bijection that minimizes the overall assignment cost. Open and closed set re-identification is tested on 30 videos collected with nine non-overlapping cameras spanning outdoor and indoor areas, with 40 subjects under observation. A false acceptance reduction scheme based on the developed model is also proposed. Images of facial expressions are often captured from various views as a result of either head movements or variable camera position. Existing methods for multiview and or view-invariant facial expression recognition typically perform classification of the observed expression using either classifiers learned separately for each view or a single classifier learned for all views. However, these approaches ignore the fact that different views of a facial expression are just different manifestations of the same facial expression. By accounting for this redundancy, we can design more effective classifiers for the target task. To this end, we propose a discriminative shared Gaussian process latent variable model (DS-GPLVM) for multiview and view-invariant classification of facial expressions from multiple views. In this model, we first learn a discriminative manifold shared by multiple views of a facial expression. Subsequently, we perform facial expression classification in the expression manifold. Finally, classification of an observed facial expression is carried out either in the view-invariant manner (using only a single view of the expression) or in the multiview manner (using multiple views of the expression). The proposed model can also be used to perform fusion of different facial features in a principled manner. We validate the proposed DS-GPLVM on both posed and spontaneously displayed facial expressions from three publicly available datasets (MultiPIE, labeled face parts in the wild, and static facial expressions in the wild). We show that this model outperforms the state-of-the-art methods for multiview and view-invariant facial expression classification, and several state-of-the-art methods for multiview learning and feature fusion.