Frame-level visual features are generally aggregated in time with the techniques such as LSTM, Fisher Vectors, NetVLAD etc. to produce a robust video-level representation. We here introduce a learnable aggregation technique whose primary objective is to retain short-time temporal structure between frame-level features and their spatial interdependencies in the representation. Also, it can be easily adapted to the cases where there have very scarce training samples. We evaluate the method on a real-fake expression prediction dataset to demonstrate its superiority. Our method obtains 65 score on the test dataset in the official MAP evaluation and there is only one misclassified decision with the best reported result in the Chalearn Challenge (i.e. 66:7 ) . Lastly, we believe that this method can be extended to different problems such as action event recognition in future.
Even if there are vast numbers of studies on facial emotion detection, the prediction of real-fake expression is limited and most of which are for still images @cite_36 . Also, they are usually based on hand-crafted features. Therefore, reviewing the literature from the perspective of cognitive science might be more informative in order to understand and propose a proper technique for real-fake expression prediction. In particular, @cite_38 @cite_31 show that small emotional changes in eyes and mount movements can be used to interpret an expression as genuine (real) or deceptive (fake) from the sequence of faces. This indicates that both spatial interdependencies of face parts and their sequential structure should be considered in the final model.
@cite_36 @cite_38 @cite_31
2005733474 2282546360
We present initial results from the application of an automated facial expression recognition system to spontaneous facial expressions of pain. In this study, 26 participants were videotaped under three experimental conditions: baseline, posed pain, and real pain. The real pain condition consisted of cold pressor pain induced by submerging the arm in ice water. Our goal was to (1) assess whether the automated measurements were consistent with expression measurements obtained by human experts, and (2) develop a classifier to automatically differentiate real from faked pain in a subject-independent manner from the automated measurements. We employed a machine learning approach in a two-stage system. In the first stage, a set of 20 detectors for facial actions from the Facial Action Coding System operated on the continuous video stream. These data were then passed to a second machine learning stage, in which a classifier was trained to detect the difference between expressions of real pain and fake pain. Naive human subjects tested on the same videos were at chance for differentiating faked from real pain, obtaining only 49 accuracy. The automated system was successfully able to differentiate faked from real pain. In an analysis of 26 subjects with faked pain before real pain, the system obtained 88 correct for subject independent discrimination of real versus fake pain on a 2-alternative forced choice. Moreover, the most discriminative facial actions in the automated system were consistent with findings using human expert FACS codes. Hiding true emotions: micro-expressions in eyes retrospectively concealed by mouth movements