Emotion in Context: Deep Semantic Feature Fusion for Video Emotion Recognition
Human emotions demonstrate high correlations with certain events that consist of the interaction of objects, and are usually constrained by particular scenes. In this paper, we exploit these abundant context clues for video emotion recognition. We first compute event, object and scene scores with state of the art detectors based on deep neural networks. The extracted high level features serve as effective contextual information demonstrating what is occurring in the video, and are further integrated in a context fusion network to generate a unified representation aiming to bridge the affective gap. The contribution of this paper is three fold: (a) we are the first to incorporate event as context for emotion recognition, and we demonstrate its superiority for emotion understanding, (b) we utilize a context fusion network to exploit a comprehensive set of high level semantics features as contextual clues and (c) the proposed framework can realize recognition in real time and achieves state of the art performance on two challenging emotion recognition benchmarks, 50.6 on VideoEmotion and 51.8 on Ekman.
ACM Multimedia