Emotion estimation in crowds: a survey <s> Affective Datasets <s> Abstract Emotions are viewed as having evolved through their adaptive value in dealing with fundamental life-tasks. Each emotion has unique features: signal, physiology, and antecedent events. Each emotion also has characteristics in common with other emotions: rapid onset, short duration, unbidden occurrence, automatic appraisal, and coherence among responses. These shared and unique characteristics are the product of our evolution, and distinguish emotions from other affective phenomena. <s> BIB001 </s> Emotion estimation in crowds: a survey <s> Affective Datasets <s> Cross-cultural research on facial expression and the developments of methods to measure facial expression are briefly summarized. What has been learned about emotion from this work on the face is then elucidated. Four questions about facial expression and emotion are discussed. What information does an expression typically convey? Can there be emotion without facial expression? Can there be a facial expression of emotion without emotion? How do individuals differ in their facial expressions of emotion? <s> BIB002 </s> Emotion estimation in crowds: a survey <s> Affective Datasets <s> We present a multimodal data set for the analysis of human affective states. The electroencephalogram (EEG) and peripheral physiological signals of 32 participants were recorded as each watched 40 one-minute long excerpts of music videos. Participants rated each video in terms of the levels of arousal, valence, like/dislike, dominance, and familiarity. For 22 of the 32 participants, frontal face video was also recorded. A novel method for stimuli selection is proposed using retrieval by affective tags from the last.fm website, video highlight detection, and an online assessment tool. An extensive analysis of the participants' ratings during the experiment is presented. Correlates between the EEG signal frequencies and the participants' ratings are investigated. Methods and results are presented for single-trial classification of arousal, valence, and like/dislike ratings using the modalities of EEG, peripheral physiological signals, and multimedia content analysis. Finally, decision fusion of the classification results from different modalities is performed. The data set is made publicly available and we encourage other researchers to use it for testing their own affective state estimation methods. <s> BIB003 </s> Emotion estimation in crowds: a survey <s> Affective Datasets <s> The ICML 2013 Workshop on Challenges in Representation Learning11http://deeplearning.net/icml2013-workshop-competition. focused on three challenges: the black box learning challenge, the facial expression recognition challenge, and the multimodal learning challenge. We describe the datasets created for these challenges and summarize the results of the competitions. We provide suggestions for organizers of future challenges and some comments on what kind of knowledge can be gained from machine learning competitions. <s> BIB004 </s> Emotion estimation in crowds: a survey <s> Affective Datasets <s> To contribute to the need for emotional databases and affective tagging, the LIRIS-ACCEDE is proposed in this paper. LIRIS-ACCEDE is an Annotated Creative Commons Emotional DatabasE composed of 9800 video clips extracted from 160 movies shared under Creative Commons licenses. It allows to make this database publicly available without copyright issues. The 9800 video clips (each 8-12 seconds long) are sorted along the induced valence axis, from the video perceived the most negatively to the video perceived the most positively. The annotation was carried out by 1518 annotators from 89 different countries using crowd sourcing. A baseline late fusion scheme using ground truth from annotations is computed to predict emotion categories in video clips. <s> BIB005 </s> Emotion estimation in crowds: a survey <s> Affective Datasets <s> People convey their emotional state in their face and voice. We present an audio-visual dataset uniquely suited for the study of multi-modal emotion expression and perception. The dataset consists of facial and vocal emotional expressions in sentences spoken in a range of basic emotional states (happy, sad, anger, fear, disgust, and neutral). 7,442 clips of 91 actors with diverse ethnicbackgrounds were rated by multiple raters in three modalities: audio, visual, and audio-visual. Categorical emotion labels andreal-value intensity values for the perceived emotion were collected using crowd-sourcing from 2,443 raters. The human recognition of intended emotion for the audio-only, visual-only, and audio-visual data are 40.9, 58.2 and 63.6 percent respectively. Recognition rates are highest for neutral, followed by happy, anger, disgust, fear, and sad. Average intensity levels of emotion are rated highest forvisual-only perception. The accurate recognition of disgust and fear requires simultaneous audio-visual cues, while anger andhappiness can be well recognized based on evidence from a single modality. The large dataset we introduce can be used to probe other questions concerning the audio-visual perception of emotion. <s> BIB006 </s> Emotion estimation in crowds: a survey <s> Affective Datasets <s> Emotion detection using facial images is a technique that researchers have been using for the last two decades to try to analyze a person's emotional state given his/her image. Detection of various kinds of emotion using facial expressions of students in educational environment is useful in providing insight into the effectiveness of tutoring sessions. Robust recognition with conventional 2D cameras is still not possible under realistic conditions, ie, in the presence of variation in lighting and illumination and pose variations of the participants. This paper discusses in brief how facial expression recognition from 3D models could be achieved for applications under various educational environments. 3D Face Model dataset for various facial expressions of about 95 undergraduate students of the Department of Information Science and Engineering was collected. The new dataset produced comprise 3D models of emotions represented by five groups of facial expressions, namely: normal, happy, sad, surprised and angry, which are mainly seen among students during their college days. These data were collected using a Kinect camera, which also gives the depth map for the facial expressions. The dataset collected is a point cloud. The Kinect camera measures a large number of points on the face of the person and this output is stored as a point cloud data ( PCD) file. The point cloud thus represents the sets of points that the camera has measured. The data are stored in the PCD file format, which supports the 3D point cloud data, and which can be viewed using any point cloud library ( PCL) viewer. [ABSTRACT FROM AUTHOR] <s> BIB007 </s> Emotion estimation in crowds: a survey <s> Affective Datasets <s> In order to create an "in-the-wild" dataset of facial emotions with large number of balanced samples, this paper proposes a game-based data collection framework. The framework mainly include three components: a game engine, a game interface, and a data collection and evaluation module. We use a deep learning approach to build an emotion classifier as the game engine. Then a emotion web game to allow gamers to enjoy the games, while the data collection module obtains automatically-labelled emotion images. Using our game, we have collected more than 15,000 images within a month of the test run and built an emotion dataset "GaMo". To evaluate the dataset, we compared the performance of two deep learning models trained on both GaMo and CIFE. The results of our experiments show that because of being large and balanced, GaMo can be used to build a more robust emotion detector than the emotion detector trained on CIFE, which was used in the game engine to collect the face images. <s> BIB008 </s> Emotion estimation in crowds: a survey <s> Affective Datasets <s> Unobtrusive recognition of the user's mood is an essential capability for affect-adaptive systems. Mood is a subtle, long-term affective state, often misrecognized even by humans. The challenge to train a machine to recognize it from, for example, a video of the user, is significant, and already begins with the lack of ground truth for supervised learning. Existing affective databases consist mainly of short videos, annotated in terms of expressed emotions rather than mood. In very few cases, we encounter perceived mood annotations, of questionable reliability, however, due to the subjectivity of mood estimation and the small number of coders involved. In this work, we introduce a new database for mood recognition from video. Our database contains 180 long, acted videos, depicting typical daily scenarios, and subtle facial and bodily expressions. The videos cover three visual modalities (face, body, Kinect data), and are annotated in terms of emotions (via G-trace) and mood (via the Self-Assessment Manikin and the AffectButton). To annotate the database exhaustively, we exploit crowdsourcing to reach out to an extensive number of nonexpert coders. We validate the reliability of our crowdsourced annotations by (1) adopting a number of criteria to filter out unreliable coders, and (2) comparing the annotations of a subset of our videos with those collected in a controlled lab setting. <s> BIB009
This subsection considers datasets intended for all kinds of affective tasks, not limiting to those fitted for crowds and using different sensors and ground truth formats in order to provide a comprehensive view of the available options. Delving in the task of detecting emotional states, facial expressions have become a popular choice due to their universality and intrinsic relation to emotions BIB002 . By means of conventional cameras and in a controlled environment, the datasets CMU   and FER-2013 BIB004 collected static images of facial expressions from participants who were requested to act different emotional states following the discrete emotions scheme BIB001 . Aiming to simplify the collection of static images and to reach a greater number of participants, the authors of the Gamo dataset BIB008 made use of a web-based interface were participants play a game by performing specific facial expressions captured by a web camera. Progressing from static images only, the CK+ dataset  provides sequences of images where participants enact a series of facial expressions, and emotions are described in terms of facial action units  . Expanding the scope of behavioral markers, the dataset CREMA-D BIB006 contains short videos of participants displaying facial and vocal expressions for the study of multi-modal emotion expression and perception, whereas the dataset LIRIS-ACCEDE BIB005 [22] goes one step further and captures body expressions. However, as the authors in BIB007 argue, using conventional 2D cameras lacks robustness as this kind of cameras are subject to poor illumination and changes in lighting conditions. In response, they propose the use of Kinetic cameras as these are able to capture depth, and produce a dataset containing 3D models of several participants performing multiple facial expressions. Moving from emotions (brief affective states) to moods (long term affective states), the work in BIB009 introduces the EMMA database which employs both 2D and kinetic cameras, and provides longer intervals of data capture as the dataset is intended for mood recognition. Focusing on physiological measurements, the DEAP dataset presented in BIB003 [26] collected the electroencephalogram (EEG), electrooculogram (EOG), Galvanic skin response (GSR), blood volume pressure (BVP), temperature and respiration signals of participants. And a frontal video face was recorded for some of those participants. One-minute long excerpts of music videos were used as the stimulus to elicit emotions along the four quadrants of the arousal-valence plane.