The measurement and analysis of electrodermal activity (EDA) offers applications in diverse areas ranging from market research to seizure detection and to human stress analysis. Unfortunately, the analysis of EDA signals is made difficult by the superposition of numerous components that can obscure the signal information related to a user's response to a stimulus. We show how simple preprocessing followed by a novel compressed sensing based decomposition can mitigate the effects of the undesired noise components and help reveal the underlying physiological signal. The proposed framework allows for decomposition of EDA signals with provable bounds on the recovery of user responses. We test our procedure on both synthetic and real-world EDA signals from wearable sensors and demonstrate that our approach allows for more accurate recovery of user responses as compared with the existing techniques.
The study of Electrodermal Activity signals, or EDA signals, dates back to the early @math century ( e.g., @cite_5 ) with the observation of a connection between changes in user skin conductance and psychological state. In recent years, this connection has been validated by examining brain function via fMRI and skin conduction via EDA concurrently in @cite_18 , and by showing the specific regions of the brain that correspond with EDA changes and video recordings of sweat glands in @cite_9 . The promise of EDA as a window into user psychology resulted in extensive work on evaluating the connection between EDA and user interactions @cite_4 , stress detection @cite_8 , content and audience segmentation @cite_7 , and reaction to video content @cite_30 ---to name only a few.
@cite_30 @cite_18 @cite_4 @cite_7 @cite_8 @cite_9 @cite_5
1239497932 2041172791 2099281997 2043464897 2150150736 2041347087 1967993123
Goal: This paper reports on a novel algorithm for the analysis of electrodermal activity (EDA) using methods of convex optimization. EDA can be considered as one of the most common observation channels of sympathetic nervous system activity, and manifests itself as a change in electrical properties of the skin, such as skin conductance (SC). Methods: The proposed model describes SC as the sum of three terms: the phasic component, the tonic component, and an additive white Gaussian noise term incorporating model prediction errors as well as measurement errors and artifacts. This model is physiologically inspired and fully explains EDA through a rigorous methodology based on Bayesian statistics, mathematical convex optimization, and sparsity. Results: The algorithm was evaluated in three different experimental sessions to test its robustness to noise, its ability to separate and identify stimulus inputs, and its capability of properly describing the activity of the autonomic nervous system in response to strong affective stimulation. Significance: Results are very encouraging, showing good performance of the proposed method and suggesting promising future applicability, e.g., in the field of affective computing. Biometric sensors and portable devices are being increasingly embedded into our everyday life, creating the need for robust physiological models that efficiently represent, analyze, and interpret the acquired signals. We propose a knowledge-driven method to represent electrodermal activity (EDA), a psychophysiological signal linked to stress, affect, and cognitive processing. We build EDA-specific dictionaries that accurately model both the slow varying tonic part and the signal fluctuations, called skin conductance responses (SCR), and use greedy sparse representation techniques to decompose the signal into a small number of atoms from the dictionary. Quantitative evaluation of our method considers signal reconstruction, compression rate, and information retrieval measures, that capture the ability of the model to incorporate the main signal characteristics, such as SCR occurrences. Compared to previous studies fitting a predetermined structure to the signal, results indicate that our approach provides benefits across all aforementioned criteria. This paper demonstrates the ability of appropriate dictionaries along with sparse decomposition methods to reliably represent EDA signals and provides a foundation for automatic measurement of SCR characteristics and the extraction of meaningful EDA features. The ability to assess fine-scale user responses has applications in advertising, content creation, recommendation, and psychology research. Unfortunately, current approaches, such as focus groups and audience surveys, are limited in size and scope. In this paper, we propose a combined biometric sensing and analysis methodology to leverage audience-scale electro-dermal activity (EDA) data for the purpose of evaluating user responses to video. We provide detailed characterization of how temporal physiological responses to video stimulus can be modeled, along with first-of-its-kind audience-scale EDA group experiments in uncontrolled real-world environments. Our study provides insights into the techniques used to analyze EDA, the effectiveness of the different temporal features, and group dynamics of audiences. Our experiments demonstrate the ability to classify movie ratings with accuracy of over 70 on specific films. Results of this study suggest the ability to assess emotional reactions of groups using minimally invasive sensing modalities in uncontrolled environments. Abstract In this study, we investigated the spatio-temporal patterns of category-specific cortical activation elicited by the visual presentation of words whose meaning relates to a color or to a visual form or shape. We recorded the event-related potentials (ERPs) of ten healthy, right-handed volunteers while they passively read words presented tachistoscopically. As early as 150 ms after stimulus onset, the ERPs revealed significant neurophysiological differences between words and strings of hash marks. Around 200 ms after stimulus onset, we found significant differences in the ERPs elicited by color- and form-related words. We used minimum norm current estimates to investigate the spatial location of these differences. This revealed that, at the 150 ms peak, the activation advantage of words over sequences of hash marks was located in a left posterior area, proximal to what has previously been called the visual word form area. At the 200 ms peak, the advantage of words related to colors over words related to forms seemed to reside in temporal cortical areas, whereas the form-related words elicited greater activation in frontal areas than color-related words. These results provide evidence for early access to detailed category-specific representations of word meaning, with subtle differences in meaning being reflected in the activation of different cortical areas, as early as 200 ms after stimulus presentation. In line with previous studies, these differences can be related to the areas involved in the conceptual processing of sensory (visual) and action-related information. Signals from peripheral physiology (e.g., ECG, EMG, and GSR) in conjunction with machine learning techniques can be used for the automatic detection of affective states. The affect detector can be user-independent, where it is expected to generalize to novel users, or user-dependent, where it is tailored to a specific user. Previous studies have reported some success in detecting affect from physiological signals, but much of the work has focused on induced affect or acted expressions instead of contextually constrained spontaneous expressions of affect. This study addresses these issues by developing and evaluating user-independent and user-dependent physiology-based detectors of nonbasic affective states (e.g., boredom, confusion, curiosity) that were trained and validated on naturalistic data collected during interactions between 27 students and AutoTutor, an intelligent tutoring system with conversational dialogues. There is also no consensus on which techniques (i.e., feature selection or classification methods) work best for this type of data. Therefore, this study also evaluates the efficacy of affect detection using a host of feature selection and classification techniques on three physiological signals (ECG, EMG, and GSR) and their combinations. Two feature selection methods and nine classifiers were applied to the problem of recognizing eight affective states (boredom, confusion, curiosity, delight, flow -engagement, surprise, and neutral). The results indicated that the user-independent modeling approach was not feasible; however, a mean kappa score of 0.25 was obtained for user-dependent models that discriminated among the most frequent emotions. The results also indicated that k-nearest neighbor and Linear Bayes Normal Classifier (LBNC) classifiers yielded the best affect detection rates. Single channel ECG, EMG, and GSR and three-channel multimodal models were generally more diagnostic than two--channel models. In this paper, we propose an approach for affective ranking of movie scenes based on the emotions that are actually felt by spectators. Such a ranking can be used for characterizing the affective, or emotional, content of video clips. The ranking can for instance help determine which video clip from a database elicits, for a given user, the most joy. This in turn will permit video indexing and retrieval based on affective criteria corresponding to a personalized user affective profile. A dataset of 64 different scenes from 8 movies was shown to eight participants. While watching, their physiological responses were recorded; namely, five peripheral physiological signals (GSR - galvanic skin resistance, EMG - electromyograms, blood pressure, respiration pattern, skin temperature) were acquired. After watching each scene, the participants were asked to self-assess their felt arousal and valence for that scene. In addition, movie scenes were analyzed in order to characterize each with various audio- and video-based features capturing the key elements of the events occurring within that scene. Arousal and valence levels were estimated by a linear combination of features from physiological signals, as well as by a linear combination of content-based audio and video features. We show that a correlation exists between arousal- and valence-based rankings provided by the spectator's self-assessments, and rankings obtained automatically from either physiological signals or audio-video features. This demonstrates the ability of using physiological responses of participants to characterize video scenes and to rank them according to their emotional content. This further shows that audio-visual features, either individually or combined, can fairly reliably be used to predict the spectator's felt emotion for a given scene. The results also confirm that participants exhibit different affective responses to movie scenes, which emphasizes the need for the emotional profiles to be user-dependant. Emotions play an important role in how we select and consume multimedia. Recent advances on affect detection are focused on detecting emotions continuously. In this paper, for the first time, we continuously detect valence from electroencephalogram (EEG) signals and facial expressions in response to videos. Multiple annotators provided valence levels continuously by watching the frontal facial videos of participants who watched short emotional videos. Power spectral features from EEG signals as well as facial fiducial points are used as features to detect valence levels for each frame continuously. We study the correlation between features from EEG and facial expressions with continuous valence. We have also verified our model's performance for the emotional highlight detection using emotion recognition from EEG signals. Finally the results of multimodal fusion between facial expression and EEG signals are presented. Having such models we will be able to detect spontaneous and subtle affective responses over time and use them for video highlight detection.