Pain is a personal, subjective experience that is commonly evaluated through visual analog scales (VAS). While this is often convenient and useful, automatic pain detection systems can reduce pain score acquisition efforts in large-scale studies by estimating it directly from the participants' facial expressions. In this paper, we propose a novel two-stage learning approach for VAS estimation: first, our algorithm employs Recurrent Neural Networks (RNNs) to automatically estimate Prkachin and Solomon Pain Intensity (PSPI) levels from face images. The estimated scores are then fed into the personalized Hidden Conditional Random Fields (HCRFs), used to estimate the VAS, provided by each person. Personalization of the model is performed using a newly introduced facial expressiveness score, unique for each person. To the best of our knowledge, this is the first approach to automatically estimate VAS from face images. We show the benefits of the proposed personalized over traditional non-personalized approach on a benchmark dataset for pain analysis from face images.
Although it is well known that pain induces changes in autonomic parameters, the extent to which these changes correlate with the experience of pain is under debate. The aim of the present study was to compare a combination of multiple autonomic parameters and each parameter alone in their ability to differentiate among 4 categories of pain intensity. Tonic heat stimuli (1 minute) were individually adjusted to induce no pain, low, medium, and high pain in 45 healthy volunteers. Electrocardiogram, photoplethysmogram, and galvanic skin response were recorded, and the following parameters were calculated: heart rate; heart rate variability—high frequency (0.15 to 0.4 Hz) spectral power; skin conductance level; number of skin conduction fluctuations; and photoplethysmographic pulse wave amplitude. A combination of parameters was created by fitting an ordinal cumulative logit model to the data and using linear coefficients of the model. Friedman test with post-hoc Wilcoxon test were used to compare between pain intensity categories for every parameter alone and for their linear combination. All of the parameters successfully differentiated between no pain and all other pain categories. However, none of the parameters differentiated between all 3 pain categories (i.e., low and medium; medium and high; low and high). In contrast, the linear combination of parameters significantly differentiated not only between pain and no pain, but also between all pain categories (P < .001 to .02). These results suggest that multiparameter approaches should be further investigated to make progress toward reliable autonomic-based pain assessment. A major factor hindering the deployment of a fully functional automatic facial expression detection system is the lack of representative data. A solution to this is to narrow the context of the target application, so enough data is available to build robust models so high performance can be gained. Automatic pain detection from a patient's face represents one such application. To facilitate this work, researchers at McMaster University and University of Northern British Columbia captured video of participant's faces (who were suffering from shoulder pain) while they were performing a series of active and passive range-of-motion tests to their affected and unaffected limbs on two separate occasions. Each frame of this data was AU coded by certified FACS coders, and self-report and observer measures at the sequence level were taken as well. This database is called the UNBC-McMaster Shoulder Pain Expression Archive Database. To promote and facilitate research into pain and augment current datasets, we have publicly made available a portion of this database which includes: 1) 200 video sequences containing spontaneous facial expressions, 2) 48,398 FACS coded frames, 3) associated pain frame-by-frame scores and sequence-level self-report and observer measures, and 4) 66-point AAM landmarks. This paper documents this data distribution in addition to describing baseline results of our AAM SVM system. This data will be available for distribution in March 2011. The purpose of this study was to use functional near-infrared spectroscopy (fNIRS) to examine patterns of both activation and deactivation that occur in the frontal lobe in response to noxious stimuli. The frontal lobe was selected because it has been shown to be activated by noxious stimuli in functional magnetic resonance imaging studies. The brain region is located behind the forehead which is devoid of hair, providing a relative ease of placement for fNIRS probes on this area of the head. Based on functional magnetic resonance imaging studies showing blood-oxygenation-level dependent changes in the frontal lobes, we evaluated functional near-infrared spectroscopy measures in response to two levels of electrical pain in awake, healthy human subjects (n = 10; male = 10). Each subject underwent two recording sessions separated by a 30-minute resting period. Data collected from 7 subjects were analyzed, containing a total of 38 36 low high intensity pain stimuli for the first recording session and 27 31 pain stimuli for the second session. Our results show that there is a robust and significant deactivation in sections of the frontal cortices. Further development and definition of the specificity and sensitivity of the approach may provide an objective measure of nociceptive activity in the brain that can be easily applied in the surgical setting.   Psychologists have long explored mechanisms with which humans recognize other humans' affective states from modalities, such as voice and face display. This exploration has led to the identification of the main mechanisms, including the important role played in the recognition process by the modalities' dynamics. Constrained by the human physiology, the temporal evolution of a modality appears to be well approximated by a sequence of temporal segments called onset, apex, and offset. Stemming from these findings, computer scientists, over the past 15 years, have proposed various methodologies to automate the recognition process. We note, however, two main limitations to date. The first is that much of the past research has focused on affect recognition from single modalities. The second is that even the few multimodal systems have not paid sufficient attention to the modalities' dynamics: The automatic determination of their temporal segments, their synchronization to the purpose of modality fusion, and their role in affect recognition are yet to be adequately explored. To address this issue, this paper focuses on affective face and body display, proposes a method to automatically detect their temporal segments or phases, explores whether the detection of the temporal phases can effectively support recognition of affective states, and recognizes affective states based on phase synchronization alignment. The experimental results obtained show the following: 1) affective face and body displays are simultaneous but not strictly synchronous; 2) explicit detection of the temporal phases can improve the accuracy of affect recognition; 3) recognition from fused face and body modalities performs better than that from the face or the body modality alone; and 4) synchronized feature-level fusion achieves better performance than decision-level fusion. Depression is one of the most common mental health disorders with strong adverse effects on personal and social functioning. The absence of any objective diagnostic aid for depression leads to a range of subjective biases in initial diagnosis and ongoing monitoring. Psychologists use various visual cues in their assessment to quantify depression such as facial expressions, eye contact and head movements. This paper studies the contribution of (upper) body expressions and gestures for automatic depression analysis. A framework based on space-time interest points and bag of words is proposed for the analysis of upper body and facial movements. Salient interest points are selected using clustering. The major contribution of this paper lies in the creation of a bag of body expressions and a bag of facial dynamics for assessing the contribution of different body parts for depression analysis. Head movement analysis is performed by selecting rigid facial fiducial points and a new histogram of head movements is proposed. The experiments are performed on real-world clinical data where video clips of patients and healthy controls are recorded during interactive interview sessions. The results show the effectiveness of the proposed system to evaluate the contribution of various body parts in depression analysis.   Automatic pain recognition can improve medical treatment, especially when the patient is not able to utter on his pain experience. Facial expressions with their intensities and dynamics contain valuable information for recognising pain. We propose a concept for distinguishing facial expressions of pain from others and assessing the pain expression intensity. It is based on a Support Vector Machine (SVM) classifier and a function model for intensity rating. The intensity model is trained using Comparative Learning, a new technique that simplifies labelling of the data. Using a database of 3D posed pain sequences we show the suitability of the concept to recognise pain expressions, distinguish different intensities and spot even slight intensity changes in its temporal context.