Pain is a personal, subjective experience that is commonly evaluated through visual analog scales (VAS). While this is often convenient and useful, automatic pain detection systems can reduce pain score acquisition efforts in large-scale studies by estimating it directly from the participants' facial expressions. In this paper, we propose a novel two-stage learning approach for VAS estimation: first, our algorithm employs Recurrent Neural Networks (RNNs) to automatically estimate Prkachin and Solomon Pain Intensity (PSPI) levels from face images. The estimated scores are then fed into the personalized Hidden Conditional Random Fields (HCRFs), used to estimate the VAS, provided by each person. Personalization of the model is performed using a newly introduced facial expressiveness score, unique for each person. To the best of our knowledge, this is the first approach to automatically estimate VAS from face images. We show the benefits of the proposed personalized over traditional non-personalized approach on a benchmark dataset for pain analysis from face images.
Although there is much research in the automated recognition of affect from facial expression (for surveys see: @cite_2 @cite_17 ), until recently only a handful of works have focused on automated pain estimation. Due to rapid advances in computer vision and also the recent release of the UNBC-McMaster dataset @cite_8 , pain analysis from face images have seen significant advances. Specifically, this dataset provides videos with each frame coded in terms of Prkachin and Solomon Pain Intensity (PSPI) @cite_22 score, defined on an ordinal scale 0-15. This is considered to be an objective pain score, in contrast to subjective pain ratings such as VAS. Despite the fact that VAS is still the most commonly accepted pain score in clinical settings, all existing automatic methods for pain estimation from pain images focused on prediction of PSPI scores. We outline below some of the recently published works.
@cite_8 @cite_22 @cite_17 @cite_2
2101545465 2106043670 2156503193 2117645142
A major factor hindering the deployment of a fully functional automatic facial expression detection system is the lack of representative data. A solution to this is to narrow the context of the target application, so enough data is available to build robust models so high performance can be gained. Automatic pain detection from a patient's face represents one such application. To facilitate this work, researchers at McMaster University and University of Northern British Columbia captured video of participant's faces (who were suffering from shoulder pain) while they were performing a series of active and passive range-of-motion tests to their affected and unaffected limbs on two separate occasions. Each frame of this data was AU coded by certified FACS coders, and self-report and observer measures at the sequence level were taken as well. This database is called the UNBC-McMaster Shoulder Pain Expression Archive Database. To promote and facilitate research into pain and augment current datasets, we have publicly made available a portion of this database which includes: 1) 200 video sequences containing spontaneous facial expressions, 2) 48,398 FACS coded frames, 3) associated pain frame-by-frame scores and sequence-level self-report and observer measures, and 4) 66-point AAM landmarks. This paper documents this data distribution in addition to describing baseline results of our AAM SVM system. This data will be available for distribution in March 2011. The present study examined psychometric properties of facial expressions of pain. A diverse sample of 129 people suffering from shoulder pain underwent a battery of active and passive range-of-motion tests to their affected and unaffected limbs. The same tests were repeated on a second occasion. Participants rated the maximum pain induced by each test on three self-report scales. Facial actions were measured with the Facial Action Coding System. Several facial actions discriminated painful from non-painful movements; however, brow-lowering, orbit tightening, levator contraction and eye closing appeared to constitute a distinct, unitary action. An index of pain expression based on these actions demonstrated test-retest reliability and concurrent validity with self-reports of pain. The findings support the concept of a core pain expression with desirable psychometric properties. They are also consistent with the suggestion of individual differences in pain expressiveness. Reasons for varying reports of relations between pain expression and self-reports in previous studies are discussed. Automated analysis of human affective behavior has attracted increasing attention from researchers in psychology, computer science, linguistics, neuroscience, and related disciplines. However, the existing methods typically handle only deliberately displayed and exaggerated expressions of prototypical emotions despite the fact that deliberate behaviour differs in visual appearance, audio profile, and timing from spontaneously occurring behaviour. To address this problem, efforts to develop algorithms that can process naturally occurring human affective behaviour have recently emerged. Moreover, an increasing number of efforts are reported toward multimodal fusion for human affect analysis including audiovisual fusion, linguistic and paralinguistic fusion, and multi-cue visual fusion based on facial expressions, head movements, and body gestures. This paper introduces and surveys these recent advances. We first discuss human emotion perception from a psychological perspective. Next we examine available approaches to solving the problem of machine understanding of human affective behavior, and discuss important issues like the collection and availability of training and test data. We finally outline some of the scientific and engineering challenges to advancing human affect sensing technology. This survey describes recent progress in the field of Affective Computing (AC), with a focus on affect detection. Although many AC researchers have traditionally attempted to remain agnostic to the different emotion theories proposed by psychologists, the affective technologies being developed are rife with theoretical assumptions that impact their effectiveness. Hence, an informed and integrated examination of emotion theories from multiple areas will need to become part of computing practice if truly effective real-world systems are to be achieved. This survey discusses theoretical perspectives that view emotions as expressions, embodiments, outcomes of cognitive appraisal, social constructs, products of neural circuitry, and psychological interpretations of basic feelings. It provides meta-analyses on existing reviews of affect detection systems that focus on traditional affect detection modalities like physiology, face, and voice, and also reviews emerging research on more novel channels such as text, body language, and complex multimodal systems. This survey explicitly explores the multidisciplinary foundation that underlies all AC applications by describing how AC researchers have incorporated psychological theories of emotion and how these theories affect research questions, methods, results, and their interpretations. In this way, models and methods can be compared, and emerging insights from various disciplines can be more expertly integrated.