Equipping social and service robots with the ability to perceive human emotional intensities during an interaction is in increasing demand. Most of existing work focuses on determining which emotion(s) participants are expressing from facial expressions but largely overlooks the emotional intensities spontaneously revealed by other social cues, especially body languages. In this paper, we present a real-time method for robots to capture fluctuations of participants' emotional intensities from their body poses. Unlike conventional joint-position-based approaches, our method adopts local joint transformations as pose descriptors which are invariant to subject body differences as well as the pose sensor positions. In addition, we use a Long Short-Term Memory Recurrent Neural Network (LSTM-RNN) architecture to take the specific emotion context into account when estimating emotional intensities from body poses. The dataset evaluation suggests that the proposed method is effective and performs better than baseline method on the test dataset. Also, a series of succeeding field tests on a physical robot demonstrates that the proposed method effectively estimates subjects emotional intensities in real-time. Furthermore, the robot equipped with our method is perceived to be more emotion-sensitive and more emotionally intelligent.
Emotional intensity is defined as the psychological states of being activated by some evoking stimulus @cite_16 , which can be measured by bio-electric signals, such as heart rate, blood pressure, skin conductance etc @cite_33 @cite_10 @cite_31 . For example, Kulic and Croft leveraged heart rate, perspiration rate, and facial muscle contraction to estimate participants' emotional intensities in terms of anxiety, calm and surprise towards robot arm motions @cite_20 ; Swangnetr and Kaber used heart rate and galvanic skin response signals to estimate the intensity of joy and excitement for the elderly patients during medical Patient-Robot Interactions @cite_0 . Saulnier exploited the natural muscle tensions to estimate participants' stress levels for domestic robots @cite_25 . However, all these studies require putting sensor on participants, e.g., via special wearable devices @cite_19 or electro-physiological monitoring devices @cite_20 @cite_6 , it may be difficult and sometimes impossible to deploy such biological methods for social or service robots in real-time HRI.
@cite_33 @cite_6 @cite_0 @cite_19 @cite_31 @cite_16 @cite_10 @cite_25 @cite_20
1964280755 2111438996 2019771222 2036081755 1977308106 2092804630 2011606949 2000982104
While the reasons underlying musical emotions are unclear, music is nevertheless a powerful elicitor of emotion, and as such, may induce autonomic nervous system responses. One typical measure of this neural pathway is the skin conductance response (SCR). This response generally depends upon stimulus arousal, one of the two motivational determinants of emotion. The objective of the present study was to verify whether emotional reactions to music elicit such event-related autonomic responses. To this aim, four musical emotions varying in arousal were employed: fear, happiness, sadness and peacefulness. SCRs were found to be greater with the two more stimulating emotions, fear and happiness, as compared to the two more relaxing emotions, sadness and peacefulness (P , 0:05). In addition, subjects’ ratings of the emotional clarity for each excerpt did not parallel the corresponding SCRs magnitudes. The results show that SCRs can be evoked and modulated by musical emotional arousal, but are not sensitive to emotional clarity. While several studies have been performed with visual scenes and environmental sounds, the present study brings similar evidence from the musical domain. q 2002 Elsevier Science Ireland Ltd. All rights reserved. Investigation into robot-assisted intervention for children with autism spectrum disorder (ASD) has gained momentum in recent years. Therapists involved in interventions must overcome the communication impairments generally exhibited by children with ASD by adeptly inferring the affective cues of the children to adjust the intervention accordingly. Similarly, a robot must also be able to understand the affective needs of these children-an ability that the current robot-assisted ASD intervention systems lack-to achieve effective interaction that addresses the role of affective states in human-robot interaction and intervention practice. In this paper, we present a physiology-based affect-inference mechanism for robot-assisted intervention where the robot can detect the affective states of a child with ASD as discerned by a therapist and adapt its behaviors accordingly. This paper is the first step toward developing ldquounderstandingrdquo robots for use in future ASD intervention. Experimental results with six children with ASD from a proof-of-concept experiment (i.e., a robot-based basketball game) are presented. The robot learned the individual liking level of each child with regard to the game configuration and selected appropriate behaviors to present the task at his her preferred liking level. Results show that the robot automatically predicted individual liking level in real time with 81.1 accuracy. This is the first time, to our knowledge, that the affective states of children with ASD have been detected via a physiology-based affect recognition technique in real time. This is also the first time that the impact of affect-sensitive closed-loop interaction between a robot and a child with ASD has been demonstrated experimentally. Due to a major shortage of nurses in the U.S., future healthcare service robots are expected to be used in tasks involving direct interaction with patients. Consequently, there is a need to design nursing robots with the capability to detect and respond to patient emotional states and to facilitate positive experiences in healthcare. The objective of this study was to develop a new computational algorithm for accurate patient emotional state classification in interaction with nursing robots during medical service. A simulated medicine delivery experiment was conducted at two nursing homes using a robot with different human-like features. Physiological signals, including heart rate (HR) and galvanic skin response (GSR), as well as subjective ratings of valence (happy-unhappy) and arousal (excited-bored) were collected on elderly residents. A three-stage emotional state classification algorithm was applied to these data, including: (1) physiological feature extraction; (2) statistical-based feature selection; and (3) a machine-learning model of emotional states. A pre-processed HR signal was used. GSR signals were nonstationary and noisy and were further processed using wavelet analysis. A set of wavelet coefficients, representing GSR features, was used as a basis for current emotional state classification. Arousal and valence were significantly explained by statistical features of the HR signal and GSR wavelet features. Wavelet-based de-noising of GSR signals led to an increase in the percentage of correct classifications of emotional states and clearer relationships among the physiological response and arousal and valence. The new algorithm may serve as an effective method for future service robot real-time detection of patient emotional states and behavior adaptation to promote positive healthcare experiences. Differences in blood pressure associated with reported happiness, anger, and anxiety are examined among 90 borderline hypertensives during 24-hr blood pressure monitoring. There were 1152 individual ambulatory blood pressure readings for which subjects classified their emotional state as happy (n = What is the structure of emotion? Emotion is too broad a class of events to be a single scientific category, and no one structure suffices. As an illustration, core affect is distinguished from prototypical emotional episode. Core affect refers to consciously accessible elemental processes of pleasure and activation, has many causes, and is always present. Its structure involves two bipolar dimensions. Prototypical emotional episode refers to a complex process that unfolds over time, involves causally connected subevents (antecedent; appraisal; physiological, affective, and cognitive changes; behavioral response; self-categorization), has one perceived cause, and is rare. Its structure involves categories (anger, fear, shame, jealousy, etc.) vertically organized as a fuzzy hierarchy and horizontally organized as part of a circumplex. Several emerging computer devices read bio-electrical signals (e.g., electro-corticographic signals, skin biopotential or facial muscle tension) and translate them into computer- understandable input. We investigated how one low-cost commercially-available device could be used to control a domestic robot. First, we used the device to issue direct motion commands; while we could control the device somewhat, it proved difficult to do reliably. Second, we interpreted one class of signals as suggestive of emotional stress, and used that as an emotional parameter to influence (but not directly control) robot behaviour. In this case, the robot would react to human stress by staying out of the person's way. Our work suggests that affecting behaviour may be a reasonable way to leverage such devices. In order for humans and robots to interact in an effective and intuitive manner, robots must obtain information about the human affective state in response to the robot's actions. This secondary mode of interactive communication is hypothesized to permit a more natural collaboration, similar to the "body language" interaction between two cooperating humans. This paper describes the implementation and validation of a hidden Markov model (HMM) for estimating human affective state in real time, using robot motions as the stimulus. Inputs to the system are physiological signals such as heart rate, perspiration rate, and facial muscle contraction. Affective state was estimated using a two- dimensional valence-arousal representation. A robot manipulator was used to generate motions expected during human-robot interaction, and human subjects were asked to report their response to these motions. The human physiological response was also measured. Robot motions were generated using both a nominal potential field planner and a recently reported safe motion planner that minimizes the potential collision forces along the path. The robot motions were tested with 36 subjects. This data was used to train and validate the HMM model. The results of the HMM affective estimation are also compared to a previously implemented fuzzy inference engine.