Existing neural conversational models process natural language primarily on a lexico-syntactic level, thereby ignoring one of the most crucial components of human-to-human dialogue: its affective content. We take a step in this direction by proposing three novel ways to incorporate affective emotional aspects into long short term memory (LSTM) encoder-decoder neural conversation models: (1) affective word embeddings, which are cognitively engineered, (2) affect-based objective functions that augment the standard cross-entropy loss, and (3) affectively diverse beam search for decoding. Experiments show that these techniques improve the open-domain conversational prowess of encoder-decoder networks by enabling them to produce emotionally rich responses that are more interesting and natural.
Affectively cognizant virtual agents are generating interest both in the academia @cite_27 and the industry, https: www.ald.softbankrobotics.com en robots pepper due to their ability to provide emotional companionship to humans. Endowing text-based dialogue generation systems with emotions is also an active area of research. Past research has mostly focused on developing hand-crafted speech and text-based features to incorporate emotions in retrieval-based or slot-based spoken dialogue systems @cite_11 @cite_26 .
@cite_27 @cite_26 @cite_11
2795571593 2905807898 219197677 2731113721
Computer-based conversational agents are becoming ubiquitous. However, for these systems to be engaging and valuable to the user, they must be able to express emotion, in addition to providing informative responses. Humans rely on much more than language during conversations; visual information is key to providing context. We present the first example of an image-grounded conversational agent using visual sentiment, facial expression and scene features. We show that key qualities of the generated dialogue can be manipulated by the features used for training the agent. We evaluate our model on a large and very challenging real-world dataset of conversations from social media (Twitter). The image-grounding leads to significantly more informative, emotional and specific responses, and the exact qualities can be tuned depending on the image features used. Furthermore, our model improves the objective quality of dialogue responses when evaluated on standard natural language metrics. Abstract Big Data and Deep Learning algorithms combined with enormous computing power have paved ways for significant technological advancements. Technology is evolving to anticipate, understand and address our unmet needs. However, to fully meet human needs, machines or computers must deeply understand human behavior including emotions. Emotions are physiological states generated in humans as a reaction to internal or external events. They are complex and studied across numerous fields including computer science. As humans, on reading “Why don't you ever text me!”, we can either interpret it as a sad or an angry emotion and the same ambiguity exists for machines as well. Lack of facial expressions and voice modulations make detecting emotions in text a challenging problem. However, in today's online world, humans are increasingly communicating using text messaging applications and digital agents. Hence, it is imperative for machines to understand emotions in textual dialogue to provide emotionally aware responses to users. In this paper, we propose a novel Deep Learning based approach to detect emotions - Happy, Sad and Angry in textual dialogues. The essence of our approach lies in combining both semantic and sentiment based representations for more accurate emotion detection. We use semi-automated techniques to gather large scale training data with diverse ways of expressing emotions to train our model. Evaluation of our approach on real world dialogue datasets reveals that it significantly outperforms traditional Machine Learning baselines as well as other off-the-shelf Deep Learning models. This paper describes our initial work in developing a real-time audio-visual Chinese speech synthesizer with a 3D expressive avatar. The avatar model is parameterized according to the MPEG-4 facial animation standard [1]. This standard offers a compact set of facial animation parameters (FAPs) and feature points (FPs) to enable realization of 20 Chinese visemes and 7 facial expressions (i.e. 27 target facial configurations). The Xface [2] open source toolkit enables us to define the influence zone for each FP and the deformation function that relates them. Hence we can easily animate a large number of coordinates in the 3D model by specifying values for a small set of FAPs and their FPs. FAP values for 27 target facial configurations were estimated from available corpora. We extended the dominance blending approach to effect animations for coarticulated visemes superposed with expression changes. We selected six sentiment-carrying text messages and synthesized expressive visual speech (for all expressions, in randomized order) with neutral audio speech. A perceptual experiment involving 11 subjects shows that they can identify the facial expression that matches the text message’s sentiment 85 of the time. The impact of the world-wide ageing population has commenced with respect to society in developed countries. Several researchers focused on exploring new methods to improve the quality of life of elderly individuals by allowing them to remain independent and healthy to the maximum possible extent. For example, new walking aids are designed to allow elderly individuals to remain mobile in a safe manner because the importance of walking is well-known. The aim of the present study involves designing a humanoid robot guide as a walking trainer for elderly individuals. It is hypothesized that the same service robot provides an assistive and social contribution with respect to interaction between elderly users by motivating them to walk more and simultaneously provides assistance, such as physical assistance and gait monitoring, while walking. This study includes a detailed statement of the research problem as well as a literature review of existing studies related to walking companion robots. A user-centred design approach is adopted to report the results of the current first feasibility study by using a commercially available humanoid robot known as Pepper developed by Softbank-Aldebaran. A quantitative questionnaire was used to investigate all elements that assess intrinsic motivation in users while performing a given activity. Conversely, basic gait data were acquired through a video analysis to test the capability of the robot to modify the gait of human users. The results in terms of the feedback received from elderly subjects and the literature review improve the design of the walking trainer for elderly individuals.