The delivery of mental health interventions via ubiquitous devices has shown much promise. A conversational chatbot is a promising oracle for delivering appropriate just-in-time interventions. However, designing emotionally-aware agents, specially in this context, is under-explored. Furthermore, the feasibility of automating the delivery of just-in-time mHealth interventions via such an agent has not been fully studied. In this paper, we present the design and evaluation of EMMA (EMotion-Aware mHealth Agent) through a two-week long human-subject experiment with N=39 participants. EMMA provides emotionally appropriate micro-activities in an empathetic manner. We show that the system can be extended to detect a user's mood purely from smartphone sensor data. Our results show that our personalized machine learning model was perceived as likable via self-reports of emotion from users. Finally, we provide a set of guidelines for the design of emotion-aware bots for mHealth.
Conversational agents have shown promise in automating the detection of psychological symptoms for both assessment and the evaluation of treatment impact @cite_13 . There is evidence suggesting that the general population can also benefit from such eHealth interventions. Anxiety and depression prevention EMIs are associated with small but positive effects on symptom reduction. The medium to long-term effects of such interventions need further exploration @cite_1 .
@cite_1 @cite_13
2751552421 2531271152 2964262738 1995641269
Abstract Background Anxiety and depression are associated with a range of adverse outcomes and represent a large global burden to individuals and health care systems. Prevention programs are an important way to avert a proportion of the burden associated with such conditions both at a clinical and subclinical level. eHealth interventions provide an opportunity to offer accessible, acceptable, easily disseminated globally low-cost interventions on a wide scale. However, the efficacy of these programs remains unclear. The aim of this study is to review and evaluate the effects of eHealth prevention interventions for anxiety and depression. Method A systematic search was conducted on four relevant databases to identify randomized controlled trials of eHealth interventions aimed at the prevention of anxiety and depression in the general population published between 2000 and January 2016. The quality of studies was assessed and a meta-analysis was performed using pooled effect size estimates obtained from a random effects model. Results Ten trials were included in the systematic review and meta-analysis. All studies were of sufficient quality and utilized cognitive behavioural techniques. At post-treatment, the overall mean difference between the intervention and control groups was 0.25 (95 confidence internal: 0.09, 0.41; p = 0.003) for depression outcome studies and 0.31 (95 CI: 0.10, 0.52; p = 0.004) for anxiety outcome studies, indicating a small but positive effect of the eHealth interventions. The effect sizes for universal and indicated selective interventions were similar (0.29 and 0.25 respectively). However, there was inadequate evidence to suggest that such interventions have an effect on long-term disorder incidence rates. Conclusions Evidence suggests that eHealth prevention interventions for anxiety and depression are associated with small but positive effects on symptom reduction. However, there is inadequate evidence on the medium to long-term effect of such interventions, and importantly, on the reduction of incidence of disorders. Further work to explore the impact of eHealth psychological interventions on long-term incidence rates. Major depressive disorder (MDD) is known to result in neurophysiological and neurocognitive changes that affect control of motor, linguistic, and cognitive functions. MDD's impact on these processes is reflected in an individual's communication via coupled mechanisms: vocal articulation, facial gesturing and choice of content to convey in a dialogue. In particular, MDD-induced neurophysiological changes are associated with a decline in dynamics and coordination of speech and facial motor control, while neurocognitive changes influence dialogue semantics. In this paper, biomarkers are derived from all of these modalities, drawing first from previously developed neurophysiologically-motivated speech and facial coordination and timing features. In addition, a novel indicator of lower vocal tract constriction in articulation is incorporated that relates to vocal projection. Semantic features are analyzed for subject avatar dialogue content using a sparse coded lexical embedding space, and for contextual clues related to the subject's present or past depression status. The features and depression classification system were developed for the 6th International Audio Video Emotion Challenge (AVEC), which provides data consisting of audio, video-based facial action units, and transcribed text of individuals communicating with the human-controlled avatar. A clinical Patient Health Questionnaire (PHQ) score and binary depression decision are provided for each participant. PHQ predictions were obtained by fusing outputs from a Gaussian staircase regressor for each feature set, with results on the development set of mean F1=0.81, RMSE=5.31, and MAE=3.34. These compare favorably to the challenge baseline development results of mean F1=0.73, RMSE=6.62, and MAE=5.52. On test set evaluation, our system obtained a mean F1=0.70, which is similar to the challenge baseline test result. Future work calls for consideration of joint feature analyses across modalities in an effort to detect neurological disorders based on the interplay of motor, linguistic, affective, and cognitive components of communication. Mental illness is one of the most pressing public health issues of our time. While counseling and psychotherapy can be effective treatments, our knowledge about how to conduct successful counseling conversations has been limited due to lack of large-scale data with labeled outcomes of the conversations. In this paper, we present a large-scale, quantitative study on the discourse of text-message-based counseling conversations. We develop a set of novel computational discourse analysis methods to measure how various linguistic aspects of conversations are correlated with conversation outcomes. Applying techniques such as sequence-based conversation models, language model comparisons, message clustering, and psycholinguistics-inspired word frequency analyses, we discover actionable conversation strategies that are associated with better conversation outcomes. We developed and evaluated a multimodal affect detector that combines conversational cues, gross body language, and facial features. The multimodal affect detector uses feature-level fusion to combine the sensory channels and linear discriminant analyses to discriminate between naturally occurring experiences of boredom, engagement flow, confusion, frustration, delight, and neutral. Training and validation data for the affect detector were collected in a study where 28 learners completed a 32- min. tutorial session with AutoTutor, an intelligent tutoring system with conversational dialogue. Classification results supported a channel × judgment type interaction, where the face was the most diagnostic channel for spontaneous affect judgments (i.e., at any time in the tutorial session), while conversational cues were superior for fixed judgments (i.e., every 20 s in the session). The analyses also indicated that the accuracy of the multichannel model (face, dialogue, and posture) was statistically higher than the best single-channel model for the fixed but not spontaneous affect expressions. However, multichannel models reduced the discrepancy (i.e., variance in the precision of the different emotions) of the discriminant models for both judgment types. The results also indicated that the combination of channels yielded superadditive effects for some affective states, but additive, redundant, and inhibitory effects for others. We explore the structure of the multimodal linear discriminant models and discuss the implications of some of our major findings.