This paper presents the EACare project, an ambitious multi-disciplinary collaboration with the aim to develop an embodied system, capable of carrying out neuropsychological tests to detect early signs of dementia, e.g., due to Alzheimer's disease. The system will use methods from Machine Learning and Social Robotics, and be trained with examples of recorded clinician-patient interactions. The interaction will be developed using a participatory design approach. We describe the scope and method of the project, and report on a first Wizard of Oz prototype.
Perhaps most similar to our work both in terms of the interaction modality (language-based) and condition of the user group (older adults diagnosed with dementia), @cite_25 conducted a laboratory Wizard of Oz experiment to evaluate the challenges in speech recognition and dialogue between participants and a personal assistant robot while performing daily household tasks such as making tea. Their results suggest that autonomous language-based interactions in this setting can be challenging not only because of speech recognition errors but also because robots will often need to proactively employ conversational repair strategies during moments of confusion.
@cite_25
2294978157 2139331960 2773498419 2109636054
Increases in the prevalence of dementia and Alzheimer’s disease (AD) are a growing challenge in many nations where healthcare infrastructures are ill-prepared for the upcoming demand for personal caregiving. To help individuals with AD live at home for longer, we are developing a mobile robot, called ED, intended to assist with activities of daily living through visual monitoring and verbal prompts in cases of difficulty. In a series of experiments, we study speech-based interactions between ED and each of 10 older adults with AD as the latter complete daily tasks in a simulated home environment. Traditional automatic speech recognition is evaluated in this environment, along with rates of verbal behaviors that indicate confusion or trouble with the conversation. Analysis reveals that speech recognition remains a challenge in this setup, especially during household tasks with individuals with AD. Across the verbal behaviors that indicate confusion, older adults with AD are very likely to simply ignore the robot, which accounts for over 40p of all such behaviors when interacting with the robot. This work provides a baseline assessment of the types of technical and communicative challenges that will need to be overcome for robots to be used effectively in the home for speech-based assistance with daily living. To help individuals with Alzheimer’s disease live at home for longer, we are developing a mobile robotic platform, called ED, intended to be used as a personal caregiver to help with the performance of activities of daily living. In a series of experiments, we study speech-based interactions between each of 10 older adults with Alzheimers disease and ED as the former makes tea in a simulated home environment. Analysis reveals that speech recognition remains a challenge for this recording environment, with word-level accuracies between 5.8 and 19.2 during household tasks with individuals with Alzheimer’s disease. This work provides a baseline assessment for the types of technical and communicative challenges that will need to be overcome in human-robot interaction for this population. Probabilistic graphical models, such as partially observable Markov decision processes (POMDPs), have been used in stochastic spoken dialog systems to handle the inherent uncertainty in speech recognition and language understanding. Such dialog systems suffer from the fact that only a relatively small number of domain variables are allowed in the model, so as to ensure the generation of good-quality dialog policies. At the same time, the non-language perception modalities on robots, such as vision-based facial expression recognition and Lidar-based distance detection, can hardly be integrated into this process. In this paper, we use a probabilistic commonsense reasoner to “guide” our POMDP-based dialog manager, and present a principled, multimodal dialog management (MDM) framework that allows the robot's dialog belief state to be seamlessly updated by both observations of human spoken language, and exogenous events such as the change of human facial expressions. The MDM approach has been implemented and evaluated both in simulation and on a real mobile robot using guidance tasks. We present SimSensei Kiosk, an implemented virtual human interviewer designed to create an engaging face-to-face interaction where the user feels comfortable talking and sharing information. SimSensei Kiosk is also designed to create interactional situations favorable to the automatic assessment of distress indicators, defined as verbal and nonverbal behaviors correlated with depression, anxiety or post-traumatic stress disorder (PTSD). In this paper, we summarize the design methodology, performed over the past two years, which is based on three main development cycles: (1) analysis of face-to-face human interactions to identify potential distress indicators, dialogue policies and virtual human gestures, (2) development and analysis of a Wizard-of-Oz prototype system where two human operators were deciding the spoken and gestural responses, and (3) development of a fully automatic virtual interviewer able to engage users in 15-25 minute interactions. We show the potential of our fully automatic virtual human interviewer in a user study, and situate its performance in relation to the Wizard-of-Oz prototype.