Detecting Interlocutor Confusion in Situated Human-Avatar Dialogue: A Pilot Study
In order to enhance levels of engagement with conversational systems, our
long term research goal seeks to monitor the confusion state of a user and
adapt dialogue policies in response to such user confusion states. To this end,
in this paper, we present our initial research centred on a user-avatar
dialogue scenario that we have developed to study the manifestation of
confusion and in the long term its mitigation. We present a new definition of
confusion that is particularly tailored to the requirements of intelligent
conversational system development for task-oriented dialogue. We also present
the details of our Wizard-of-Oz based data collection scenario wherein users
interacted with a conversational avatar and were presented with stimuli that
were in some cases designed to invoke a confused state in the user. Post study
analysis of this data is also presented. Here, three pre-trained deep learning
models were deployed to estimate base emotion, head pose and eye gaze. Despite
a small pilot study group, our analysis demonstrates a significant relationship
between these indicators and confusion states. We understand this as a useful
step forward in the automated analysis of the pragmatics of dialogue.