2nd Proceedings of Alexa Prize (Alexa Prize 2018).  Advancing the State of the Art in Open Domain Dialog Systems through the Ale
Advancing the State of the Art in Open Domain Dialog Systems through the Alexa Prize
Building open domain conversational systems that allow users to have engaging
conversations on topics of their choice is a challenging task. Alexa Prize was
launched in 2016 to tackle the problem of achieving natural, sustained,
coherent and engaging open-domain dialogs. In the second iteration of the
competition in 2018, university teams advanced the state of the art by using
context in dialog models, leveraging knowledge graphs for language
understanding, handling complex utterances, building statistical and
hierarchical dialog managers, and leveraging model-driven signals from user
responses. The 2018 competition also included the provision of a suite of tools
and models to the competitors including the CoBot (conversational bot) toolkit,
topic and dialog act detection models, conversation evaluators, and a sensitive
content detection model so that the competing teams could focus on building
knowledge-rich, coherent and engaging multi-turn dialog systems. This paper
outlines the advances developed by the university teams as well as the Alexa
Prize team to achieve the common goal of advancing the science of
Conversational AI. We address several key open-ended problems such as
conversational speech recognition, open domain natural language understanding,
commonsense reasoning, statistical dialog management, and dialog evaluation.
These collaborative efforts have driven improved experiences by Alexa users to
an average rating of 3.61, the median duration of 2 mins 18 seconds, and
average turns to 14.6, increases of 14%, 92%, 54% respectively since the launch
of the 2018 competition. For conversational speech recognition, we have
improved our relative Word Error Rate by 55% and our relative Entity Error Rate
by 34% since the launch of the Alexa Prize. Socialbots improved in quality
significantly more rapidly in 2018, in part due to the release of the CoBot
toolkit.
Chandra Khatri Behnam Hedayatnia Anu Venkatesh Jeff Nunn Yi Pan Qing Liu Han Song Anna Gottardi Sanjeev Kwatra Sanju Pancholi Ming Cheng Qinglang Chen Lauren Stubel Karthik Gopalakrishnan Kate Bland Raefer Gabriel Arindam Mandal Dilek Hakkani-Tur Gene Hwang Nate Michel Eric King Rohit Prasad