ANTICIPATING SAFETY ISSUES IN
E2E C ONVERSATIONAL AI:
FRAMEWORK AND TOOLING
Emily Dinan1, Gavin Abercrombie2, A. Stevie Bergman3
Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling
Over the last several years, end-to-end neural conversational agents have
vastly improved in their ability to carry a chit-chat conversation with humans.
However, these models are often trained on large datasets from the internet,
and as a result, may learn undesirable behaviors from this data, such as toxic
or otherwise harmful language. Researchers must thus wrestle with the issue of
how and when to release these models. In this paper, we survey the problem
landscape for safety for end-to-end conversational AI and discuss recent and
related work. We highlight tensions between values, potential positive impact
and potential harms, and provide a framework for making decisions about whether
and how to release these models, following the tenets of value-sensitive
design. We additionally provide a suite of tools to enable researchers to make
better-informed decisions about training and releasing end-to-end
conversational AI models.
Emily Dinan Gavin Abercrombie A. Stevie Bergman Shannon Spruit Dirk Hovy Y-Lan Boureau Verena Rieser