Dynamic Planning in Open-Ended Dialogue using
Reinforcement Learning
Deborah Cohen, Moonkyung Ryu, Yinlam Chow, Orgad Keller, Id
Dynamic Planning in Open-Ended Dialogue using Reinforcement Learning
Despite recent advances in natural language understanding and generation, and
decades of research on the development of conversational bots, building
automated agents that can carry on rich open-ended conversations with humans
"in the wild" remains a formidable challenge. In this work we develop a
real-time, open-ended dialogue system that uses reinforcement learning (RL) to
power a bot's conversational skill at scale. Our work pairs the succinct
embedding of the conversation state generated using SOTA (supervised) language
models with RL techniques that are particularly suited to a dynamic action
space that changes as the conversation progresses. Trained using crowd-sourced
data, our novel system is able to substantially exceeds the (strong) baseline
supervised model with respect to several metrics of interest in a live
experiment with real users of the Google Assistant.
Deborah Cohen Moonkyung Ryu Yinlam Chow Orgad Keller Ido Greenberg Avinatan Hassidim Michael Fink Yossi Matias Idan Szpektor Craig Boutilier Gal Elidan