Dialogue systems have typically approached the problem of generating realistic dialogue from the perspective of supervised learning [1]}, [2]}, [3]}, [4]}, [5]}, [6]}, [7]}, [8]}. However, dialogue can also be viewed as a sequential decision making process, which is well-suited to planning and reinforcement learning (RL) algorithms. A challenge with the classical RL approach to dialogue is the requirement for active interaction with humans [9]}. Training such a system with active human-in-the-loop interaction quickly becomes expensive and cumbersome, making it desirable to develop techniques for goal-directed training of dialogue systems that can effectively leverage offline data.
<FIGURE>