Recent advancements in language models, such as recurrent neural networks [1]}, [2]}, [3]}, [4]}, [5]}, [6]} and attention-based architectures [7]}, [8]}, [9]}, [10]}, have spurred increasing interest in such end-to-end dialogue systems [11]}, [12]}, [13]}. Model-based approaches, in which a learned agent is substituted for a human, allow learning to be done entirely within simulation without human intervention [14]}, [15]}, [16]}, [17]}, [18]}. In contrast to these approaches, CALM augments the traditional language modeling objective with task-specific rewards in order to finetune a model that is more aware of task goals, which significantly improves performance over a naïve language model without the need for simulating human responses in an interactive training loop. [19]} recently proposed a model-free, offline approach to undirected dialogue, or dialogue without a specific task goal. Our method differs in that we aim to solve goal-oriented dialogue which allows us to optimize task-specific objectives, and that we take a model-based RL approach which enables us to leverage fine-tuned language models.
