Statistical language models such as neural networks
learn to generate fluent responses via training from DA-labeled corpora.
One canonical model, known as Semantically Conditioned LSTM (SC-LSTM) [1]}, [2]}, encodes dialog acts with one-hot representations and uses it as an extra feature to inform the sentence generation process. Despite its good performance on simple domains, it suffers from the scalability issue. SC-LSTM requires large amounts of task-specific annotated data which is not available for many tasks in real-world applications, especially for more complex tasks where the number of DAs grows exponentially with the number of slots.
To address the issue, there is a growing interest in leveraging large-scale pre-trained models, such as GPT-2 [3]}, for controllable text generation [4]}.
