Therefore, while the models of [1]}, [2]} need to be trained on labeled, task-specific dialog data, one for each task, GTG can be pre-trained on open-domain datasets and then fine-tuned for specific tasks using much less task labels. For example, in [3]} the parameters of GTG model are initialized using GPT-2, and then pre-trained on large heterogeneous dialog corpora using multi-task learning according to the loss of Equation REF .
