As observed by [1]}, for some structured tasks, such as table question answering, pre-training on a simplified version of the given task with a synthetic context can help the model to focus learning on the “skills” that are most relevant to utilize the task context, which leads to improved downstream task performance. We instantiate this idea in our method by pre-training our model on a simplified (dialogue-free) version of the task.
Instead of simultaneously modeling all the details of the raw dialogue, as is required to learn \(P(\tau |c_o)\) , the key observation here is that in our case the task reward only depends on the tuple \(\lbrace c_h, c_o, a_T\rbrace \) . This enables us to effectively learn to execute the task by only modeling \(P(c_h, a_T|c_o)\) , without any dialogue at all.
By pre-training our model to first learn this simplified distribution, we effectively focus on learning the necessary skills for completing the task. It is expected that the skills learned during this pre-training phase should also generalize and transfer when we later perform training on the real dialogue. The particular instantiation of this principle in the case of AirDialogue is described in Section .
