In what follows, we describe in detail task-grounded pre-training.
The model parameters \(\theta \)  are optimized on training data \(\mathcal {D}=\lbrace  x_n\rbrace _{n=1}^{N}\)  using multi-task learning [1]}, as illustrated in Figure REF , where a dialog turn is presented using a simple text format, with special delimiter tokens to indicate the types of different segments, (e.g., [EOB] indicates the end of the belief state segment).
In addition to belief prediction and grounded response generation, whose losses are defined as
\(\mathcal {L}_{\text{dst}} = -\log (P_r(b|c ; \theta ))\) 
