Training and evaluation with the PyDial user simulator follows the PyDial benchmarking tasks<cite> (Casanueva et al., 2017)</cite> , where each task (see Table 1) is trained on 10000 dialogs split into ten training iterations of 1000 dialogs each.