Following the PyDial benchmarking process, we leave all hyperparameters constant across all environments and dialog domains<cite> (Casanueva et al., 2017)</cite> , thus also evaluating the generalization capabilities of the agents.