
For ecological validity [1]}), we give minimal instructions. The human user should tell the human agent where to go, and the agent should determine the place, confirm, and drive the user there. This naturalistic approach contrasts with two-step paradigms, which first generate a formal dialog outline, then paraphrase that outline into English [2]}.

Our WOZ interface collects on-line conversations between two humans, versus single-turn extensions [3]}, [4]} or machine-partnered dialogs [5]}, [6]}, [7]}, [8]}.

As we aim for end-to-end training [9]}, [10]}, [4]}, [12]}, [13]}, we do not collect intermediate annotations [7]}, [8]}, [16]}, [17]}. For evaluation purposes, we do collect a success/fail confirmation from the user at the end of each dialog.

As real users may want to go anywhere in the world, we employ a publicly-accessible backend that supports tens of millions of places, in contrast with more narrowly-bounded backend databases [8]}.

In most WOZ systems, human agents are allowed to type free text, to increase expressiveness or to support studies of information presentation [17]}. However, machine learning of human-agent language is a burden. As in [20]}, we only allow free text on the user side. Human agents select utterances from an extensible list of templates, and they select template-filler words by clicking boxes. Agents similarly control API calls and parmeters with box clicks. We find that human agents can easily satisfy users' goals inside the WOZ, even though the domain is complex.

