We introduce Task-driven Embodied Agents that Chat (TEACh), a dataset of over 3000 situated dialogues in which a human Commander and human Follower collaborate in natural language to complete household tasks in the AI2-THOR simulation environment.
TEACh contains dialogue phenomena related to grounding dialogue in objects and actions in the environment, varying levels of instruction granularity, and interleaving of utterances between speakers in the absence of enforced turn taking.
We also introduce a task definition language that is extensible to new tasks and even other simulators.
We propose three benchmarks based on TEACh.
To study Follower models, we define the Execution from Dialogue History (EDH) and Trajectory from Dialogue (TfD) benchmarks, and evaluate an adapted Episodic Transformer [1]} as an initial baseline.
To study the potential of Commander and Follower models, we define the Two-Agent Task Completion benchmark, and explore the difficulty of defining rule-based agents from TEACh data.
