There has been work in linguistic grounding that allows for temporal aspects, for instance in videos [1]}, [2]}, [3]}, and both time and interaction, notably in the field of robotics [4]}, [5]}, [6]}. However, robotics does not currently constitute a suitable platform for language learning, since physical embodiment is not yet feasible. Virtual embodiment does not suffer from the same limitations. There has been recent work on grounding in virtual worlds, notably in video games [7]}. Work applying deep reinforcement learning to video games points the way towards agents learning from each other [8]}, [9]}, [10]}. An alternative would be virtual or augmented reality, which offers the benefit of joint multi-modal data over time, but this crucially lacks the element of interaction.
