Pragmatically Learning from Pedagogical Demonstrations in Multi-Goal Environments
Learning from demonstration methods usually leverage close to optimal
demonstrations to accelerate training. By contrast, when demonstrating a task,
human teachers deviate from optimal demonstrations and pedagogically modify
their behavior by giving demonstrations that best disambiguate the goal they
want to demonstrate. Analogously, human learners excel at pragmatically
inferring the intent of the teacher, facilitating communication between the two
agents. These mechanisms are critical in the few demonstrations regime, where
inferring the goal is more difficult. In this paper, we implement pedagogy and
pragmatism mechanisms by leveraging a Bayesian model of goal inference from
demonstrations. We highlight the benefits of this model in multi-goal
teacher-learner setups with two artificial agents that learn with
goal-conditioned Reinforcement Learning. We show that combining a pedagogical
teacher and a pragmatic learner results in faster learning and reduced goal
ambiguity over standard learning from demonstrations, especially in the few
demonstrations regime.