Imitation learning (IL) is an effective class of algorithms for designing and optimizing controllers for robot systems. While recent advances in Reinforcement Learning have shown it is capable of producing agents that learn robot controllers from scratch, IL remains a more practical alternative for cases where it is easier to specify robot behaviours through examples than through rewards. We take an approach similar to existing work [1]}, [2]} on learning from demonstrations (LfD). We use expert data to build a reward model to be maximized with existing RL algorithms. Unlike LfD, where an expert demonstrates which actions to perform at some robot states, we focus on the case where action supervision is not available: the agent only gets access to a dataset of state/observations sequences – a setting known as Imitation Learning from Observations (ILfO). While LfD usually requires providing demonstrations by teleoperation of the robot, ILfO aims to utilize streams of state and observational data, much like a human can learn to do a task by watching other people.
