Robotic manipulation in simulation. In this section, we answer the core question of our work: can we learn to imitate others from diverse third-person videos? In particular, we collect human demonstrations for manipulation tasks as explained in Section REF  and learn a reward function as explained in Section . This is a challenging setting because as shown in Figure REF , the collected data and robotic environments belong to different domains and do not share any appearance characteristics. Further, unlike previous works [1]}, [2]}, we do not use any environment reward as an additional supervision to the reinforcement learning agent. Figure REF  presents our results. For the Reach task, GraphIRL and environment reward are able to achieve a success rate of \(1.0\) , while other baseline methods are substantially behind GraphIRL (e.g. \(0.477\)  XIRL and \(0.155\)  TCN). The poor performance of vision-based baselines could be attributed to substantial visual domain shift. Due to domain shift, the learned rewards for these baselines produce low rewards for successful episodes, please refer to Appendix  for a more detailed qualitative analysis. In the Push setting, we find that vision-based baseline methods still perform poorly. Similar to Reach, XIRL performs the best out of the vision-based baselines with a success rate of \(0.187\) , and GraphIRL performs better than environment reward (e.g. \(0.832\)  GraphIRL vs \(0.512\)  Environment Reward). This result shows clear advantage of our method as we are able to outperform a hand-designed reward function without using any task specific information. The Peg in Box task is rigorous to solve since it requires 3-d reasoning and a precise reward function. Here, while all vision-based methods fail, our GraphIRL method is able to solve the task with a success rate comparable to that achieved with the hand-designed environment reward. Overall, our GraphIRL method is able to solve 2D and 3D reasoning tasks with a real-robot without a hand-designed reward function or access to 3D scene information.
