Robotic Manipulation. Figure REF  shows initial and success configurations for each of the three task that we consider: (1) Reach in which the agent needs to reach a goal (red disc) with its end-effector, (2) Push in which the goal is to push a cube to a goal position, and (3) Peg in Box where the goal is to put a peg tied to the robot's end-effector inside a box. The last task is particularly difficult because it requires geometric 3D understanding of the objects. Further, a very specific trajectory is required to avoid collision with the box and complete the task. We collect a total of 256 and 162 video demonstrations for Reach and Peg in Box, respectively, and use 198 videos provided from [1]} for Push. The videos consist of human actors performing the same tasks but with a number of diverse objects and goal markers, as well as varied positions of objects. Unlike the data collected by [1]}, we do not fix the goal position in our demonstrations. In order to detect objects in our training demonstrations, we use a trained model from [3]}. The model is trained on a large-scale dataset collected from YouTube and can detect hands and objects in an image.; refer to Appendix REF  for more details on data collection. Additionally, we do not require the demonstrations to resemble the robotic environment in terms of appearance or distribution of goal location. We use an xArm robot as our robot platform and capture image observations using a static third-person RGB camera in our real setup; details in Appendix .
