Inverse RL. To address the aforementioned limitations, inverse RL has been proposed [1]}, [2]}, [3]}, [4]}, [5]}, [6]} and it has recently emerged as a promising paradigm for cross-embodiment imitation in particular [7]}, [8]}, [9]}, [10]}, [11]}, [12]}, [13]}, [14]}, [15]}. For example, [7]} proposes a method for integrating video demonstrations without corresponding actions into off-policy RL algorithms via a latent inverse dynamics model and heuristic reward assignment, and [14]} (XIRL) learns a reward function from video demonstrations using temporal cycle-consistency and trains an RL agent to maximize the learned rewards. In practice, however, inverse RL methods such as XIRL are found to require limited visual diversity in demonstrations. Our work extends XIRL to the setting of diverse videos by introducing a graph abstraction that models agent-object and object-object interactions while still enforcing temporal cycle-consistency.
