We collect \(n=150\)  demonstration trajectories on three Mujoco [1]} simulated environments: Hopper-v2, Walker2d-v2, and HalfCheetah-v2, using a SAC expert trained for 1M timestep, using \(n\)  random seeds. To evaluate the performance with varying amounts of demonstrations, we use a subset of 40, 20 or 10 trajectories (respectively \((40, 20, 10) \cdot 10^3\)  data points) as the training dataset \(D_{expert}\)  for the density estimator. We compare IL-flOw to the three variants of the f-IRL [2]} algorithm, as well as a state only version of MaxEnt IRL [3]}, using the implementations provided by [2]}.
MaxEnt IRL minimizes forward KL divergence in trajectory space under the maximum entropy RL framework.
f-IRL is an imitation learning from observation algorithm that operates by state marginal distribution matching, through optimization of the analytical gradient of any f-divergence (JS, FKL, RKL). It also learns a stationary reward that is reusable, although the imitation agent still faces a moving reward function in training through their iterative training process, while for IL-flOw the reward learning and the RL process are sequential, to convergence.
