Adversarial Imitation Learning between Agents with Different Numbers of State Dimensions
Designing a reward is difficult in reinforcement learning because there is often a tradeoff between the two roles played by the reward, one being to make an agent behave as desired and the other being to allow an agent to learn a policy easily. Meanwhile, adversarial imitation learning can eliminate the design of a reward by estimating the reward that the expert is maximizing from trajectories of the expert. However, previous methods of adversarial imitation learning can be applied only when the number of state dimensions is the same for the agent and expert. We here define the new problem of adversarial imitation learning between agents having different numbers of state dimensions and propose a method of solving the problem. Our method makes the numbers of dimensions equal by inserting an arbitrary constant into the smaller state so that the feature extractor can be shared between agents. In addition, we verify whether domain adaptation can extract a feature independent of the number of dimensions. We find that the imitation rate of our method is at least 80% in a simulation of a reaching task while rates of previous methods are no more than 55% We also find that our method is robust against the value to be inserted into the state, the dimension into which the value is inserted, and the numbers of joints of the agent and expert.
2019 IEEE Second International Conference on Artificial Intelligence and Knowledge Engineering (AIKE)