Autonomous drifting using simulation aided reinforcement learning
We introduce a framework that combines simple and complex continuous state action simulators with a real world robot to efficiently find good control policies, while minimizing the number of samples needed from the physical robot. The framework combines the strengths of various simulation levels by first finding optimal policies in a simple model, and then using that solution to initialize a gradient based learner in a more complex simulation. The policy and transition dynamics from the complex simulation are in turn used to guide the learning in the physical world. A method is developed for transferring information gathered in the physical world back to the learning agent in the simulation. The new information is used to re evaluate whether the original simulated policy is still optimal given the updated knowledge from the real world. This reverse transfer is critical to minimizing samples from the physical world. The new framework is demonstrated on a robotic car learning to perform controlled drifting maneuvers. A video of the car's performance can be found at https: /youtu.be/opsmd5yuBF0.
2016 IEEE International Conference on Robotics and Automation (ICRA)