Reinforcement Learning (RL) is a framework that has been utilized in areas largely outside of surgical robotics to incorporate artificial intelligence to a variety of problems [1]}. The problems solved, however, have mostly been in extremely structured environments such as video games [2]} and board games [3]}. There has also been recent success in robotic manipulation and specifically grasping, and with evidence that the learned policies are transferable from simulation to real robots [4]}, [5]}. These successes have hinged on having simulation environments that are lightweight and efficient, as RL tends to require thousands to millions of simulated attempts to evaluate and explore policy options. For robotics, this is crucial for real-world use of RL due to the impracticality of running millions of attempts on a physical system only to learn a low-level behavior.
<FIGURE>