Surgical robots, such as Intuitive Surgical's da Vinci® Surgical System, have brought about more efficient surgeries by improving the dexterity and reducing fatigue of the surgeon through teleoperational control. While these systems are already providing great care to patients, they have also opened the door to a variety of research including surgeon performance metrics [1]}, remote teleoperation [2]}, and surgical task automation [3]}. Surgical task automation have furthermore been an increasing area of research in an effort to improve patient throughput, reduce quality-of-care variance among surgeries, and potentially deliver automated surgery in the future. Automation efforts includes automating subtasks includes knot tying [4]}, [5]}, compliant object manipulation [6]}, endoscopic motions [7]}, surgical cutting [8]}, [9]}, suture needle manipulation [10]}, [11]} and debris removal [12]}, [13]}. One of the challenges moving forward for the surgical robotics community is that despite these successes, many have been based around hand-crafted control policies that can be difficult to both develop at scale and generalize across a variety of environments. RL offers a solution to these problems by shifting human time-costs and the limitations of feature- and controller-design, to autonomously learning these via large-scale, faster-than-real-time, parallelized simulations (Fig. REF ).
