To bridge reinforcement learning with surgical robotics, simulation environments need to be provided such that RL algorithms of past, present, and future can be prototyped and tested on. OpenAI's Gym [1]} has offered perhaps one of the most impactful resource to the RL community for testing a range of environments and domains through a common API, and has been wildly successful in engaging a broad range of machine learning researchers, engineers, and hobbyists. This is primarily due to its incredibly simple interface, with mainly four function calls (make, reset, step,  and render) that allows all kinds of scenarios to be learned. In this paper, we aim to bring RL to the surgical robotics domain via the first open-sourced reinforcement learning environments for the da Vinci Research Kit (dVRK) [2]}, called dVRL. We are motivated to engage the broader community that include surgical robotics and also non-domain experts, such that RL enthusiasts with no domain knowledge of surgery can still easily prototype their algorithms with such an environment and contribute to solutions that would have real world significance to robotic surgery and the patients that undergo those procedures. To move towards this goal, we present the following novel contributions:
