VIDEO ANALYSIS OF SKILL AND TECHNIQUE (VAST) MACHINE LEARNING TO ASSESS SURGEONS PERFORMING ROBOTIC PROSTATECTOMY: PD46 04
INTRODUCTION AND OBJECTIVES: A surgeon's technical skill may be a major determinant of patient outcomes. Because robotic surgery can be recorded, computer vision video analysis of skill and technique (VAST) methods may have advantages for assessment that is objective and scalable. To test the hypothesis that specific features in a video can categorize skill, we studied crowdsourced annotated videos of surgeons performing robotic prostatectomy and applied machine learning to determine skill. METHODS: Videos of the anastomosis from 12 surgeons in the Michigan Urological Surgery Improvement Collaborative underwent blinded review by 25 peer surgeons using the Global Evaluative Assessment of Robotic Skills (GEARS) tool (max score 25) Surgeons were categorized into low and high skill based on 0bimanual dexterity0 and 0efficiency0. Robotic instruments were annotated by crowdworkers via a custom designed Mechanical Turk platform. Using the videos we trained a linear support vector machine (SVM) sampling consecutive frames to study VAST metrics for instruments including velocity, trajectory, smoothness of movement, and relationship to contralateral instrument. We applied the SVM to learn and classify videos into high/low skill. To evaluate performance we used 11 videos as training, and tested on the remaining 1 video, repeating it 12 times and averaged the accuracy. RESULTS: GEARS scores ranged from 15.75 to 23.11, with 9 and 3 surgeons categorized into high and low skill, respectively. In total, 146,309 video frames were annotated by 925 crowdworkers. Instrument annotation included individual points as well as wristed joint movement (Figure) SVM accuracy in skill categorization using individual points on an instrument was 83.3% Accuracy improved to 91.7% when we assessed joint movement. When we combined assessment with the contralateral instrument, accuracy was 100% in categorizing binary skill level. Instrument metrics most closely related to skill prediction were relationship between needle driver forceps and joint, acceleration, and velocity. CONCLUSIONS: Computer video analysis can be used to predict skill in practicing robotic surgeons. In the future, methods utilizing deep learning to track instruments and calculate skill, may have significant implications for credentialing and quality improvement. Source of Funding: Blue Cross Blue Shield of Michigan; Intuitive Surgical