A Survey of auditory display in image-guided interventions <s> Laboratory evaluations <s> OBJECTIVES ::: Direct haptic (force or tactile) feedback is not yet available in commercial robotic surgical systems. Previous work by our group and others suggests that haptic feedback might significantly enhance the execution of surgical tasks requiring fine suture manipulation, specifically those encountered in cardiothoracic surgery. We studied the effects of substituting direct haptic feedback with visual and auditory cues to provide the operating surgeon with a representation of the forces he or she is applying with robotic telemanipulators. ::: ::: ::: METHODS ::: Using the robotic da Vinci surgical system (Intuitive Surgical, Inc, Sunnyvale, Calif), we compared applied forces during a standardized surgical knot-tying task under 4 different sensory-substitution scenarios: no feedback, auditory feedback, visual feedback, and combined auditory-visual feedback. ::: ::: ::: RESULTS ::: The forces applied with these sensory-substitution modes more closely approximate suture tensions achieved under ideal haptic conditions (ie, hand ties) than forces applied without such sensory feedback. The consistency of applied forces during robot-assisted suture tying aided by visual feedback or combined auditory-visual feedback sensory substitution is superior to that achieved with hand ties. Robot-assisted ties aided with auditory feedback revealed levels of consistency that were generally equivalent or superior to those attained with hand ties. Visual feedback and auditory feedback improve the consistency of robotically applied forces. ::: ::: ::: CONCLUSIONS ::: Sensory substitution, in the form of visual feedback, auditory feedback, or both, confers quantifiable advantages in applied force accuracy and consistency during the performance of a simple surgical task. <s> BIB001 </s> A Survey of auditory display in image-guided interventions <s> Laboratory evaluations <s> NASA-TLX is a multi-dimensional scale designed to obtain workload estimates from one or more operators while they are performing a task or immediately afterwards. The years of research that precede... <s> BIB002 </s> A Survey of auditory display in image-guided interventions <s> Laboratory evaluations <s> In this study the authors measured the effect of auditory feedback during image-guided surgery (IGS) in a phantom model and in a clinical setting. In the phantom setup, advanced IGS with complementary auditory feedback was compared with results obtained with 2 routine forms of IGS, either with an on-screen image display or with image injection via a microscope. The effect was measured by means of volumetric resection assessments. The authors also present their first clinical data concerning the effects of complementary auditory feedback on instrument handling during image-guided neurosurgery. When using image-injection through the microscope for navigation, however, resection quality was significantly worse. In the clinical portion of the study, the authors performed resections of cerebral mass lesions in 6 patients with the aid of auditory feedback. Instrument tip speeds were slightly (although significantly) influenced by this feedback during resection. Overall, the participating neurosurgeons reported that the auditory feedback helped in decision-making during resection without negatively influencing instrument use. Postoperative volumetric imaging studies revealed resection rates of > or = 95% when IGS with auditory feedback was used. There was only a minor amount of brain shift, and postoperative resection volumes corresponded well with the preoperative intentions of the neurosurgeon. Although the results of phantom surgery with auditory feedback revealed no significant effect on resection quality or extent, auditory cues may help prevent damage to eloquent brain structures. <s> BIB003 </s> A Survey of auditory display in image-guided interventions <s> Laboratory evaluations <s> BACKGROUND ::: A neuronavigation interface with extended function as compared with current systems was developed to aid during temporal bone surgery. The interface, named EVADE, updates the prior anatomical image and visualizes the bone drilling process virtually in real-time without need for intra-operative imaging. Furthermore, EVADE continuously calculates the distance from the drill tip to segmented temporal bone critical structures (e.g. the sigmoid sinus and facial nerve) and produces audiovisual warnings if the surgeon drills in too close vicinity. The aim of this study was to evaluate the accuracy and surgical utility of EVADE in physical phantoms. ::: ::: ::: METHODOLOGY/PRINCIPAL FINDINGS ::: We performed 228 measurements assessing the position accuracy of tracking a navigated drill in the operating theatre. A mean target registration error of 1.33±0.61 mm with a maximum error of 3.04 mm was found. Five neurosurgeons each drilled two temporal bone phantoms, once using EVADE, and once using a standard neuronavigation interface. While using standard neuronavigation the surgeons damaged three modeled temporal bone critical structures. No structure was hit by surgeons utilizing EVADE. Surgeons felt better orientated and thought they had improved tumor exposure with EVADE. Furthermore, we compared the distances between surface meshes of the virtual drill cavities created by EVADE to actual drill cavities: average maximum errors of 2.54±0.49 mm and -2.70±0.48 mm were found. ::: ::: ::: CONCLUSIONS/SIGNIFICANCE ::: These results demonstrate that EVADE gives accurate feedback which reduces risks of harming modeled critical structures compared to a standard neuronavigation interface during temporal bone phantom drilling. <s> BIB004 </s> A Survey of auditory display in image-guided interventions <s> Laboratory evaluations <s> Purpose We developed a surgical navigation system that warns the surgeon with auditory and visual feedback to protect the facial nerve with real-time monitoring of the safe region during drilling. <s> BIB005 </s> A Survey of auditory display in image-guided interventions <s> Laboratory evaluations <s> Background ::: ::: An alternative mode of interaction with navigation systems for open liver surgery was requested. Surgeons who use such systems are impeded by having to constantly switch between viewing the navigation system screen and the patient during an operation. ::: ::: ::: ::: Methods ::: ::: To this end, an auditory display system for open liver surgery is introduced with support for guiding the tracked instrument towards and remaining on a predefined resection line. To evaluate the method, a clinically orientated user study with 12 surgeons was conducted. ::: ::: ::: ::: Results ::: ::: It is shown in qualitative results from the user study that the proposed auditory display is recognized as a useful addition to the current visual mode of interaction. It was revealed in a statistical analysis that participants spent less time looking on the screen (10% vs. 96%). Accuracy for resection guidance was significantly improved when using auditory display as an additional information channel (0.6 vs. 1.4 mm); however, the overall time for the resection task was shorter without auditory display (47 vs. 24 s). ::: ::: ::: ::: Conclusions ::: ::: By reducing dependence on the visual modality during resection guidance, the auditory display is well suited to become integrated in navigation systems for liver surgery. Copyright © 2012 John Wiley & Sons, Ltd. <s> BIB006 </s> A Survey of auditory display in image-guided interventions <s> Laboratory evaluations <s> OBJECTIVES/HYPOTHESIS ::: Image-guided surgery (IGS) systems are frequently utilized during cranial base surgery to aid in orientation and facilitate targeted surgery. We wished to assess the performance of our recently developed localized intraoperative virtual endoscopy (LIVE)-IGS prototype in a preclinical setting prior to deployment in the operating room. This system combines real-time ablative instrument tracking, critical structure proximity alerts, three-dimensional virtual endoscopic views, and intraoperative cone-beam computed tomographic image updates. ::: ::: ::: STUDY DESIGN ::: Randomized-controlled trial plus qualitative analysis. ::: ::: ::: METHODS ::: Skull base procedures were performed on 14 cadaver specimens by seven fellowship-trained skull base surgeons. Each subject performed two endoscopic transclival approaches; one with LIVE-IGS and one using a conventional IGS system in random order. National Aeronautics and Space Administration Task Load Index (NASA-TLX) scores were documented for each dissection, and a semistructured interview was recorded for qualitative assessment. ::: ::: ::: RESULTS ::: The NASA-TLX scores for mental demand, effort, and frustration were significantly reduced with the LIVE-IGS system in comparison to conventional navigation (P < .05). The system interface was judged to be intuitive and most useful when there was a combination of high spatial demand, reduced or absent surface landmarks, and proximity to critical structures. The development of auditory icons for proximity alerts during the trial better informed the surgeon while limiting distraction. ::: ::: ::: CONCLUSIONS ::: The LIVE-IGS system provided accurate, intuitive, and dynamic feedback to the operating surgeon. Further refinements to proximity alerts and visualization settings will enhance orientation while limiting distraction. The system is currently being deployed in a prospective clinical trial in skull base surgery. <s> BIB007 </s> A Survey of auditory display in image-guided interventions <s> Laboratory evaluations <s> Image-guided navigation (IGN) systems provide automation support of intra-operative information analysis and decision-making for surgeons. Previous research showed that navigated-control (NC) systems which represent high levels of decision-support and directly intervene in surgeons' workflow provide benefits with respect to patient safety and surgeons' physiological stress but also involve several cost effects (e.g. prolonged surgery duration, reduced secondary-task performance). It was hypothesised that less automated distance-control (DC) systems would provide a better solution in terms of human performance consequences. N = 18 surgeons performed a simulated mastoidectomy with NC, DC and without IGN assistance. Effects on surgical performance, physiological effort, workload and situation awareness (SA) were compared. As expected, DC technology had the same benefits as the NC system but also led to less unwanted side effects on surgery duration, subjective workload and SA. This suggests that IGN systems ... <s> BIB008 </s> A Survey of auditory display in image-guided interventions <s> Laboratory evaluations <s> Image-guided medical interventions more frequently rely on Augmented Reality (AR) visualization to enable surgical navigation. Current systems use 2-D monitors to present the view from external cameras, which does not provide an ideal perception of the 3-D position of the region of interest. Despite this problem, most research targets the direct overlay of diagnostic imaging data, and only few studies attempt to improve the perception of occluded structures in external camera views. The focus of this paper lies on improving the 3-D perception of an augmented external camera view by combining both auditory and visual stimuli in a dynamic multi-sensory AR environment for medical applications. Our approach is based on Temporal Distance Coding (TDC) and an active surgical tool to interact with occluded virtual objects of interest in the scene in order to gain an improved perception of their 3-D location. Users performed a simulated needle biopsy by targeting virtual lesions rendered inside a patient phantom. Experimental results demonstrate that our TDC-based visualization technique significantly improves the localization accuracy, while the addition of auditory feedback results in increased intuitiveness and faster completion of the task. <s> BIB009 </s> A Survey of auditory display in image-guided interventions <s> Laboratory evaluations <s> Purpose ::: During medical needle placement using image-guided navigation systems, the clinician must concentrate on a screen. To reduce the clinician’s visual reliance on the screen, this work proposes an auditory feedback method as a stand-alone method or to support visual feedback for placing the navigated medical instrument, in this case a needle. <s> BIB010
The majority (12) of the reviewed literature describe evaluations in laboratory conditions on phantoms. Of these, 4 of 15 also performed a clinical evaluation described in the same article; see "Clinical evaluations" section. See Table 3 for an overview. For the task of telerobotic suture tying reported by Kitagawa et al. BIB001 , 5 surgeons completed suture ties using no feedback, auditory feedback, visual feedback, and audiovisual feedback after 1 hour training with the robot system. Findings indicate that consistency of tying tension with sensory substitution using visual and audiovisual displays were superior to those of hand ties and that the consistency of tying tension using auditory display were comparable to those of hand ties. Willems et al.  compare volume resection on floral foam phantoms with 3 experienced surgeons who each completed one resection each using both auditory display and standard visual navigation. Results indicate that using auditory display, the similarity of the resected volume to target volume increased marginally, the amount of target tissue not removed was reduced, but that the amount of non-target tissue removed increased. Using a similar task, Woerdeman et al. BIB003 describe an evaluation with 4 surgeons performing volume resection with auditory display, conventional IGS display, and a heads-up display. Task completion time between auditory display, conventional display and heads-up display did not differ, and target volume removal did not differ between auditory and conventional displays. However, auditory display was subjectively perceived to improve performance compared to conventional display. Strauss et al.  compared surgeon-reported and actual distance measurement points from instrument to risk structures with 5 "advanced beginner" surgeons using a combined audiovisual collision-warning system. Results indicate that the audiovisual display improved reported accuracy 76% over conventional display. Voormolen et al. BIB004 evaluate 5 surgeons each performing a temporal bone drilling task in two phantoms, once with conventional image guidance and once with the combined audiovisual assistance. Using the audiovisual system, no critical structures were damaged (opposed to three structures using conventional methods), and participants reported improved subjective orientation and improved tumor exposure with the system. Cho et al. BIB005 describe a laboratory study with one inexperienced surgeon who drilled 10 bone phantoms, 5 using an audiovisual display and 5 without navigation. The drill distance to the facial nerve was recorded to determine when the surgeon encroached the safe margin of 2 mm to the nerve and when the nerve was hit. Using no navigation, the nerve was hit in 4 of 5 attempts, whereas with audiovisual display the nerve was hit once. In addition, the uniformity of the safe margin in the resected area appeared greater with audiovisual display. Hansen et al. BIB006 compare resection line marking on a floral foam phantom with 12 surgeons using combined audiovisual and conventional 3-D navigation display. Findings indicate that the auditory display reduced the percent of time viewing the screen from 96% using visual display to 10% using combined audiovisual display. Auditory display increased accuracy of the marking task, but also increased task completion time. Dixon et al. BIB007 report an evaluation of 14 cadaver specimens with 7 surgeons who each performed dissection and clivus ablation on 2 heads, once each using conventional display and audiovisual display. Using the audiovisual display reduced NASA-TLX perceived workload BIB002 scores for mental demand, effort, and frustration. Luz et al. BIB008 evaluate 18 surgeons who performed a simulated mastoidectomy surgery using navigated control and distance control, as well as manually without image guid- ance. Effects on surgical performance, physiological effort, workload, and situation awareness were evaluated. Using simpler distance control instead of more complex navigated control led to faster task completion and lower subjective workload. Bork et al. BIB009 evaluate lesion targeting using a biopsy needle with 15 non-clinical participants. Each participant completed 3 attempts using simple lesion overlay, auditoryonly feedback, visual feedback, and audiovisual feedback. Participants verbally confirmed reaching each lesion point. Results show that targeting using combined audiovisual feedback resulted in most target hits and least localization error. Auditory, visual, and audiovisual displays improve accuracy but resulted in slower task completion times than the simple overlay. Audiovisual display outperformed auditory and visual display with respect to accuracy, task completion time, and number of lesions hit. Black et al. BIB010 evaluate tip and handle placement of an ablation needle with 12 non-clinical participants. Each participant completed tip and needle placement using auditory, visual, and combined audiovisual display. For each placement, the participants verbally confirmed when tip and needle were placed to their satisfaction. Results showed that combined audiovisual display provided the highest accuracy of the three feedback modalities. Purely auditory display exhibited longer task completion times and increased subjective workload compared to visual and combined audiovisual display. 