A Survey of auditory display in image-guided interventions <s> Parameter-mapping models <s> Current medical visualization technology intended for positional guidance in surgical applications may only ever have limited utility in the operating room due to the preexisting visual requirements of surgical practice. Additionally, visual systems impose limits as a result of their high latency, poor image resolution, problems with stereopsis and physical strain upon the user. Audio technology is relatively unexamined in the broad range of available methodologies for medical devices. The potential to translate surgical instrument position into audio feedback presents a novel solution to the human factors and engineering problems faced by visual display technology because audio technology employs a rich and as yet unburdened sensory modality. We describe an experimental system we have developed for investigating this new interface design approach using commercially available hardware. <s> BIB001 </s> A Survey of auditory display in image-guided interventions <s> Parameter-mapping models <s> One unique feature of mixed and augmented reality (MR/AR) systems is that hidden and occluded objects an be readily visualized. We call this specialized use of MR/AR, obscured information visualization (OIV). In this paper, we describe the beginning of a research program designed to develop such visualizations through the use of principles derived from perceptual psychology and cognitive science. In this paper we surveyed the cognitive science literature as it applies to such visualization tasks, described experimental questions derived from these cognitive principles, and generated general guidelines that can be used in designing future OIV systems (as well improving AR displays more generally). We also report the results from an experiment that utilized a functioning AR-OIV system: we found that in relative depth judgment, subjects reported rendered objects as being in front of real-world objects, except when additional occlusion and motion cues were presented together. <s> BIB002 </s> A Survey of auditory display in image-guided interventions <s> Parameter-mapping models <s> In this study the authors measured the effect of auditory feedback during image-guided surgery (IGS) in a phantom model and in a clinical setting. In the phantom setup, advanced IGS with complementary auditory feedback was compared with results obtained with 2 routine forms of IGS, either with an on-screen image display or with image injection via a microscope. The effect was measured by means of volumetric resection assessments. The authors also present their first clinical data concerning the effects of complementary auditory feedback on instrument handling during image-guided neurosurgery. When using image-injection through the microscope for navigation, however, resection quality was significantly worse. In the clinical portion of the study, the authors performed resections of cerebral mass lesions in 6 patients with the aid of auditory feedback. Instrument tip speeds were slightly (although significantly) influenced by this feedback during resection. Overall, the participating neurosurgeons reported that the auditory feedback helped in decision-making during resection without negatively influencing instrument use. Postoperative volumetric imaging studies revealed resection rates of > or = 95% when IGS with auditory feedback was used. There was only a minor amount of brain shift, and postoperative resection volumes corresponded well with the preoperative intentions of the neurosurgeon. Although the results of phantom surgery with auditory feedback revealed no significant effect on resection quality or extent, auditory cues may help prevent damage to eloquent brain structures. <s> BIB003 </s> A Survey of auditory display in image-guided interventions <s> Parameter-mapping models <s> BACKGROUND ::: A neuronavigation interface with extended function as compared with current systems was developed to aid during temporal bone surgery. The interface, named EVADE, updates the prior anatomical image and visualizes the bone drilling process virtually in real-time without need for intra-operative imaging. Furthermore, EVADE continuously calculates the distance from the drill tip to segmented temporal bone critical structures (e.g. the sigmoid sinus and facial nerve) and produces audiovisual warnings if the surgeon drills in too close vicinity. The aim of this study was to evaluate the accuracy and surgical utility of EVADE in physical phantoms. ::: ::: ::: METHODOLOGY/PRINCIPAL FINDINGS ::: We performed 228 measurements assessing the position accuracy of tracking a navigated drill in the operating theatre. A mean target registration error of 1.33±0.61 mm with a maximum error of 3.04 mm was found. Five neurosurgeons each drilled two temporal bone phantoms, once using EVADE, and once using a standard neuronavigation interface. While using standard neuronavigation the surgeons damaged three modeled temporal bone critical structures. No structure was hit by surgeons utilizing EVADE. Surgeons felt better orientated and thought they had improved tumor exposure with EVADE. Furthermore, we compared the distances between surface meshes of the virtual drill cavities created by EVADE to actual drill cavities: average maximum errors of 2.54±0.49 mm and -2.70±0.48 mm were found. ::: ::: ::: CONCLUSIONS/SIGNIFICANCE ::: These results demonstrate that EVADE gives accurate feedback which reduces risks of harming modeled critical structures compared to a standard neuronavigation interface during temporal bone phantom drilling. <s> BIB004 </s> A Survey of auditory display in image-guided interventions <s> Parameter-mapping models <s> Radiofrequency ablation is applied to treat a lesion using a needle inserted into the patient, which delivers local radiofrequency energy. Guided surgical methods allow surgeons to view the placement of the needle in relation to the patient to aid in guiding the tip of the needle to the target point. Unfortunately, such methods require that surgeons remove attention from the patient in order to receive guidance information from a screen. We introduce a novel method to align and insert an ablation needle using auditory display, allowing the surgeon to retain attention on the patient. First evaluation results show that novice users can successfully guide a needle towards a target point using primarily auditory display. We hypothesize that successful auditory display will lead to increased attention on the patient and reduce unnecessary operator head and neck movements. <s> BIB005 </s> A Survey of auditory display in image-guided interventions <s> Parameter-mapping models <s> Background ::: ::: An alternative mode of interaction with navigation systems for open liver surgery was requested. Surgeons who use such systems are impeded by having to constantly switch between viewing the navigation system screen and the patient during an operation. ::: ::: ::: ::: Methods ::: ::: To this end, an auditory display system for open liver surgery is introduced with support for guiding the tracked instrument towards and remaining on a predefined resection line. To evaluate the method, a clinically orientated user study with 12 surgeons was conducted. ::: ::: ::: ::: Results ::: ::: It is shown in qualitative results from the user study that the proposed auditory display is recognized as a useful addition to the current visual mode of interaction. It was revealed in a statistical analysis that participants spent less time looking on the screen (10% vs. 96%). Accuracy for resection guidance was significantly improved when using auditory display as an additional information channel (0.6 vs. 1.4 mm); however, the overall time for the resection task was shorter without auditory display (47 vs. 24 s). ::: ::: ::: ::: Conclusions ::: ::: By reducing dependence on the visual modality during resection guidance, the auditory display is well suited to become integrated in navigation systems for liver surgery. Copyright © 2012 John Wiley & Sons, Ltd. <s> BIB006 </s> A Survey of auditory display in image-guided interventions <s> Parameter-mapping models <s> Image-guided medical interventions more frequently rely on Augmented Reality (AR) visualization to enable surgical navigation. Current systems use 2-D monitors to present the view from external cameras, which does not provide an ideal perception of the 3-D position of the region of interest. Despite this problem, most research targets the direct overlay of diagnostic imaging data, and only few studies attempt to improve the perception of occluded structures in external camera views. The focus of this paper lies on improving the 3-D perception of an augmented external camera view by combining both auditory and visual stimuli in a dynamic multi-sensory AR environment for medical applications. Our approach is based on Temporal Distance Coding (TDC) and an active surgical tool to interact with occluded virtual objects of interest in the scene in order to gain an improved perception of their 3-D location. Users performed a simulated needle biopsy by targeting virtual lesions rendered inside a patient phantom. Experimental results demonstrate that our TDC-based visualization technique significantly improves the localization accuracy, while the addition of auditory feedback results in increased intuitiveness and faster completion of the task. <s> BIB007 </s> A Survey of auditory display in image-guided interventions <s> Parameter-mapping models <s> PURPOSE ::: In this article, we systematically examine the current state of research of systems that focus on touchless human-computer interaction in operating rooms and interventional radiology suites. We further discuss the drawbacks of current solutions and underline promising technologies for future development. ::: ::: ::: METHODS ::: A systematic literature search of scientific papers that deal with touchless control of medical software in the immediate environment of the operation room and interventional radiology suite was performed. This includes methods for touchless gesture interaction, voice control and eye tracking. ::: ::: ::: RESULTS ::: Fifty-five research papers were identified and analyzed in detail including 33 journal publications. Most of the identified literature (62 %) deals with the control of medical image viewers. The others present interaction techniques for laparoscopic assistance (13 %), telerobotic assistance and operating room control (9 % each) as well as for robotic operating room assistance and intraoperative registration (3.5 % each). Only 8 systems (14.5 %) were tested in a real clinical environment, and 7 (12.7 %) were not evaluated at all. ::: ::: ::: CONCLUSION ::: In the last 10 years, many advancements have led to robust touchless interaction approaches. However, only a few have been systematically evaluated in real operating room settings. Further research is required to cope with current limitations of touchless software interfaces in clinical environments. The main challenges for future research are the improvement and evaluation of usability and intuitiveness of touchless human-computer interaction and the full integration into productive systems as well as the reduction of necessary interaction steps and further development of hands-free interaction. <s> BIB008 </s> A Survey of auditory display in image-guided interventions <s> Parameter-mapping models <s> Purpose ::: During medical needle placement using image-guided navigation systems, the clinician must concentrate on a screen. To reduce the clinician’s visual reliance on the screen, this work proposes an auditory feedback method as a stand-alone method or to support visual feedback for placing the navigated medical instrument, in this case a needle. <s> BIB009
In contrast to alerts and auditory icons, parameter mapping links continuous changes in one set of data to continuous changes in audio parameters, providing a higher level of complexity. In essence, the underlying data delivered by the navigation system are used to "play" a real-time software instrument according to those changes. Because audio has a wide range of parameters  that may be altered, such as frequency, intensity, and timbre, continuous parameter mapping is also suitable for displaying multivariate data. This technique attempts to make the listener an active participant in the listening process by providing interactive, changing mappings that relate data to audio. This method is useful for smoothly representing continuous changes in events. In image-guided intervention support systems which implement parameter-mapping auditory display, the tracked instrument itself is in essence the physical musical instrument: The clinician plays the real-time software instrument by moving the tracked instrument. The range of parameter mappings found in the literature extend from fairly simple frequency and volume mappings BIB004 BIB003 up to complex methods such as 3-D audio spatialization and wave-terrain synthesis BIB001  . For volumetric resection for neuronavigation, Woerdeman et al. BIB003 adapt a previous approach of the group  , which employed a simple alert, to play back a parameter-mapping alert. They describe a "soft warning sound (an intermittent pure tone)" with a duration of 0.1 seconds that plays back at a rate of 3 times per second at a distance of 5 mm from the tumor outline. Upon entering the 5-mm threshold, volume and tone frequency increased proportionally until the outline of the tumor was reached. After encroaching the tumor outline, a "continuous pure tone" was played back. Voormolen et al. BIB004 cite and employ this method for temporal bone drilling but do not describe further adaptation details. Biopsy needle placement support is described by Bork et al. BIB007 , who use a method of parameter mapping called temporal distance coding BIB002 , in which the time an object is rendered depends on the distance from the tracked instrument. In this case, auditory temporal distance coding allowed playback of the distance from the tip of the biopsy needle to objects of interest within the AR environment. Virtual "spheres" propagate from the needle tip at a certain speed. The longer it takes for these spheres to collide with the objects of interest, the longer a metronome sound is played back. Once an object is "hit," a bell tone is played. Thus, the more metronome sounds are played before a bell tone is played, the further the object is from the needle tip. The user is not explicitly guided toward a target using auditory display, but rather receives information concerning the distance of objects of interest in the area. A different method for needle placement is investigated by Black et al. BIB005 BIB009 . This auditory display encodes the distance of the tip of the needle to the correct insertion point, the distance of the shaft to the correct position, and the depth as the distance of the tip to the target lesion. Two auditory display methods are described. In both methods, the task of needle placement is split into tip placement, handle placement, and insertion phases. The first method employs a tone with a moving pitch and a reference tone; the pitch of the moving tone is mapped to the distance in the y-axis. These are brought together, creating an auditory display that mimics tuning an instrument. Distance in the x-axis is mapped to the inter-onset interval of train of tones, from 250 ms at the outer edge of the navigation area to 100 ms at the center. The second method further separates motion along the x-axis and y-axis. Placement along the x-axis is first performed using changes in inter-onset interval, repeated again for movement along the y-axis corridor. After correct tip and handle placement, the needle is inserted and depth to target is mapped to the increasing pitch of 10 consecutive tones, whereafter a bell tone is played back upon reaching the target lesion. Hansen et al. BIB006 support resection line marking for open liver surgery with a parameter-mapping auditory display. In this method, the navigation system delivers the nearest distance between the instrument tip and the planned resection line. The distance is divided into three margins: "safe," "warning," and "outside." When the instrument is in the safe margin, signaling to the clinician that the position is correct, a confirmation tone is played back with a frequency of 698.5 Hz and an inter-onset interval of 660 ms at the center of the safe margin and 180 ms at the edge of the safe margin. In the warning margin, the distance is mapped to inter-onset interval, pitch, and tone length. Pitches to the left of the resection line become consecutively lower, while those to the right of the resection line become consecutively higher, thus providing directional information. Outside the warning zone, no sound is played to prevent unwanted sound when the instrument is not near the resection line. Wegner and Karron BIB001 map a discrete error function in the plane perpendicular to the trajectory path to MIDI 2 tones. For placement in this plane, a chordal drone is produced, with deviations from the correct placement producing inharmonicity. Another tone was produced at regular points along the trajectory path to "tag" the distance traveled. A second method  employs beat interference between 3 pairs of sinusoids which correspond to 3 axes in space. By reducing the beat interference between each of the 3 pairs, correct position is found. Figure 1 visualizes the primary mapping approaches encountered in the literature. BIB008 Musical Instrument Digital Interface, a protocol for electronic musical instrument communication. 