Touchless computer interfaces in hospitals: A review <s> use a combination of voice commands for discrete commands and mode changes and gestures for the control of continuous image parameters. <s> For a lot of applications, and particularly for medical intra-operative applications, the exploration of and navigation through 3-D image data provided by sensors like ToF (time-of-flight) cameras, MUSTOF (multisensor-time-of-flight) endoscopes or CT (computed tomography) [8], requires a user-interface which avoids physical interaction with an input device. Thus, we process a touchless user-interface based on gestures classified by the data provided by a ToF camera. Reasonable and necessary user interactions are described. For those interactions a suitable set of gestures is introduced. A user-interface is then proposed, which interprets the current gesture and performs the assigned functionality. For evaluating the quality of the developed user-interface we considered the aspects of classification rate, real-time applicability, usability, intuitiveness and training time. The results of our evaluation show that our system, which provides a classification rate of 94.3% at a framerate of 11 frames per second, satisfactorily addresses all these quality requirements. <s> BIB001 </s> Touchless computer interfaces in hospitals: A review <s> use a combination of voice commands for discrete commands and mode changes and gestures for the control of continuous image parameters. <s> In this paper, an open-source system for a controller-free, highly interactive exploration of medical images is presented. By using a Microsoft Xbox Kinect™ as the only input device, the system's user interface allows users to interact at a distance through hand and arm gestures. The paper also details the interaction techniques we have designed specifically for the deviceless exploration of medical imaging data. Since the user interface is touch-free and does not require complex calibration steps, it is suitable for use in operating rooms, where non-sterilizable devices cannot be used. <s> BIB002 </s> Touchless computer interfaces in hospitals: A review <s> use a combination of voice commands for discrete commands and mode changes and gestures for the control of continuous image parameters. <s> Keyboards, mice, and touch screens are a potential source of infection or contamination in operating rooms, intensive care units, and autopsy suites. The authors present a low-cost prototype of a system, which allows for touch-free control of a medical image viewer. This touch-free navigation system consists of a computer system (IMac, OS X 10.6 Apple, USA) with a medical image viewer (OsiriX, OsiriX foundation, Switzerland) and a depth camera (Kinect, Microsoft, USA). They implemented software that translates the data delivered by the camera and a voice recognition software into keyboard and mouse commands, which are then passed to OsiriX. In this feasibility study, the authors introduced 10 medical professionals to the system and asked them to re-create 12 images from a CT data set. They evaluated response times and usability of the system compared with standard mouse/keyboard control. Users felt comfortable with the system after approximately 10 minutes. Response time was 120 ms. Users required 1.4 times more time to re-create an image with gesture control. Users with OsiriX experience were significantly faster using the mouse/keyboard and faster than users without prior experience. They rated the system 3.4 out of 5 for ease of use in comparison to the mouse/keyboard. The touch-free, gesture-controlled system performs favorably and removes a potential vector for infection, protecting both patients and staff. Because the camera can be quickly and easily integrated into existing systems, requires no calibration, and is low cost, the barriers to using this technology are low. <s> BIB003 </s> Touchless computer interfaces in hospitals: A review <s> use a combination of voice commands for discrete commands and mode changes and gestures for the control of continuous image parameters. <s> Review of prior and real-time patient images is critical during an interventional radiology procedure; however, it often poses the challenge of efficiently reviewing images while maintaining a sterile field. Although interventional radiologists can "scrub out" of the procedure, use sterile console covers, or verbally relay directions to an assistant, the ability of the interventionalist to directly control the images without having to touch the console could offer potential gains in terms of sterility, procedure efficiency, and radiation reduction. The authors investigated a potential solution with a low-cost, touch-free motion-tracking device that was originally designed as a video game controller. The device tracks a person's skeletal frame and its motions, a capacity that was adapted to allow manipulation of medical images by means of hand gestures. A custom software program called the Touchless Radiology Imaging Control System translates motion information obtained with the motion-tracking device into commands to review images on a workstation. To evaluate this system, 29 radiologists at the authors' institution were asked to perform a set of standardized tasks during a routine abdominal computed tomographic study. Participants evaluated the device for its efficacy as well as its possible advantages and disadvantages. The majority (69%) of those surveyed believed that the device could be useful in an interventional radiology practice and did not foresee problems with maintaining a sterile field. This proof-of-concept prototype and study demonstrate the potential utility of the motion-tracking device for enhancing imaging-guided treatment in the interventional radiology suite while maintaining a sterile field. Supplemental material available at http://radiographics.rsna.org/lookup/suppl/doi:10.1148/rg.332125101/-/DC1. <s> BIB004 </s> Touchless computer interfaces in hospitals: A review <s> use a combination of voice commands for discrete commands and mode changes and gestures for the control of continuous image parameters. <s> This paper presents a method to improve the navigation and manipulation of radiological images through a sterile hand gesture recognition interface based on attentional contextual cues. Computer vision algorithms were developed to extract intention and attention cues from the surgeon's behavior and combine them with sensory data from a commodity depth camera. The developed interface was tested in a usability experiment to assess the effectiveness of the new interface. An image navigation and manipulation task was performed, and the gesture recognition accuracy, false positives and task completion times were computed to evaluate system performance. Experimental results show that gesture interaction and surgeon behavior analysis can be used to accurately navigate, manipulate and access MRI images, and therefore this modality could replace the use of keyboard and mice-based interfaces. <s> BIB005 </s> Touchless computer interfaces in hospitals: A review <s> use a combination of voice commands for discrete commands and mode changes and gestures for the control of continuous image parameters. <s> There are several intra-operative use cases which require the surgeon to interact with medical devices. We used the Leap Motion Controller as input device and implemented two use-cases: 2D-Interaction (e.g. advancing EPR data) and selection of a value (e.g. room illumination brightness). The gesture detection was successful and we mapped its output to several devices and systems. <s> BIB006 </s> Touchless computer interfaces in hospitals: A review <s> use a combination of voice commands for discrete commands and mode changes and gestures for the control of continuous image parameters. <s> Real-time control of visual display systems via mid-air hand gestures offers many advantages over traditional interaction modalities. In medicine, for example, it allows a practitioner to adjust display values, e.g. contrast or zoom, on a medical visualization interface without the need to re-sterilize the interface. However, when users are holding a small tool (such as a pen, surgical needle, or computer stylus) the need to constantly put the tool down in order to make hand gesture interactions is not ideal. This work presents a novel interface that automatically adjusts for gesturing with hands and hand-held tools to precisely control medical displays. The novelty of our interface is that it uses a single set of gestures designed to be equally effective for fingers and hand-held tools without using markers. This type of interface was previously not feasible with low-resolution depth sensors such as Kinect, but is now achieved by using the recently released Leap Motion controller. Our interface is validated through a user study on a group of people given the task of adjusting parameters on a medical image. <s> BIB007 </s> Touchless computer interfaces in hospitals: A review <s> use a combination of voice commands for discrete commands and mode changes and gestures for the control of continuous image parameters. <s> We adapted and evaluated the Microsoft Kinect (touchless interface), Hillcrest Labs Loop Pointer (gyroscopic mouse), and the Apple iPad (multi-touch tablet) for intra-procedural imaging review efficacy in a simulation using MIM Software DICOM viewers. Using each device, 29 radiologists executed five basic interactions to complete the overall task of measuring an 8.1-cm hepatic lesion: scroll, window, zoom, pan, and measure. For each interaction, participants assessed the devices on a 3-point subjective scale (3 = highest usability score). The five individual scores were summed to calculate a subjective composite usability score (max 15 points). Overall task time to completion was recorded. Each user also assessed each device for its potential to jeopardize a sterile field. The composite usability scores were as follows: Kinect 9.9 (out of 15.0; SD = 2.8), Loop Pointer 12.9 (SD = 13.5), and iPad 13.5 (SD = 1.8). Mean task completion times were as follows: Kinect 156.7 s (SD = 86.5), Loop Pointer 51.5 s (SD = 30.6), and iPad 41.1 s (SD = 25.3). The mean hepatic lesion measurements were as follows: Kinect was 7.3 cm (SD = 0.9), Loop Pointer 7.8 cm (SD = 1.1), and iPad 8.2 cm (SD = 1.2). The mean deviations from true hepatic lesion measurement were as follows: Kinect 1.0 cm and for both the Loop Pointer and iPad, 0.9 cm (SD = 0.7). The Kinect had the least and iPad had the most subjective concern for compromising the sterile field. A new intra-operative imaging review interface may be near. Most surveyed foresee these devices as useful in procedures, and most do not anticipate problems with a sterile field. An ideal device would combine iPad’s usability and accuracy with the Kinect’s touchless aspect. <s> BIB008
Soutschek et al. BIB001 found that a majority of processing time was spent on acquiring images and preprocessing, for example, resizing. However, as computers have become more powerful, such processing is now easily accomplished allowing for more sophisticated real-time solutions. System performance and user familiarity impact directly on the user experience. Soutschek et al. BIB001 assert that users dislike systems when there is a perceptible delay during use. Unintended gestures and clutching. Not all gestures are intended as commands, for example, pointing out a feature of interest to a colleague, BIB006 and the misinterpretation of such gestures can adversely affect the user experience. The inclusion of a lock and unlock gesture (an example of a clutching mechanism) is essential according to Mauser and Burgert; BIB006 O'Hara et al.  used a voice command to lock and unlock the system. The aim is to ignore inadvertent commands: the system should be inactive until hailed by a distinctive action and should be locked using another distinctive action.  Deliberate gestures and unintentional movements need to be distinguished from each other. Unintentional movement usually occurs when the user interacts with other people or is resting their hands.  Rossol et al. BIB007 found unintentional finger -or tool -tip movement being interpreted as input to be a drawback with their system. In order to combat overlapping gestures, they suppressed any recognised gestures that overlapped the previous gesture's time window. BIB007 Tan et al. BIB004 state that fine movements were the most difficult. How to define starting and ending of a gesture BIB002 BIB006 is also a difficult issue. One advantage of depth cameras is the ability to take account of motion in the Z plane to reduce unwanted gestures when returning to idle. 16 Physical issues. At a practical level, gesture recognition works best at particular distances.  When designing gestures, O'Hara et al.  looked for ways to facilitate the work of clinicians, while maintaining sterile practices, by restricting movements to the spatial area in front of the torso. Tracking movements relative to the body may be the most appropriate, 9 specifically information from the operator's upper limbs and torso to implement the functionality of a mouse-like device.  Regarding the 3D gesture zone, 'this zone extends roughly from the waist inferiorly to the shoulders superiorly and from the chest to the limit of the outstretched arms anteriorly and to about 20 cm outside each shoulder laterally'.  Using environmental cues for intent, Jacob et al. BIB005 allowed users perform gestures anywhere in the field of view. However, depth segmentation has required an upper and lower threshold, BIB001 meaning that to use a system the user cannot be too close or too far away. Comfort and fatigue. Regarding the issue of interaction space, Wachs et al.  ask is it right to assume that the user and device are static and the user will be within a standard interaction envelope? Tan et al. BIB004 also say that ample space is required to operate their touchless system. Sufficient operational space is one of the factors affecting user comfort. This relates to the question of fatigue, and the need to avoid intense muscle tension over time. Consideration of static ('the effort required to maintain a posture for a fixed amount of time') and dynamic ('the effort required to move a hand through a trajectory') stress is key in promoting user comfort.  Training and calibration. Systems should be easy to integrate into existing ORs with minimal distraction, training or human resources according to Strickland et al.  Rosa and Elizondo 24 state that with a little training of the user, use of their gesture interface is easier and faster than changing sterile gloves or having an assistant outside the sterile environment. In O'Hara et al.,  the surgical team became familiar with the system through ongoing use of the system, 'learning on the job', rather than a dedicated training system, supported by prompts from the lead surgeon who acted as champion for the system. Chao et al. BIB008 determined that prior use of a device had a significant impact on task completion time, and found that gamers were faster on all devices. BIB008 However, Ebert et al. BIB003 