Touchless computer interfaces in hospitals: A review <s> Technologies <s> The use of doctor-computer interaction devices in the operation room (OR) requires new modalities that support medical imaging manipulation while allowing doctors' hands to remain sterile, supporting their focus of attention, and providing fast response times. This paper presents "Gestix," a vision-based hand gesture capture and recognition system that interprets in real-time the user's gestures for navigation and manipulation of images in an electronic medical record (EMR) database. Navigation and other gestures are translated to commands based on their temporal trajectories, through video capture. "Gestix" was tested during a brain biopsy procedure. In the in vivo experiment, this interface prevented the surgeon's focus shift and change of location while achieving a rapid intuitive reaction and easy interaction. Data from two usability tests provide insights and implications regarding human-computer interaction based on nonverbal conversational modalities. <s> BIB001 </s> Touchless computer interfaces in hospitals: A review <s> Technologies <s> A challenging goal today is the use of computer networking and advanced monitoring technologies to extend human intellectual capabilities in medical decision making. Modern commercial eye trackers are used in many of research fields, but the improvement of eye tracking technology, in terms of precision on the eye movements capture, has led to consider the eye tracker as a tool for vision analysis, so that its application in medical research, e.g. in ophthalmology, cognitive psychology and in neuroscience has grown considerably. The improvements of the human eye tracker interface become more and more important to allow medical doctors to increase their diagnosis capacity, especially if the interface allows them to remotely administer the clinical tests more appropriate for the problem at hand. In this paper, we propose a client/server eye tracking system that provides an interactive system for monitoring patients eye movements depending on the clinical test administered by the medical doctors. The system supports the retrieval of the gaze information and provides statistics to both medical research and disease diagnosis. <s> BIB002 </s> Touchless computer interfaces in hospitals: A review <s> Technologies <s> In this paper, an open-source system for a controller-free, highly interactive exploration of medical images is presented. By using a Microsoft Xbox Kinect™ as the only input device, the system's user interface allows users to interact at a distance through hand and arm gestures. The paper also details the interaction techniques we have designed specifically for the deviceless exploration of medical imaging data. Since the user interface is touch-free and does not require complex calibration steps, it is suitable for use in operating rooms, where non-sterilizable devices cannot be used. <s> BIB003 </s> Touchless computer interfaces in hospitals: A review <s> Technologies <s> Computerized medical systems play a vital role in the operating room, however, sterility requirements and interventional workflow often make interaction with these devices challenging for surgeons. Typical solutions, such as delegating physical control of keyboard and mouse to assistants, add an undesirable level of indirection. We present a touchless, gesture-based interaction framework for the operating room that lets surgeons define a personalized set of gestures for controlling arbitrary medical computerized systems. Instead of using cameras for capturing gestures, we rely on a few wireless inertial sensors, placed on the arms of the surgeon, eliminating the dependence on illumination and line-of-sight. A discriminative gesture recognition approach based on kernel regression allows us to simultaneously classify performed gestures and to track the relative spatial pose within each gesture, giving surgeons fine-grained control of continuous parameters. An extensible software architecture enables a dynamic association of learned gestures to arbitrary intraoperative computerized systems. Our experiments illustrate the performance of our approach and encourage its practical applicability. <s> BIB004 </s> Touchless computer interfaces in hospitals: A review <s> Technologies <s> Purpose ::: In the operating room (OR) a touchless interface is an ideal solution since it does not demand any physical contact and still can provide the necessary control features in a cleansed and sterilized environment. <s> BIB005 </s> Touchless computer interfaces in hospitals: A review <s> Technologies <s> Keyboards, mice, and touch screens are a potential source of infection or contamination in operating rooms, intensive care units, and autopsy suites. The authors present a low-cost prototype of a system, which allows for touch-free control of a medical image viewer. This touch-free navigation system consists of a computer system (IMac, OS X 10.6 Apple, USA) with a medical image viewer (OsiriX, OsiriX foundation, Switzerland) and a depth camera (Kinect, Microsoft, USA). They implemented software that translates the data delivered by the camera and a voice recognition software into keyboard and mouse commands, which are then passed to OsiriX. In this feasibility study, the authors introduced 10 medical professionals to the system and asked them to re-create 12 images from a CT data set. They evaluated response times and usability of the system compared with standard mouse/keyboard control. Users felt comfortable with the system after approximately 10 minutes. Response time was 120 ms. Users required 1.4 times more time to re-create an image with gesture control. Users with OsiriX experience were significantly faster using the mouse/keyboard and faster than users without prior experience. They rated the system 3.4 out of 5 for ease of use in comparison to the mouse/keyboard. The touch-free, gesture-controlled system performs favorably and removes a potential vector for infection, protecting both patients and staff. Because the camera can be quickly and easily integrated into existing systems, requires no calibration, and is low cost, the barriers to using this technology are low. <s> BIB006 </s> Touchless computer interfaces in hospitals: A review <s> Technologies <s> Sterility restrictions in surgical settings make touch-less interaction an interesting solution for surgeons to interact directly with digital images. The HCI community has already explored several methods for touch-less interaction including those based on camera-based gesture tracking and voice control. In this paper, we present a system for gesture-based interaction with medical images based on a single wristband sensor and capacitive floor sensors, allowing for hand and foot gesture input. The first limited evaluation of the system showed an acceptable level of accuracy for 12 different hand & foot gestures; also users found that our combined hand and foot based gestures are intuitive for providing input. <s> BIB007 </s> Touchless computer interfaces in hospitals: A review <s> Technologies <s> Review of prior and real-time patient images is critical during an interventional radiology procedure; however, it often poses the challenge of efficiently reviewing images while maintaining a sterile field. Although interventional radiologists can "scrub out" of the procedure, use sterile console covers, or verbally relay directions to an assistant, the ability of the interventionalist to directly control the images without having to touch the console could offer potential gains in terms of sterility, procedure efficiency, and radiation reduction. The authors investigated a potential solution with a low-cost, touch-free motion-tracking device that was originally designed as a video game controller. The device tracks a person's skeletal frame and its motions, a capacity that was adapted to allow manipulation of medical images by means of hand gestures. A custom software program called the Touchless Radiology Imaging Control System translates motion information obtained with the motion-tracking device into commands to review images on a workstation. To evaluate this system, 29 radiologists at the authors' institution were asked to perform a set of standardized tasks during a routine abdominal computed tomographic study. Participants evaluated the device for its efficacy as well as its possible advantages and disadvantages. The majority (69%) of those surveyed believed that the device could be useful in an interventional radiology practice and did not foresee problems with maintaining a sterile field. This proof-of-concept prototype and study demonstrate the potential utility of the motion-tracking device for enhancing imaging-guided treatment in the interventional radiology suite while maintaining a sterile field. Supplemental material available at http://radiographics.rsna.org/lookup/suppl/doi:10.1148/rg.332125101/-/DC1. <s> BIB008 </s> Touchless computer interfaces in hospitals: A review <s> Technologies <s> During the last few years, we have been witnessing a widespread adoption of touchless technologies in the context of surgical procedures. Touchless interfaces are advantageous in that they can preserve sterility around the patient, allowing surgeons to visualize medical images without having to physically touch any control or to rely on a proxy. Such interfaces have been tailored to interact with 2D medical images but not with 3D reconstructions of anatomical data, since such an interaction requires at least three degrees of freedom. In this paper, we discuss the results of a user study in which a mouse-based interface has been compared with two Kinect-based touchless interfaces which allow users to interact with 3D data with up to nine degrees of freedom. The experimental results show that there is a significant relation between the number of degrees of freedom simultaneously controlled by the user and the number of degrees of freedom required to perform, in a touchless way, an accurate manipulation task. <s> BIB009 </s> Touchless computer interfaces in hospitals: A review <s> Technologies <s> Background ::: Current surgical robots are controlled by a mechanical master located away from the patient, tracking surgeon’s hands by wire and pulleys or mechanical linkage. Contactless hand tracking for surgical robot control is an attractive alternative, because it can be executed with minimal footprint at the patient’s bedside without impairing sterility, while eliminating current disassociation between surgeon and patient. We compared technical and technologic feasibility of contactless hand tracking to the current clinical standard master controllers. <s> BIB010 </s> Touchless computer interfaces in hospitals: A review <s> Technologies <s> We adapted and evaluated the Microsoft Kinect (touchless interface), Hillcrest Labs Loop Pointer (gyroscopic mouse), and the Apple iPad (multi-touch tablet) for intra-procedural imaging review efficacy in a simulation using MIM Software DICOM viewers. Using each device, 29 radiologists executed five basic interactions to complete the overall task of measuring an 8.1-cm hepatic lesion: scroll, window, zoom, pan, and measure. For each interaction, participants assessed the devices on a 3-point subjective scale (3 = highest usability score). The five individual scores were summed to calculate a subjective composite usability score (max 15 points). Overall task time to completion was recorded. Each user also assessed each device for its potential to jeopardize a sterile field. The composite usability scores were as follows: Kinect 9.9 (out of 15.0; SD = 2.8), Loop Pointer 12.9 (SD = 13.5), and iPad 13.5 (SD = 1.8). Mean task completion times were as follows: Kinect 156.7 s (SD = 86.5), Loop Pointer 51.5 s (SD = 30.6), and iPad 41.1 s (SD = 25.3). The mean hepatic lesion measurements were as follows: Kinect was 7.3 cm (SD = 0.9), Loop Pointer 7.8 cm (SD = 1.1), and iPad 8.2 cm (SD = 1.2). The mean deviations from true hepatic lesion measurement were as follows: Kinect 1.0 cm and for both the Loop Pointer and iPad, 0.9 cm (SD = 0.7). The Kinect had the least and iPad had the most subjective concern for compromising the sterile field. A new intra-operative imaging review interface may be near. Most surveyed foresee these devices as useful in procedures, and most do not anticipate problems with a sterile field. An ideal device would combine iPad’s usability and accuracy with the Kinect’s touchless aspect. <s> BIB011 </s> Touchless computer interfaces in hospitals: A review <s> Technologies <s> Purpose: The purposes of this study were to develop a workstation computer that allowed intraoperative touchless control of diagnostic and surgical images by a surgeon, and to report the preliminary experience with the use of the system in a series of cases in which dental surgery was performed. Materials and Methods: A custom workstation with a new motion sensing input device (Leap Motion) was set up in order to use a natural user interface (NUI) to manipulate the imaging software by hand gestures. The system allowed intraoperative touchless control of the surgical images. Results: For the first time in the literature, an NUI system was used for a pilot study during 11 dental surgery procedures including tooth extractions, dental implant placements, and guided bone regeneration. No complications were reported. The system performed very well and was very useful. Conclusion: The proposed system fulfilled the objective of providing touchless access and control of the system of images and a three-dimensional surgical plan, thus allowing the maintenance of sterile conditions. The interaction between surgical staff, under sterile conditions, and computer equipment has been a key issue. The solution with an NUI with touchless control of the images seems to be closer to an ideal. The cost of the sensor system is quite low; this could facilitate its incorporation into the practice of routine dental surgery. This technology has enormous potential in dental surgery and other healthcare specialties. (Imaging Sci Dent 2014; 44: 155-60) <s> BIB012 </s> Touchless computer interfaces in hospitals: A review <s> Technologies <s> There are several intra-operative use cases which require the surgeon to interact with medical devices. We used the Leap Motion Controller as input device and implemented two use-cases: 2D-Interaction (e.g. advancing EPR data) and selection of a value (e.g. room illumination brightness). The gesture detection was successful and we mapped its output to several devices and systems. <s> BIB013 </s> Touchless computer interfaces in hospitals: A review <s> Technologies <s> Real-time control of visual display systems via mid-air hand gestures offers many advantages over traditional interaction modalities. In medicine, for example, it allows a practitioner to adjust display values, e.g. contrast or zoom, on a medical visualization interface without the need to re-sterilize the interface. However, when users are holding a small tool (such as a pen, surgical needle, or computer stylus) the need to constantly put the tool down in order to make hand gesture interactions is not ideal. This work presents a novel interface that automatically adjusts for gesturing with hands and hand-held tools to precisely control medical displays. The novelty of our interface is that it uses a single set of gestures designed to be equally effective for fingers and hand-held tools without using markers. This type of interface was previously not feasible with low-resolution depth sensors such as Kinect, but is now achieved by using the recently released Leap Motion controller. Our interface is validated through a user study on a group of people given the task of adjusting parameters on a medical image. <s> BIB014 </s> Touchless computer interfaces in hospitals: A review <s> Technologies <s> Interaction with mobile consumer devices leads to a higher acceptance and affinity of persons to natural user interfaces and perceptional interaction possibilities. New interaction modalities become accessible and are capable to improve human machine interaction even in complex and high risk environments, like the operation room. Here, manifold medical disciplines cause a great variety of procedures and thus staff and equipment. One universal challenge is to meet the sterility requirements, for which common contact-afflicted remote interfaces always pose a potential risk causing a hazard for the process. The proposed operating table control system overcomes this process risk and thus improves the system usability significantly. The 3D sensor system, the Microsoft Kinect, captures the motion of the user, allowing a touchless manipulation of an operating table. Three gestures enable the user to select, activate and manipulate all segments of the motorised system in a safe and intuitive way. The gesture dynamics are synchronised with the table movement. In a usability study, 15 participants evaluated the system with a system usability score by Broke of 79. This states a high potential for implementation and acceptance in interventional environments. In the near future, even processes with higher risks could be controlled with the proposed interface, while interfaces become safer and more direct. <s> BIB015 </s> Touchless computer interfaces in hospitals: A review <s> Technologies <s> When patients visit hospitals, they tend to develop negative feelings such as anxiety and lack of comfort. This paper uses the techniques of Interaction Design and User-Centered Design with the main objective to ask users about their needs and requirements to develop an interactive hospital room with a natural interaction device like Kinect, so we could achieve a reduction on those negative feelings described previously. This will allow that people could use their voices and a set of body gestures to control basic devices that could be find in a hospital room, making the hospitalization process a different, more comfortable and with more autonomy. Prototypes were developed and evaluated into two stages by people that could use the application potentially. The results show that people agree to use this form of interaction in terms of comfort, performance and utility. <s> BIB016 </s> Touchless computer interfaces in hospitals: A review <s> Technologies <s> Introduction ::: Training surgeons in minimally invasive surgery (MIS) requires surgical residents to operate under the direction of a consultant. The inability of the instructing surgeon to point at the laparoscopic monitor without releasing the instruments remains a barrier to effective instruction. The wireless hands-free surgical pointer (WHaSP) has been developed to aid instruction during MIS. <s> BIB017
Available technologies have improved markedly over time. At the time of publication of the study by Gallo et al., BIB003 there existed no reliable and mature technology for effective gesture control. Over time, researchers have investigated using eye gaze technology (EGT), BIB002 capacitive floor sensors BIB007 and inertial orientation sensors; BIB004 BIB007 colour cameras such as the Canon VC-C4, BIB001 the Loop Pointer BIB011 and MESA SR-31000 ToF cameras; 13 Siemens integrated OR system; 38 wireless hands-free surgical pointer; BIB017 to the Apple iPad; 17 leap motion controllers; BIB012 BIB013 BIB014 and the Microsoft Kinect ToF camera G1. BIB008 BIB009 BIB010 BIB003 BIB011 BIB005 BIB015 BIB016 Data frequency has ranged from devices with 15-fps output, up to the leap motion controller with greater than 100-fps output. Regarding camera configurations, Chao et al. BIB011 suggest that moving to a stereo camera set-up might improve accuracy for touchless interaction. Along with advances in the hardware, there have also been complementary developments in available software for implementation of touchless systems. Many of the papers that utilised the Microsoft Kinect used additional specialised software, such as Primesense drivers, BIB011 OpenNI software 7,17 and skeletal tracking NITE. BIB011 Across the papers, there was a mix of purely gestural commands, and combined gesture and voice commands (a form of multimodal interaction). The primary modes of human communication are speech, hand and body gestures; facial expressions; and eye gaze, BIB001 and much of the HCI literature on these forms of interaction draws motivation from this 'natural' nature rather than the touchless properties of these interaction modalities: 'Gestures are useful for computer interaction since they are the most primary and expressive form of human communication'.  Only one paper by Chao et al. directly compared the efficacy of multiple devices. In their paper, the authors compared the Microsoft Kinect, Hillcrest Labs Loop Pointer, and the Apple iPad. Theirs was the only paper to use either the Hillcrest Labs Loop Pointer or the Apple iPad. Their Users reported that they accessed more medical images than normal using the system. Moving, zooming, windowing medical images BIB006 Medical professionals