Touchless computer interfaces in hospitals: A review <s> Non-functional requirements <s> Whether attempting to review digital radiologic images during a procedure or reviewing labs on a clinical ward, computer keyboards and mice are potential sources for contamination of clinicians during sterile and non-sterile activities related to clinical care. The authors describe and demonstrate a live system prototype for hands-free, gesture-based control of an electronic medical record (EMR) system. <s> BIB001 </s> Touchless computer interfaces in hospitals: A review <s> Non-functional requirements <s> The use of doctor-computer interaction devices in the operation room (OR) requires new modalities that support medical imaging manipulation while allowing doctors' hands to remain sterile, supporting their focus of attention, and providing fast response times. This paper presents "Gestix," a vision-based hand gesture capture and recognition system that interprets in real-time the user's gestures for navigation and manipulation of images in an electronic medical record (EMR) database. Navigation and other gestures are translated to commands based on their temporal trajectories, through video capture. "Gestix" was tested during a brain biopsy procedure. In the in vivo experiment, this interface prevented the surgeon's focus shift and change of location while achieving a rapid intuitive reaction and easy interaction. Data from two usability tests provide insights and implications regarding human-computer interaction based on nonverbal conversational modalities. <s> BIB002 </s> Touchless computer interfaces in hospitals: A review <s> Non-functional requirements <s> For a lot of applications, and particularly for medical intra-operative applications, the exploration of and navigation through 3-D image data provided by sensors like ToF (time-of-flight) cameras, MUSTOF (multisensor-time-of-flight) endoscopes or CT (computed tomography) [8], requires a user-interface which avoids physical interaction with an input device. Thus, we process a touchless user-interface based on gestures classified by the data provided by a ToF camera. Reasonable and necessary user interactions are described. For those interactions a suitable set of gestures is introduced. A user-interface is then proposed, which interprets the current gesture and performs the assigned functionality. For evaluating the quality of the developed user-interface we considered the aspects of classification rate, real-time applicability, usability, intuitiveness and training time. The results of our evaluation show that our system, which provides a classification rate of 94.3% at a framerate of 11 frames per second, satisfactorily addresses all these quality requirements. <s> BIB003 </s> Touchless computer interfaces in hospitals: A review <s> Non-functional requirements <s> In this paper, an open-source system for a controller-free, highly interactive exploration of medical images is presented. By using a Microsoft Xbox Kinect™ as the only input device, the system's user interface allows users to interact at a distance through hand and arm gestures. The paper also details the interaction techniques we have designed specifically for the deviceless exploration of medical imaging data. Since the user interface is touch-free and does not require complex calibration steps, it is suitable for use in operating rooms, where non-sterilizable devices cannot be used. <s> BIB004 </s> Touchless computer interfaces in hospitals: A review <s> Non-functional requirements <s> Keyboards, mice, and touch screens are a potential source of infection or contamination in operating rooms, intensive care units, and autopsy suites. The authors present a low-cost prototype of a system, which allows for touch-free control of a medical image viewer. This touch-free navigation system consists of a computer system (IMac, OS X 10.6 Apple, USA) with a medical image viewer (OsiriX, OsiriX foundation, Switzerland) and a depth camera (Kinect, Microsoft, USA). They implemented software that translates the data delivered by the camera and a voice recognition software into keyboard and mouse commands, which are then passed to OsiriX. In this feasibility study, the authors introduced 10 medical professionals to the system and asked them to re-create 12 images from a CT data set. They evaluated response times and usability of the system compared with standard mouse/keyboard control. Users felt comfortable with the system after approximately 10 minutes. Response time was 120 ms. Users required 1.4 times more time to re-create an image with gesture control. Users with OsiriX experience were significantly faster using the mouse/keyboard and faster than users without prior experience. They rated the system 3.4 out of 5 for ease of use in comparison to the mouse/keyboard. The touch-free, gesture-controlled system performs favorably and removes a potential vector for infection, protecting both patients and staff. Because the camera can be quickly and easily integrated into existing systems, requires no calibration, and is low cost, the barriers to using this technology are low. <s> BIB005 </s> Touchless computer interfaces in hospitals: A review <s> Non-functional requirements <s> Review of prior and real-time patient images is critical during an interventional radiology procedure; however, it often poses the challenge of efficiently reviewing images while maintaining a sterile field. Although interventional radiologists can "scrub out" of the procedure, use sterile console covers, or verbally relay directions to an assistant, the ability of the interventionalist to directly control the images without having to touch the console could offer potential gains in terms of sterility, procedure efficiency, and radiation reduction. The authors investigated a potential solution with a low-cost, touch-free motion-tracking device that was originally designed as a video game controller. The device tracks a person's skeletal frame and its motions, a capacity that was adapted to allow manipulation of medical images by means of hand gestures. A custom software program called the Touchless Radiology Imaging Control System translates motion information obtained with the motion-tracking device into commands to review images on a workstation. To evaluate this system, 29 radiologists at the authors' institution were asked to perform a set of standardized tasks during a routine abdominal computed tomographic study. Participants evaluated the device for its efficacy as well as its possible advantages and disadvantages. The majority (69%) of those surveyed believed that the device could be useful in an interventional radiology practice and did not foresee problems with maintaining a sterile field. This proof-of-concept prototype and study demonstrate the potential utility of the motion-tracking device for enhancing imaging-guided treatment in the interventional radiology suite while maintaining a sterile field. Supplemental material available at http://radiographics.rsna.org/lookup/suppl/doi:10.1148/rg.332125101/-/DC1. <s> BIB006 </s> Touchless computer interfaces in hospitals: A review <s> Non-functional requirements <s> This paper presents a method to improve the navigation and manipulation of radiological images through a sterile hand gesture recognition interface based on attentional contextual cues. Computer vision algorithms were developed to extract intention and attention cues from the surgeon's behavior and combine them with sensory data from a commodity depth camera. The developed interface was tested in a usability experiment to assess the effectiveness of the new interface. An image navigation and manipulation task was performed, and the gesture recognition accuracy, false positives and task completion times were computed to evaluate system performance. Experimental results show that gesture interaction and surgeon behavior analysis can be used to accurately navigate, manipulate and access MRI images, and therefore this modality could replace the use of keyboard and mice-based interfaces. <s> BIB007 </s> Touchless computer interfaces in hospitals: A review <s> Non-functional requirements <s> Interaction with mobile consumer devices leads to a higher acceptance and affinity of persons to natural user interfaces and perceptional interaction possibilities. New interaction modalities become accessible and are capable to improve human machine interaction even in complex and high risk environments, like the operation room. Here, manifold medical disciplines cause a great variety of procedures and thus staff and equipment. One universal challenge is to meet the sterility requirements, for which common contact-afflicted remote interfaces always pose a potential risk causing a hazard for the process. The proposed operating table control system overcomes this process risk and thus improves the system usability significantly. The 3D sensor system, the Microsoft Kinect, captures the motion of the user, allowing a touchless manipulation of an operating table. Three gestures enable the user to select, activate and manipulate all segments of the motorised system in a safe and intuitive way. The gesture dynamics are synchronised with the table movement. In a usability study, 15 participants evaluated the system with a system usability score by Broke of 79. This states a high potential for implementation and acceptance in interventional environments. In the near future, even processes with higher risks could be controlled with the proposed interface, while interfaces become safer and more direct. <s> BIB008 </s> Touchless computer interfaces in hospitals: A review <s> Non-functional requirements <s> We adapted and evaluated the Microsoft Kinect (touchless interface), Hillcrest Labs Loop Pointer (gyroscopic mouse), and the Apple iPad (multi-touch tablet) for intra-procedural imaging review efficacy in a simulation using MIM Software DICOM viewers. Using each device, 29 radiologists executed five basic interactions to complete the overall task of measuring an 8.1-cm hepatic lesion: scroll, window, zoom, pan, and measure. For each interaction, participants assessed the devices on a 3-point subjective scale (3 = highest usability score). The five individual scores were summed to calculate a subjective composite usability score (max 15 points). Overall task time to completion was recorded. Each user also assessed each device for its potential to jeopardize a sterile field. The composite usability scores were as follows: Kinect 9.9 (out of 15.0; SD = 2.8), Loop Pointer 12.9 (SD = 13.5), and iPad 13.5 (SD = 1.8). Mean task completion times were as follows: Kinect 156.7 s (SD = 86.5), Loop Pointer 51.5 s (SD = 30.6), and iPad 41.1 s (SD = 25.3). The mean hepatic lesion measurements were as follows: Kinect was 7.3 cm (SD = 0.9), Loop Pointer 7.8 cm (SD = 1.1), and iPad 8.2 cm (SD = 1.2). The mean deviations from true hepatic lesion measurement were as follows: Kinect 1.0 cm and for both the Loop Pointer and iPad, 0.9 cm (SD = 0.7). The Kinect had the least and iPad had the most subjective concern for compromising the sterile field. A new intra-operative imaging review interface may be near. Most surveyed foresee these devices as useful in procedures, and most do not anticipate problems with a sterile field. An ideal device would combine iPad’s usability and accuracy with the Kinect’s touchless aspect. <s> BIB009
The most commonly referenced non-functional requirements in the corpus are usability and reliability of gesture and voice control. System design should consider real-time interaction, sterility, fatigue, intuitiveness/naturalness, robustness, ease of learning, unencumbered use and the scope for unintentional commands. BIB002 Another requirement is low cognitive load, with short, simple and natural gestures.  Natural interaction is taken to include the use of voice and gesture commands.  Furthermore, it is suggested that systems should focus on being stable and providing basic access rather than trying to be more powerful and versatile.  The system needs to support both coarse and fine-grained system control through careful design of the gesture vocabulary,  and how gestures are mapped to control elements in the interface.  Reliability is identified as a key non-functional requirement,  which is impacted by the issue of unintentional gestures, and gesture control interfaces may need to sense the human body position, configuration and movement in order to achieve this. BIB004 Evaluations A variety of outcomes are studied in the literature ( Table 3) with accuracy of gesture recognition being the most frequently reported outcome (seven papers). There are a number of factors that should be considered when evaluating a system.  Validation of sensitivity and recall of gestures, precision and positive predictive value, f-measure, likelihood ratio and recognition accuracy should all be rigorously evaluated using standard, public data sets.  Furthermore, usability criteria such as task completion time and subjective workload assessment and user independence should be evaluated. The usability of interfaces is described by different standards, which focus on efficiency, effectiveness and user satisfaction. BIB008 Soutschek et al. BIB003 deem aspects such as classification rate, real-time applicability, usability, intuitiveness and training time as relevant aspects for evaluation. Both quantitative and qualitative assessments of a system should also be carried out comparing gesture interaction to other technologies such as voice control and mouse and keyboard.  Subjective evaluation by experienced physicians is important and is likely more insightful than technical factor comparisons. BIB009 This obviously introduces an obstacle in terms of access to potentially large numbers of qualified personnel, as well as ethical and safety concerns for eventual real-world evaluations. However, a number of the papers did perform evaluations with representative end users, for example, Tan et al. BIB006 asked 29 radiologists to evaluate their system for efficacy as well as possible advantages and disadvantages. Technical evaluations. In Jacob et al., BIB007 development and validation involved three steps, lexicon generation, development of gesture-recognition software and validation of the technology. Papers have evaluated their systems on both technical and subjective (user experience) levels. BIB003 When performing a technical evaluation, data regarding technical accuracy need to be collected and analysed. BIB003 Technical evaluation might focus purely on accuracy of gesture recognition at the early stages of development, or on task performance at later stages, in which performance time can also be measured. The choice of realistic tasks is important. High accuracy is a requirement for medical implementation, with an accuracy of 95 per cent upwards suggested for use in a medical context. BIB001 Feasibility. With the advances in technology using touchless technology, the focus is no longer on technical feasibility. Rather, it is important that we understand how systems and their design impact on the patterns of behaviour of hospital staff. 14 However, it is claimed that much existing work from a medical background lacks consideration of practical elements and implementations, remaining experimental and work originating from a technology background often suffers from oversimplification of medical complexity.  Acceptability and satisfaction. A range of methods have been used for qualitative evaluations including contextual interviews, individual questionnaires and subjective satisfaction questionnaires. Wachs et al. BIB002 and Ebert et al. BIB005 made use of subjective questionnaires to determine user satisfaction with the system. Questionnaires may be used to gauge issues such as previous task experience, perceived ease of task performance, task completion time and overall task satisfaction. BIB002 Robustness is key to the acceptability of a system; Wachs et al.  specifically mention robustness for camera sensor and lens characteristics, scene and background details, lighting conditions and user differences. Overall, there is a consensus that systems should be subject to both technical and qualitative evaluations using public data sets and demonstrate a very high level of gesture-recognition success in order to be appropriate for medical use. BIB001 There is also a recognised need to minimise unwanted side effects such as accidental gesture recognition.