What do we know about volumetric medical image interpretation?: a review of the basic science and medical image perception literatures <s> Concluding remarks and future directions <s> Objective:Eye tracking in three dimensions is novel, but established descriptors derived from two-dimensional (2D) studies are not transferable. We aimed to develop metrics suitable for statistical comparison of eye-tracking data obtained from readers of three-dimensional (3D) “virtual” medical imaging, using CT colonography (CTC) as a typical example.Methods:Ten experienced radiologists were eye tracked while observing eight 3D endoluminal CTC videos. Subsequently, we developed metrics that described their visual search patterns based on concepts derived from 2D gaze studies. Statistical methods were developed to allow analysis of the metrics.Results:Eye tracking was possible for all readers. Visual dwell on the moving region of interest (ROI) was defined as pursuit of the moving object across multiple frames. Using this concept of pursuit, five categories of metrics were defined that allowed characterization of reader gaze behaviour. These were time to first pursuit, identification and assessment time, ... <s> BIB001 </s> What do we know about volumetric medical image interpretation?: a review of the basic science and medical image perception literatures <s> Concluding remarks and future directions <s> OBJECTIVE ::: We aimed to identify the effect of computer-aided detection (CAD) on visual search and performance in CT Colonography (CTC) of inexperienced and experienced readers. ::: ::: ::: METHODS ::: Fifteen endoluminal CTC examinations were recorded, each with one polyp, and two videos were generated, one with and one without a CAD mark. Forty-two readers (17 experienced, 25 inexperienced) interpreted the videos during infrared visual search recording. CAD markers and polyps were treated as regions of interest in data processing. This multi-reader, multi-case study was analysed using multilevel modelling. ::: ::: ::: RESULTS ::: CAD drew readers' attention to polyps faster, accelerating identification times: median 'time to first pursuit' was 0.48 s (IQR 0.27 to 0.87 s) with CAD, versus 0.58 s (IQR 0.35 to 1.06 s) without. For inexperienced readers, CAD also held visual attention for longer. All visual search metrics used to assess visual gaze behaviour demonstrated statistically significant differences when "with" and "without" CAD were compared. A significant increase in the number of correct polyp identifications across all readers was seen with CAD (74 % without CAD, 87 % with CAD; p < 0.001). ::: ::: ::: CONCLUSIONS ::: CAD significantly alters visual search and polyp identification in readers viewing three-dimensional endoluminal CTC. For polyp and CAD marker pursuit times, CAD generally exerted a larger effect on inexperienced readers. ::: ::: ::: KEY POINTS ::: • Visual gaze is attracted by computer-assisted detection (CAD) marks on polyps • Inexperienced readers' gaze is affected more by CAD than experienced readers. • CAD marks could mean that the unannotated endoluminal surface is relatively neglected. • Correct polyp identification is increased significantly by CAD. <s> BIB002 </s> What do we know about volumetric medical image interpretation?: a review of the basic science and medical image perception literatures <s> Concluding remarks and future directions <s> Even experts routinely miss infrequent targets, such as weapons in baggage scans or tumors in mammograms, because the visual system is not equipped to notice the unusual. To date, limited progress ... <s> BIB003
This review of the literature highlights the many contributions made by researchers toward better understanding volumetric image interpretation. However, due to the contemporary nature of much of this research, much of our knowledge is driven by the data rather than grounded in theory. Although exploratory analyses often lead to important predictions for future research, there is a limit to what can be learned from simply characterizing search behavior. Much like research on 2D medical image interpretation, this approach has revealed substantial variability between observers and experience levels in volumetric image search. However, contrary to the 2D medical image perception literature, models of expertise have not yet been well-established for volumetric image interpretation. In fact, it is unknown how even some of the most ubiquitous findings from the literature, such as increased global processing ability with expertise, apply to volumetric image search. In addition, relatively few expertise studies have been conducted using volumetric images while allowing the observer to freely scroll through depth, which leaves substantial unanswered questions about how scrolling behavior might relate to task performance or develop with experience. Given the increasing popularity of volumetric imaging and the recent Food and Drug Administration (FDA) approval of both breast tomosynthesis and digital pathology, this represents a significant opportunity for researchers interested in helping clinicians understand how to best examine these complex images. The challenge for the field going forward will be to transition from describing search behavior in volumetric images to establishing models of expertise with testable predictions. Ultimately, these models should be able to account for the stimulus, the task, and the observer. Fortunately, researchers are well-situated to make this transition. Almost 50 years of research into medical image perception can be leveraged to make predictions about expertise in volumetric images. This research has led to models of expertise that are associated with a number of well-established eye-tracking metrics, and research has demonstrated it is feasible to adapt these measures to volumetric images (e.g., BIB001 BIB002 . Furthermore, there are many examples in the literature that highlight the promise of using basic science to make better predictions about medical image perception (e.g., BIB003 , and there is substantial untapped potential for using this approach in volumetric imaging as well. A basic science approach may help the field transition from a more descriptive, computational analysis of what radiologists do to a better understanding of how the radiologists perform their task, which represents another level of analysis in our understanding of complex systems  . As outlined in this review, there are many relevant findings from basic science that are untested in volumetric medical images. Some of the most exciting avenues for future research may be determining the limitations of memory in volumetric image search, the features that capture attention in volumetric images (e.g., motion onset cues), or the interactions between motor and perceptual processes when scrolling through depth. By grounding future research firmly in the literature on basic science and medical image perception, the field is poised to make substantial progress in our understanding of volumetric image search in the coming years. 