A Breadth-First Survey of Eye Tracking Applications <s> Selective Systems <s> In seeking hitherto-unused methods by which users and computers can communicate, we investigate the usefulness of eye movements as a fast and convenient auxiliary user-to-computer communication mode. The barrier to exploiting this medium has not been eye-tracking technology but the study of interaction techniques that incorporate eye movements into the user-computer dialogue in a natural and unobtrusive way. This paper discusses some of the human factors and technical considerations that arise in trying to use eye movements as an input medium, describes our approach and the first eye movement-based interaction techniques that we have devised and implemented in our laboratory, and reports our experiences and observations on them. <s> BIB001 </s> A Breadth-First Survey of Eye Tracking Applications <s> Selective Systems <s> Contents: Production Systems and the ACT-R Theory. Knowledge Representation. Performance. Learning. N. Kushmerick, C. Lebiere, Navigation and Conflict Resolution. N. Kushmerick, C. Lebiere, The Tower of Hanoi and Goal Structures. F.G. Conrad, A.T. Corbett, The LISP Tutor and Skill Acquisition. F.S. Bellezza, C.F. Boyle, The Geometry Tutor and Skill Acquisition. M.K. Singley, The Identical Elements Theory of Transfer. F.G. Conrad, A.T. Corbett, J.M. Fincham, D. Hoffman, Q. Wu, Computer Programming and Transfer. A.T. Corbett, Tutoring of Cognitive Skill. Creating Production-Rule Models. Reflections on the Theory. <s> BIB002 </s> A Breadth-First Survey of Eye Tracking Applications <s> Selective Systems <s> We describe an adaptive display algorithm for interactive frame rates during visualization of very complex virtual environments. The algorithm relies upon a hierarchical model representation in which objects are described at multiple levels of detail and can be drawn with various rendering algorithms. The idea behind the algorithm is to adjust image quality adaptively to maintain a uniform, user-specified target frame rate. We perform a constrained optimization to choose a level of detail and rendering algorithm for each potentially visible object in order to generate the “best” image possible within the target frame time. Tests show that the algorithm generates more uniform frame rates than other previously described detail elision algorithms with little noticeable difference in image quality during visualization of complex models. CR <s> BIB003 </s> A Breadth-First Survey of Eye Tracking Applications <s> Selective Systems <s> Abstract Eye movement-based analysis can enhance traditional performance, protocol, and walk-through evaluations of computer interfaces. Despite a substantial history of eye movement data collection in tasks, there is still a great need for an organized definition and evaluation of appropriate measures. Several measures based upon eye movement locations and scanpaths were evaluated here, to assess their validity for assessment of interface quality. Good and poor interfaces for a drawing tool selection program were developed by manipulating the grouping of tool icons. These were subsequently evaluated by a collection of 50 interface designers and typical users. Twelve subjects used the interfaces while their eye movements were collected. Compared with a randomly organized set of component buttons, well-organized functional grouping resulted in shorter scanpaths, covering smaller areas. The poorer interface resulted in more, but similar duration, fixations than the better interface. Whereas the poor interface produced less efficient search behavior, the layout of component representations did not influence their interpretability. Overall, data obtained from eye movements can significantly enhance the observation of users' strategies while using computer interfaces, which can subsequently improve the precision of computer interface evaluations. Relevance to industry The software development industry requires improved methods for the objective analysis and design of software interfaces. This study provides a foundation for using eye movement analysis as part of an objective evaluation tool for many phases of interface analysis. The present approach is instructional in its definition of eye movement-based measures, and is evaluative with respect to the utility of these measures. <s> BIB004 </s> A Breadth-First Survey of Eye Tracking Applications <s> Selective Systems <s> Click-down (or pull-down) menus have long been a key componentof graphical user interfaces, yet we know surprisingly little abouthow users actually interact with such menus. Nilsens [8] study onmenu selection has led to the development of a number of models ofhow users perform the task [6, 21. However, the validity of thesemodels has not been empirically assessed with respect to eyemovements (though [l] presents some interesting data that bear onthese models). The present study is an attempt to provide data thatcan help refine our understanding of how users interact with suchmenus. <s> BIB005 </s> A Breadth-First Survey of Eye Tracking Applications <s> Selective Systems <s> In this paper, we discuss why, in designing multiparty mediatedsystems, we should focus first on providing non-verbal cues whichare less redundantly coded in speech than those normally conveyedby video. We show how conveying one such cue, gaze direction, maysolve two problems in multiparty mediated communication andcollaboration: knowing who is talking to whom, and who is talkingabout what. As a candidate solution, we present the GAZE GroupwareSystem, which combines support for gaze awareness in multipartymediated communication and collaboration with small and linearbandwidth requirements. The system uses an advanced, desk- mountedeyetracker to metaphorically convey gaze awareness in a 3D virtualmeeting room and within shared documents. <s> BIB006 </s> A Breadth-First Survey of Eye Tracking Applications <s> Selective Systems <s> Eye movement-based interaction offers the potential of easy, natural, and fast ways of interacting in virtual environments. However, there is little empirical evidence about the advantages or disadvantages of this approach. We developed a new interaction technique for eye movement interaction in a virtual environment and compared it to more conventional 3-D pointing. We conducted an experiment to compare performance of the two interaction types and to assess their impacts on spatial memory of subjects and to explore subjects' satisfaction with the two types of interactions. We found that the eye movement-based interaction was faster than pointing, especially for distant objects. However, subjects' ability to recall spatial information was weaker in the eye condition than the pointing one. Subjects reported equal satisfaction with both types of interactions, despite the technology limitations of current eye tracking equipment. <s> BIB007 </s> A Breadth-First Survey of Eye Tracking Applications <s> Selective Systems <s> Eye gaze interaction can provide a convenient and natural addition to user-computer dialogues. We have previously reported on our interaction techniques using eye gaze [10]. While our techniques seemed useful in demonstration, we now investigate their strengths and weaknesses in a controlled setting. In this paper, we present two experiments that compare an interaction technique we developed for object selection based on a where a person is looking with the most commonly used selection method using a mouse. We find that our eye gaze interaction technique is faster than selection with a mouse. The results show that our algorithm, which makes use of knowledge about how the eyes behave, preserves the natural quickness of the eye. Eye gaze interaction is a reasonable addition to computer interaction and is convenient in situations where it is important to use the hands for other tasks. It is particularly beneficial for the larger screen workspaces and virtual environments of the future, and it will become increasingly practical as eye tracker technology matures. <s> BIB008
Interactive uses of eye trackers typically employ gaze as a pointing modality, e.g,. using gaze in a similar manner to a mouse pointer. Prominent applications involve selection of interface items (menus, buttons, etc.), as well as selection of objects or areas in Virtual Reality (VR). A prototypical application of gaze as an interactive modality is eye typing, particularly for handicapped users. Other uses involve gaze as an indirect pointing modality, for example as a deictic reference in the context of collaborative systems, or as an indirect pointing aid in user interfaces. Diagnostic uses of eye trackers are being adopted for usability studies, i.e., testing the effectiveness of interfaces as evidenced by where users look on the display. Eye-Based Interaction. One of the first eye-based interactive systems, introduced by BIB001 , demonstrated an intelligent gaze-based informational display. In Jacob's system a text window would scroll to show information on visually selected items. Jacob's paper was one of the first to use eye tracking technology interactively and is well known for its identification of an important problem in eye-based interactive systems: the Midas Touch problem. Essentially, if the eyes are used in a manner similar to a mouse, a difficulty arises in determining intended activation of foveated features (the eyes do not register button clicks!). To avoid the Midas Touch problem, Jacob discusses several possible solutions including blinks, finally promoting the use of dwell time to act as a selection mechanism. Another early eye-tracked interactive system was presented by  . This system provided the user with gaze-controlled navigation in a three-dimensional graphics world. The graphical environment contained story world characters who responded in interesting ways to the user's gaze. Fixations activated character "behaviors", since the system would maintain and increase the user's visual interest level of a fixated character. When fixated, characters would blush and/or provide a verbal narrative. Unlike Jacob's use of dwell time, in this system dwell time was used to zoom into the graphics world. Recently, BIB007 presented a new gaze-based interactive system, this time with gaze acting as a selective mechanism in VR. In this system Tanriverdi and Jacob compared eye-based interaction with hand-based interaction and found that performance with gaze selection was significantly faster than hand pointing, especially in environments where objects were placed far away from the user's location. In contrast, no performance difference was found in "close" environments. Furthermore, while pointing speed may increase with gaze selection, there appears to be a cognitive tradeoff for this gain in efficiency: subjects had more difficulty recalling locations they interacted with when using gaze-based selection than when using hand selection. The archetypical gaze-based pointing application is eye typing. Eye typing is and has been a useful communication modality for the severely handicapped. Most eye typing systems are implemented by presenting the user with a virtual keyboard, either on a typical computer monitor, or in some cases projected onto a wall. Based on analysis of tracked gaze, the system decides which letter the user is looking at and decides (e.g., by dwell time) whether or not to type this letter. The system may provide feedback to the user by either visual or auditory means, or a combination of both. Majaranta and Raiha (2002) provide an excellent survey of additional selection and feedback techniques employed in eye typing systems. Gaze-based communication systems such as those featuring eye typing offer certain (sometimes obvious) advantages but are also problematic. Gaze may provide an often faster pointing modality than a mouse or other pointing device, especially if the targets are sufficiently large. However, gaze location is not as precise as with a mouse since the fovea limits the accuracy of the measured point of regard. Another significant problem is accuracy of the eye tracker. Following initial calibration, eye tracker accuracy may exhibit significant drift, where the measured point of regard gradually falls off from the actual point of gaze. Together with the Midas Touch problem, drift remains a significant problem for gaze input. Zhai, Morimoto, and Ihde (1999) take another approach to gaze-based interaction and test the use of gaze as a sort of predictive pointing aid rather than a direct effector of selection. This is a particularly interesting and significant departure from "eye pointing" since this strategy is based on the assertion that loading of the visual perception channel with a motor control task seems fundamentally at odds with users' natural mental model in which the eye searches for, and takes in information, while coordinating with the hand for manipulation of external objects. In their paper on Manual Gaze Input Cascaded (MAGIC) Pointing, Zhai et al. present an acceleration technique where a (two-dimensional) cursor is warped to the vicinity of a fixated target. The acceleration is either immediately triggered by eye movement (liberal MAGIC pointing), or delayed until the mouse moves as well (conservative MAGIC pointing). Zhai et al. report that while the speed advantage is not obvious over manual (mouse) pointing, almost all users subjectively felt faster with either MAGIC pointing technique. Usability. Besides the use of gaze for interactive means, diagnostic eye tracking is gaining acceptance within the Human-Computer Interaction (HCI) community as another means to test usability of an interface. It is believed that eye movements can significantly enhance the observation of users' strategies while using computer interfaces BIB004 . Among various experiments, eye movements have been used to evaluate the grouping of tool icons, compare gaze-based and mouse interaction techniques BIB008 , evaluate the organization of click-down menus, and more recently to test the organization of web pages. BIB005 tested the arrangement of items during visual search of click-down menus contrasting two computational cognitive models designed to predict latency, accuracy, and ease of learning for a wide variety of HCI-related tasks: EPIC (Executive Process Interactive Control) (Kieras & Meyer, 1995) and ACT-R (Adaptive Control of Thought-Rational) (J. R. Anderson, personal communication, July 2002) . The EPIC architecture provides a general framework for simulating a human interacting with their environment to accomplish a task  . ACT-R is a framework for understanding human cognition whose basic claim is that cognitive skill is composed of production rules BIB002 . These models specifically predict, in different ways, the relationship between eye and mouse movement. A visual search experiment, conducted by Byrne et al., presented a target item prior to display of a click-down menu. Based on the number of fixations, both models were supported since in some cases visual search exhibited exclusively top-down search (ACT-R) and in others both top-down and random patterns were observed (EPIC). This particular study is informative for two reasons. First, it shows the importance of a good model which can be used for designing user interfaces. Lacking such a model can lead to a design based on inaccurate assumptions. Second, the paper points out that eye tracking is an effective usability modality and that a new model of visual search over menus is needed. In a usability study of web pages, BIB003 derive specific recommendations for a prototype web interface tool and discuss gaze-based evaluation of web pages in which the system permits free navigation across multiple pages. This is a significant advancement for usability studies of web browsers since prior to this study recording of gaze over multiple web pages has been difficult due to the synchronization of gaze with windows that scroll or hide from view. Collaborative Systems. Gaze can also be utilized to aid multiparty communication in collaborative systems. In BIB006 GAZE Groupware system, an eye tracker is used to convey gaze direction in a multiparty teleconferencing and document sharing system, providing a solution to two problems in multiparty mediated communication and collaboration: knowing who is talking to whom, and who is talking about what. The system displays 2D images of remotely located participants in a virtual world. These images rotate to depict gaze direction alleviating the problem of turntaking in multiparty communication systems. Furthermore, a gaze-directed "lightspot" is shown over a shared document indicating the users' fixated regions and thereby providing a deictic ("look at this") reference.