This paper explores the estimation of user attention in the setting of a cooperative handheld robot: a robot designed to behave as a handheld tool but that has levels of task knowledge. We use a tool-mounted gaze tracking system, which, after modelling via a pilot study, we use as a proxy for estimating the attention of the user. This information is then used for cooperation with users in a task of selecting and engaging with objects on a dynamic screen. Via a video game setup, we test various degrees of robot autonomy from fully autonomous, where the robot knows what it has to do and acts, to no autonomy where the user is in full control of the task. Our results measure performance and subjective metrics and show how the attention model benefits the interaction and preference of users.
Answering the question of how to best estimate a user's focus of attention in handheld robot task set-ups is the central aim of our research project. The literature of using eye and or head gaze for attention estimation is vast and spans many years. Recently, standard methods to detect gaze directions commonly involve eye trackers @cite_4 or determining head orientation @cite_17 @cite_1 . The information about both head orientation and eye gaze has been linked to a person's focus of attention in the past @cite_18 @cite_7 . found that there is a link between object interaction and preceding eye gaze during manual task execution @cite_12 @cite_10 . This is supported by who studied hand-eye coordination and found that there is a small share of saccades on objects which were used shortly after these @cite_15 . This observation was later linked to task planning by who suggest that visual attention is turned towards a subsequent goal right before interaction rather than relying on spatial memory only @cite_16 .
@cite_18 @cite_4 @cite_7 @cite_1 @cite_15 @cite_16 @cite_10 @cite_12 @cite_17
2129265898 1844401294 2121947860 2398402543
In this paper, the recognition of the visual focus of attention (VFOA) of meeting participants (as defined by their eye gaze direction) from their head pose is addressed. To this end, the head pose observations are modeled using an hidden Markov model (HMM) whose hidden states corresponds to the VFOA. The novelties are threefold. First, contrary to previous studies on the topic, in our set-up, the potential VFOA of a person is not restricted to other participants only, but includes environmental targets (a table and a projection screen), which increases the complexity of the task, with more VFOA targets spread in the pan and tilt (as well) gaze space. Second, the HMM parameters are set by exploiting results from the cognitive science on saccadic eye motion, which allows to predict what the head pose should be given an actual gaze target. Third, an unsupervised parameter adaptation step is proposed which accounts for the specific gazing behaviour of each participant. Using a publicly available corpus of 8 meetings featuring 4 persons, we analyze the above methods by evaluating, through objective performance measures, the recognition of the VFOA from head pose information obtained either using a magnetic sensor device or a vision based tracking system. We propose a method of predicting human egocentric visual attention using bottom-up visual saliency and egomotion information. Computational models of visual saliency are often employed to predict human attention; however, its mechanism and effectiveness have not been fully explored in egocentric vision. The purpose of our framework is to compute attention maps from an egocentric video that can be used to infer a person's visual attention. In addition to a standard visual saliency model, two kinds of attention maps are computed based on a camera's rotation velocity and direction of movement. These rotation-based and translation-based attention maps are aggregated with a bottom-up saliency map to enhance the accuracy with which the person's gaze positions can be predicted. The efficiency of the proposed framework was examined in real environments by using a head-mounted gaze tracker, and we found that the egomotion-based attention maps contributed to accurately predicting human visual attention. Previous studies of eye gaze have shown that when looking at images containing human faces, observers tend to rapidly focus on the facial regions. But is this true of other high-level image features as well? We here investigate the extent to which natural scenes containing faces, text elements, and cell phones - as a suitable control - attract attention by tracking the eye movements of subjects in two types of tasks - free viewing and search. We observed that subjects in free-viewing conditions look at faces and text 16.6 and 11.1 times more than similar regions normalized for size and position of the face and text. In terms of attracting gaze, text is almost as effective as faces. Furthermore, it is difﬁcult to avoid looking at faces and text even when doing so imposes a cost. We also found that subjects took longer in making their initial saccade when they were told to avoid faces text and their saccades landed on a non-face non-text object. We reﬁne a well-known bottom–up computer model of saliency-driven attention that includes conspicuity maps for color, orientation, and intensity by adding high-level semantic information (i.e., the location of faces or text) and demonstrate that this signiﬁcantly improves the ability to predict eye ﬁxations in natural images. Our enhanced model’s predictions yield an area under the ROC curve over 84 for images that contain faces or text when compared against the actual ﬁxation pattern of subjects. This suggests that the primate visual system allocates attention using such an enhanced saliency map. Abstract—Automated estimation of the allocation of adriver’s visual attention may be a critical component offuture Advanced Driver Assistance Systems. In theory,vision-based tracking of the eye can provide a good estimateof gaze location. In practice, eye tracking from videois challenging because of sunglasses, eyeglass reﬂections,lighting conditions, occlusions, motion blur, and otherfactors. Estimation of head pose, on the other hand, isrobust to many of these effects, but cannot provide as ﬁne-grained of a resolution in localizing the gaze. However,for the purpose of keeping the driver safe, it is sufﬁcientto partition gaze into regions. In this effort, we proposea system that extracts facial features and classiﬁes theirspatial conﬁguration into six regions in real-time. Ourproposed method achieves an average accuracy of 91.4 at an average decision rate of 11 Hz on a dataset of 50drivers from an on-road study.Index Terms—Head pose estimation, gaze tracking, driverdistraction, driver assistance systems, on-road study.