Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> A new hypothesis about the role of focused attention is proposed. The feature-integration theory of attention suggests that attention must be directed serially to each stimulus in a display whenever conjunctions of more than one separable feature are needed to characterize or distinguish the possible objects presented. A number of predictions were tested in a variety of paradigms including visual search, texture segregation, identification and localization, and using both separable dimensions (shape and color) and local elements or parts of figures (lines, curves, etc. in letters) as the features to be integrated into complex wholes. The results were in general consistent with the hypothesis. They offer a new set of criteria for distinguishing separable from integral features and a new rationale for predicting which tasks will show attention limits and which will not. <s> BIB001 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> Early stages of visual processing may exploit the characteristic structure of natural visual stimuli. This structure may differ from the intrinsic structure of natural scenes, because sampling of the environment is an active process. For example, humans move their eyes several times a second when looking at a scene. The portions of a scene that fall on the fovea are sampled at high spatial resolution, and receive a disproportionate fraction of cortical processing. We recorded the eye positions of human subjects while they viewed images of natural scenes. We report that active selection affected the statistics of the stimuli encountered by the fovea, and also by the parafovea up to eccentricities of 4 degrees. We found two related effects. First, subjects looked at image regions that had high spatial contrast. Second, in these regions, the intensities of nearby image points (pixels) were less correlated with each other than in images selected at random. These effects could serve to increase the information available to the visual system for further processing. We show that both of these effects can be simply obtained by constructing an artificial ensemble comprised of the highest-contrast regions of images. <s> BIB002 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> The authors investigated selective attention in patients with Alzheimer's disease (AD), using a well-known visual search procedure. In simple feature search, the deficit observed in AD patients represented a baseline shift in the median hit reaction time (RT). On the conjoined feature search task, the median hit RT for AD patients increased disproportionately with increasing array size, indicating an additional cognitive impairment on this task. Of particular importance, the cognitive deficit observed in conjunction search was more profound than that predicted on the basis of previous reports of global cognitive slowing in AD. There was some evidence that the performance of AD patients improved more than the performance of controls over the duration of the experimental test session. Patients also had more difficulty in detecting targets on the right side of hemispace and in more peripheral locations. <s> BIB003 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> Five important trends have emerged from recent work on computational models of focal visual attention that emphasize the bottom-up, image-based control of attentional deployment. First, the perceptual saliency of stimuli critically depends on the surrounding context. Second, a unique 'saliency map' that topographically encodes for stimulus conspicuity over the visual scene has proved to be an efficient and plausible bottom-up control strategy. Third, inhibition of return, the process by which the currently attended location is prevented from being attended again, is a crucial element of attentional deployment. Fourth, attention and eye movements tightly interplay, posing computational challenges with respect to the coordinate system used to control attention. And last, scene understanding and object recognition strongly constrain the selection of attended locations. Insights from these five key areas provide a framework for a computational and neurobiological understanding of visual attention. <s> BIB004 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> We asked whether the poor performance on visual search tasks typical of patients with Alzheimer’s disease (AD) is the result of a selective deficit in the ability to shift attention from item to item, or the consequence of an inefficient processing of each item within the search set. We attempted to manipulate the ease of attention shifting and item processing in a visual search task by manipulating target salience and task difficulty, respectively. Significant effects of both target saliency and task difficulty for both AD patients and age-matched controls were obtained, with the AD group displaying greater effects of both of these manipulations than the controls. This interaction remained even when the reaction time data were log-transformed to account for the overall slower reaction times of the AD group. We conclude that inefficiency in visual search tasks in AD probably represents the product of both attention shifting and target processing factors. <s> BIB005 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> Visual attention is the ability of a vision system, be it biological or artificial, to rapidly detect potentially relevant parts of a visual scene, on which higher level vision tasks, such as object recognition, can focus. The saliency-based model of visual attention represents one of tile main attempts to simulate this visual mechanism on computers. Though biologically inspired, this model has only been partially assessed in comparison with human behavior. Our methodology consists in comparing the computational saliency map with human eye movement patterns. This paper presents an in-depth analysis of the model by assessing the contribution of different cues to visual attention. It reports the results of a quantitative comparison of human visual attention derived from fixation patterns with visual attention as modeled by different versions of the computer model. More specifically, a one-cue gray-level model is compared to a two-cues color model. The experiments conducted with over 40 images of different nature and involving 20 human subjects assess the quantitative contribution of chromatic features in visual attention. <s> BIB006 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> A Bayesian system identification technique was used to determine which image characteristics predict where people fixate when viewing natural images. More specifically an estimate was derived for the mapping between image characteristics at a given location and the probability that this location was fixated. Using a large database of eye fixations to natural images, we determined the most probable (a posteriori) model of this mapping. From a set of candidate feature maps consisting of edge, contrast and luminance maps (at two different spatial scales), fixation probability was dominated by high spatial frequency edge information. The best model applied compressive non-linearity to the high frequency edge detecting filters (approximately a square root). Both low spatial frequency edges and contrast had weaker, but inhibitory, effects. The contributions of the other maps were so small as to be behaviourally irrelevant. This Bayesian method identifies not only the relevant weighting of the different maps, but how this weighting varies as a function of distance from the point of fixation. It was found that rather than centre surround inhibition, the weightings simply averaged over an area of about 2 degrees. <s> BIB007 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> Previous studies of eye gaze have shown that when looking at images containing human faces, observers tend to rapidly ::: focus on the facial regions. But is this true of other high-level image features as well? We here investigate the extent to ::: which natural scenes containing faces, text elements, and cell phones - as a suitable control - attract attention by tracking ::: the eye movements of subjects in two types of tasks - free viewing and search. We observed that subjects in free-viewing ::: conditions look at faces and text 16.6 and 11.1 times more than similar regions normalized for size and position of the face ::: and text. In terms of attracting gaze, text is almost as effective as faces. Furthermore, it is difﬁcult to avoid looking at faces ::: and text even when doing so imposes a cost. We also found that subjects took longer in making their initial saccade when ::: they were told to avoid faces/text and their saccades landed on a non-face/non-text object. We reﬁne a well-known bottom–up ::: computer model of saliency-driven attention that includes conspicuity maps for color, orientation, and intensity by adding ::: high-level semantic information (i.e., the location of faces or text) and demonstrate that this signiﬁcantly improves the ::: ability to predict eye ﬁxations in natural images. Our enhanced model’s predictions yield an area under the ROC curve ::: over 84% for images that contain faces or text when compared against the actual ﬁxation pattern of subjects. This ::: suggests that the primate visual system allocates attention using such an enhanced saliency map. <s> BIB008 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> Recognizing objects being manipulated in hands can provide essential information about a person's activities and have far-reaching impacts on the application of vision in everyday life. The egocentric viewpoint from a wearable camera has unique advantages in recognizing handled objects, such as having a close view and seeing objects in their natural positions. We collect a comprehensive dataset and analyze the feasibilities and challenges of the egocentric recognition of handled objects. We use a lapel-worn camera and record uncompressed video streams as human subjects manipulate objects in daily activities. We use 42 day-to-day objects that vary in size, shape, color and textureness. 10 video sequences are shot for each object under different illuminations and backgrounds. We use this dataset and a SIFT-based recognition system to analyze and quantitatively characterize the main challenges in egocentric object recognition, such as motion blur and hand occlusion, along with its unique constraints, such as hand color, location prior and temporal consistency. SIFT-based recognition has an average recognition rate of 12%, and reaches 20% through enforcing temporal consistency. We use simulations to estimate the upper bound for SIFT-based recognition at 64%, the loss of accuracy due to background clutter at 20%, and that of hand occlusion at 13%. Our quantitative evaluations show that the egocentric recognition of handled objects is a challenging but feasible problem with many unique characteristics and many opportunities for future research. <s> BIB009 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> We examined eye movements in a patient, FK, who has action disorganisation syndrome (ADS), as he performed the everyday task of making a cup of tea. We compared his eye movements with those of a person with Alzheimer's disease and with healthy volunteers. Despite showing very disorganised behaviour many aspects of FK's eye movements were relatively normal. However, unlike normal participants FK made no advance glances to objects that were about to be used, and he made increased numbers of fixations to irrelevant objects during the task. There were also differences in the durations of his eye movements during correct actions and during his perseverative and task-addition responses. We discuss the implications for understanding ADS and the cognitive processes required for correctly performing everyday tasks. <s> BIB010 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> Visual attention is one of the key mechanisms of perception that enables humans to efficiently select the visual data of most potential interest. Machines face similar challenges as humans: they have to deal with a large amount of input data and have to select the most promising parts. In this chapter, we explain the underlying biological and psychophysical grounding of visual attention, show how these mechanisms can be implemented computationally, and discuss why and under what conditions machines, especially robots, profit from such a concept. <s> BIB011 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> This paper addresses the problem of learning object models from egocentric video of household activities, using extremely weak supervision. For each activity sequence, we know only the names of the objects which are present within it, and have no other knowledge regarding the appearance or location of objects. The key to our approach is a robust, unsupervised bottom up segmentation method, which exploits the structure of the egocentric domain to partition each frame into hand, object, and background categories. By using Multiple Instance Learning to match object instances across sequences, we discover and localize object occurrences. Object representations are refined through transduction and object-level classifiers are trained. We demonstrate encouraging results in detecting novel object instances using models produced by weakly-supervised learning. <s> BIB012 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> We propose a method of predicting human egocentric visual attention using bottom-up visual saliency and egomotion information. Computational models of visual saliency are often employed to predict human attention; however, its mechanism and effectiveness have not been fully explored in egocentric vision. The purpose of our framework is to compute attention maps from an egocentric video that can be used to infer a person's visual attention. In addition to a standard visual saliency model, two kinds of attention maps are computed based on a camera's rotation velocity and direction of movement. These rotation-based and translation-based attention maps are aggregated with a bottom-up saliency map to enhance the accuracy with which the person's gaze positions can be predicted. The efficiency of the proposed framework was examined in real environments by using a head-mounted gaze tracker, and we found that the egomotion-based attention maps contributed to accurately predicting human visual attention. <s> BIB013 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> Humans and other primates shift their gaze to allocate processing resources to a subset of the visual input. Understanding and emulating the way that human observers free-view a natural scene has both scientific and economic impact. It has therefore attracted the attention from researchers in a wide range of science and engineering disciplines. With the ever increasing computational power, machine learning has become a popular tool to mine human data in the exploration of how people direct their gaze when inspecting a visual scene. This paper reviews recent advances in learning saliency-based visual attention and discusses several key issues in this topic. <s> BIB014 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> Modeling visual attention-particularly stimulus-driven, saliency-based attention-has been a very active research area over the past 25 years. Many different models of attention are now available which, aside from lending theoretical contributions to other fields, have demonstrated successful applications in computer vision, mobile robotics, and cognitive systems. Here we review, from a computational perspective, the basic concepts of attention implemented in these models. We present a taxonomy of nearly 65 models, which provides a critical comparison of approaches, their capabilities, and shortcomings. In particular, 13 criteria derived from behavioral and computational studies are formulated for qualitative comparison of attention models. Furthermore, we address several challenging issues with models, including biological plausibility of the computations, correlation with eye movement datasets, bottom-up and top-down dissociation, and constructing meaningful performance measures. Finally, we highlight current research trends in attention modeling and provide insights for future. <s> BIB015 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> Visual attention is a process that enables biological and machine vision systems to select the most relevant regions from a scene. Relevance is determined by two components: 1) top-down factors driven by task and 2) bottom-up factors that highlight image regions that are different from their surroundings. The latter are often referred to as “visual saliency.” Modeling bottom-up visual saliency has been the subject of numerous research efforts during the past 20 years, with many successful applications in computer vision and robotics. Available models have been tested with different datasets (e.g., synthetic psychological search arrays, natural images or videos) using different evaluation scores (e.g., search slopes, comparison to human eye tracking) and parameter settings. This has made direct comparison of models difficult. Here, we perform an exhaustive comparison of 35 state-of-the-art saliency models over 54 challenging synthetic patterns, three natural image datasets, and two video datasets, using three evaluation scores. We find that although model rankings vary, some models consistently perform better. Analysis of datasets reveals that existing datasets are highly center-biased, which influences some of the evaluation scores. Computational complexity analysis shows that some models are very fast, yet yield competitive eye movement prediction accuracy. Different models often have common easy/difficult stimuli. Furthermore, several concerns in visual saliency modeling, eye movement datasets, and evaluation scores are discussed and insights for future work are provided. Our study allows one to assess the state-of-the-art, helps to organizing this rapidly growing field, and sets a unified comparison framework for gauging future efforts, similar to the PASCAL VOC challenge in the object recognition and detection domains. <s> BIB016 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> In this paper, we present a saliency guided image object segment method. We suppose that saliency maps can indicate informative regions, and filter out background in images. To produce perceptual satisfactory salient objects, we use our bio-inspired saliency measure which integrating three factors: dissimilarity, spatial distance and central bias to compute saliency map. Then the saliency map is used as the importance map in the salient object segment method. Experimental results demonstrate that our method outperforms previous saliency detection method, yielding higher precision (0.7669) and better recall rates (0.825), F-Measure (0.7545), when evaluated using one of the largest publicly available data sets. <s> BIB017 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> This paper presents a saliency-based video object extraction (VOE) framework. The proposed framework aims to automatically extract foreground objects of interest without any user interaction or the use of any training data (i.e., not limited to any particular type of object). To separate foreground and background regions within and across video frames, the proposed method utilizes visual and motion saliency information extracted from the input video. A conditional random field is applied to effectively combine the saliency induced features, which allows us to deal with unknown pose and scale variations of the foreground object (and its articulated parts). Based on the ability to preserve both spatial continuity and temporal consistency in the proposed VOE framework, experiments on a variety of videos verify that our method is able to produce quantitatively and qualitatively satisfactory VOE results. <s> BIB018 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> We present a model for gaze prediction in egocentric video by leveraging the implicit cues that exist in camera wearer's behaviors. Specifically, we compute the camera wearer's head motion and hand location from the video and combine them to estimate where the eyes look. We further model the dynamic behavior of the gaze, in particular fixations, as latent variables to improve the gaze prediction. Our gaze prediction results outperform the state-of-the-art algorithms by a large margin on publicly available egocentric vision datasets. In addition, we demonstrate that we get a significant performance boost in recognizing daily actions and segmenting foreground objects by plugging in our gaze predictions into state-of-the-art methods. <s> BIB019 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> The emergence of new wearable technologies, such as action cameras and smart glasses, has increased the interest of computer vision scientists in the first person perspective. Nowadays, this field is attracting attention and investments of companies aiming to develop commercial devices with first person vision (FPV) recording capabilities. Due to this interest, an increasing demand of methods to process these videos, possibly in real time, is expected. The current approaches present a particular combinations of different image features and quantitative methods to accomplish specific objectives like object detection, activity recognition, user-machine interaction, and so on. This paper summarizes the evolution of the state of the art in FPV video analysis between 1997 and 2014, highlighting, among others, the most commonly used features, methods, challenges, and opportunities within the field. <s> BIB020 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> Detecting and segmenting salient objects from natural scenes, often referred to as salient object detection, has attracted great interest in computer vision. While many models have been proposed and several applications have emerged, a deep understanding of achievements and issues remains lacking. We aim to provide a comprehensive review of recent progress in salient object detection and situate this field among other closely related areas such as generic scene segmentation, object proposal generation, and saliency for fixation prediction. Covering 228 publications, we survey i) roots, key concepts, and tasks, ii) core techniques and main modeling trends, and iii) datasets and evaluation metrics for salient object detection. We also discuss open problems such as evaluation metrics and dataset bias in model performance, and suggest future research directions. <s> BIB021 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> In this paper, we propose a human activity recognition method from first-person videos, which provides a supplementary method to improve the recognition accuracy. Conventional methods detect objects and derive a user's behavior based on their taxonomy. One of the recent works has achieved accuracy improvement by determining key objects based on hand manipulation. However, such manipulation-based approach has a restriction on applicable scenes and object types because the user's hands don't always present significant information. In contrast, our proposed attention-based approach provides a solution to detect visually salient objects as key objects in a non-contact manner. Experimental results show that the proposed method classifies first-person actions more accurately than the previous method by 6.4 percentage points and its average accuracy reaches 43.3%. <s> BIB022 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> In this paper, we present a unified statistical framework for modeling both saccadic eye movements and visual saliency. By analyzing the statistical properties of human eye fixations on natural images, we found that human attention is sparsely distributed and usually deployed to locations with abundant structural information. This observations inspired us to model saccadic behavior and visual saliency based on super-Gaussian component (SGC) analysis. Our model sequentially obtains SGC using projection pursuit, and generates eye movements by selecting the location with maximum SGC response. Besides human saccadic behavior simulation, we also demonstrated our superior effectiveness and robustness over state-of-the-arts by carrying out dense experiments on synthetic patterns and human eye fixation benchmarks. Multiple key issues in saliency modeling research, such as individual differences, the effects of scale and blur, are explored in this paper. Based on extensive qualitative and quantitative experimental results, we show promising potentials of statistical approaches for human behavior research. <s> BIB023 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> The social difficulties that are a hallmark of autism spectrum disorder (ASD) are thought to arise, at least in part, from atypical attention toward stimuli and their features. To investigate this hypothesis comprehensively, we characterized 700 complex natural scene images with a novel three-layered saliency model that incorporated pixel-level (e.g., contrast), object-level (e.g., shape), and semantic-level attributes (e.g., faces) on 5,551 annotated objects. Compared with matched controls, people with ASD had a stronger image center bias regardless of object distribution, reduced saliency for faces and for locations indicated by social gaze, and yet a general increase in pixel-level saliency at the expense of semantic-level saliency. These results were further corroborated by direct analysis of fixation characteristics and investigation of feature interactions. Our results for the first time quantify atypical visual attention in ASD across multiple levels and categories of objects. <s> BIB024 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> Visually-guided behavior recruits a network of brain regions so extensive that it is often affected by neuropsychiatric disorders, producing measurable atypical oculomotor signatures. Wang et al. (2015) combine eye tracking with computational attention models to decipher the neurobehavioral signature of autism. <s> BIB025 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> We propose a new top down probabilistic saliency model for egocentric video content. It aims to predict top-down visual attention maps focused on manipulated objects, that are then used for psycho-visual weighting of features in the problem of manipulated object recognition. The model is probabilistically defined using both global and local appearance features extracted from automatically segmented arm areas and objects. A psycho-visual experiment has been conducted in a guided framework that compares our proposal and other popular state-of-the-art models with respect to human gaze fixations. The obtained results show that our approach outperforms several popular bottom-up saliency approaches in a well-known egocentric dataset. Furthermore, an additional task-driven assessment for object recognition in egocentric video reveals that the proposed method improves the performance of several state-of-the-art techniques for object detection. HighlightsWe propose a new top down probabilistic saliency model for egocentric video content.Top-down saliency maps focused on manipulated objects based on the arms positions.Psycho-visual experiment conducted demonstrating our proposal increased performances over SOA saliency models.SOA performances in manipulated object recognition for this dataset. <s> BIB026 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> Mild cognitive impairment (MCI) refers to the intermediate period between the typical cognitive decline of normal aging and more severe decline associated with dementia, and it is associated with greater risk for progression to dementia. Research has suggested that functional abilities are compromised in MCI, but the degree of impairment and underlying mechanisms remain poorly understood. The development of sensitive measures to assess subtle functional decline poses a major challenge for characterizing functional limitations in MCI. Eye-tracking methodology has been used to describe visual processes in everyday, naturalistic action among healthy older adults as well as several case studies of severely impaired individuals, and it has successfully differentiated healthy older adults from those with MCI on specific visual tasks. These studies highlight the promise of eye-tracking technology as a method to characterize subtle functional decline in MCI. However, to date no studies have examined visual behaviors during completion of naturalistic tasks in MCI. This review describes the current understanding of functional ability in MCI, summarizes findings of eye-tracking studies in healthy individuals, severe impairment, and MCI, and presents future research directions to aid with early identification and prevention of functional decline in disorders of aging. <s> BIB027 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> We focus on the problem of wearer's action recognition in first person a.k.a. egocentric videos. This problem is more challenging than third person activity recognition due to unavailability of wearer's pose and sharp movements in the videos caused by the natural head motion of the wearer. Carefully crafted features based on hands and objects cues for the problem have been shown to be successful for limited targeted datasets. We propose convolutional neural networks (CNNs) for end to end learning and classification of wearer's actions. The proposed network makes use of egocentric cues by capturing hand pose, head motion and saliency map. It is compact. It can also be trained from relatively small number of labeled egocentric videos that are available. We show that the proposed network can generalize and give state of the art performance on various disparate egocentric action datasets. <s> BIB028 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> Data-driven saliency has recently gained a lot of attention thanks to the use of Convolutional Neural Networks for predicting gaze fixations. In this paper we go beyond standard approaches to saliency prediction, in which gaze maps are computed with a feed-forward network, and present a novel model which can predict accurate saliency maps by incorporating neural attentive mechanisms. The core of our solution is a Convolutional LSTM that focuses on the most salient regions of the input image to iteratively refine the predicted saliency map. Additionally, to tackle the center bias typical of human eye fixations, our model can learn a set of prior maps generated with Gaussian functions. We show, through an extensive evaluation, that the proposed architecture outperforms the current state of the art on public saliency prediction datasets. We further study the contribution of each key component to demonstrate their robustness on different scenarios. <s> BIB029 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> Recently, saliency detection has been becoming a popular topic in computer vision. In this paper we propose an object-level saliency detection algorithm which explicitly explores bottom-up visual attention and objectness cues. Firstly, some category-independent object candidates are segmented from the image by the quantized color attributes. Then two metrics, global cues and candidate objectness, are developed to estimate the saliency in the whole image and the object candidates respectively. We use global cues to describe the focusness and spatial distribution of color attributes in the entire image. As supplementary, candidate objectness can reveal the objectness of the object candidates. Finally, we integrate the two saliency cues to derive a saliency map of the image. By explicitly fusing candidate objectness and global cues, our proposed method is more suitable for processing images with complex background. Experimental results on three large benchmark databases demonstrate that the proposed method achieves more favorable performance than 15 recent state-of-the-art methods in terms of three evaluation criterions. We propose an object-level salient detection algorithm which explicitly explores bottom-up visual attention and objectness cues.Some category-independent object candidates are firstly segmented from the image by the quantized color attributes of images.Global cues and candidate objectness are developed, to evaluate bottom-up visual attention of the whole image and the objectness of the object candidates respectively.By explicitly combining local objectness cue with global low-level saliency cues with candidates location and color attributes, our proposed method is more suitable for processing images with complex background. <s> BIB030 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> Human eye fixations often correlate with locations of salient objects in the scene. However, only a handful of approaches have attempted to simultaneously address the related aspects of eye fixations and object saliency. In this work, we propose a deep convolutional neural network (CNN) capable of predicting eye fixations and segmenting salient objects in a unified framework. We design the initial network layers, shared between both the tasks, such that they capture the object level semantics and the global contextual aspects of saliency, while the deeper layers of the network address task specific aspects. In addition, our network captures saliency at multiple scales via inceptionstyle convolution blocks. Our network shows a significant improvement over the current state-of-the-art for both eye fixation prediction and salient object segmentation across a number of challenging datasets. <s> BIB031 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> Salient object detection has recently witnessed substantial progress due to powerful features extracted using deep convolutional neural networks (CNNs). However, existing CNN-based methods operate at the patch level instead of the pixel level. Resulting saliency maps are typically blurry, especially near the boundary of salient objects. Furthermore, image patches are treated as independent samples even when they are overlapping, giving rise to significant redundancy in computation and storage. In this paper, we propose an end-to-end deep contrast network to overcome the aforementioned limitations. Our deep network consists of two complementary components, a pixel-level fully convolutional stream and a segment-wise spatial pooling stream. The first stream directly produces a saliency map with pixel-level accuracy from an input image. The second stream extracts segment-wise features very efficiently, and better models saliency discontinuities along object boundaries. Finally, a fully connected CRF model can be optionally incorporated to improve spatial coherence and contour localization in the fused result from these two streams. Experimental results demonstrate that our deep model significantly improves the state of the art. <s> BIB032 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> Traditional1 salient object detection models often use hand-crafted features to formulate contrast and various prior knowledge, and then combine them artificially. In this work, we propose a novel end-to-end deep hierarchical saliency network (DHSNet) based on convolutional neural networks for detecting salient objects. DHSNet first makes a coarse global prediction by automatically learning various global structured saliency cues, including global contrast, objectness, compactness, and their optimal combination. Then a novel hierarchical recurrent convolutional neural network (HRCNN) is adopted to further hierarchically and progressively refine the details of saliency maps step by step via integrating local context information. The whole architecture works in a global to local and coarse to fine manner. DHSNet is directly trained using whole images and corresponding ground truth saliency masks. When testing, saliency maps can be generated by directly and efficiently feedforwarding testing images through the network, without relying on any other techniques. Evaluations on four benchmark datasets and comparisons with other 11 state-of-the-art algorithms demonstrate that DHSNet not only shows its significant superiority in terms of performance, but also achieves a real-time speed of 23 FPS on modern GPUs. <s> BIB033 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> Convolutional-deconvolution networks can be adopted to perform end-to-end saliency detection. But, they do not work well with objects of multiple scales. To overcome such a limitation, in this work, we propose a recurrent attentional convolutional-deconvolution network (RACDNN). Using spatial transformer and recurrent network units, RACDNN is able to iteratively attend to selected image sub-regions to perform saliency refinement progressively. Besides tackling the scale problem, RACDNN can also learn context-aware features from past iterations to enhance saliency refinement in future iterations. Experiments on several challenging saliency detection datasets validate the effectiveness of RACDNN, and show that RACDNN outperforms state-of-the-art saliency detection methods. <s> BIB034 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> In a wearable camera video, we see what the camera wearer sees. While this makes it easy to know roughly Open image in new window , it does not immediately reveal Open image in new window . Specifically, at what moments did his focus linger, as he paused to gather more information about something he saw? Knowing this answer would benefit various applications in video summarization and augmented reality, yet prior work focuses solely on the “what” question (estimating saliency, gaze) without considering the “when” (engagement). We propose a learning-based approach that uses long-term egomotion cues to detect engagement, specifically in browsing scenarios where one frequently takes in new visual information (e.g., shopping, touring). We introduce a large, richly annotated dataset for ego-engagement that is the first of its kind. Our approach outperforms a wide array of existing methods. We show engagement can be detected well independent of both scene appearance and the camera wearer’s identity. <s> BIB035 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> Abstract Egocentric videos are characterized by their ability to have the first person view. With the popularity of Google Glass and GoPro, use of egocentric videos is on the rise. With the substantial increase in the number of egocentric videos, the value and utility of recognizing actions of the wearer in such videos has also thus increased. Unstructured movement of the camera due to natural head motion of the wearer causes sharp changes in the visual field of the egocentric camera causing many standard third person action recognition techniques to perform poorly on such videos. Objects present in the scene and hand gestures of the wearer are the most important cues for first person action recognition but are difficult to segment and recognize in an egocentric video. We propose a novel representation of the first person actions derived from feature trajectories. The features are simple to compute using standard point tracking and do not assume segmentation of hand/objects or recognizing object or hand pose unlike in many previous approaches. We train a bag of words classifier with the proposed features and report a performance improvement of more than 11% on publicly available datasets. Although not designed for the particular case, we show that our technique can also recognize wearer's actions when hands or objects are not visible. <s> BIB036 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> Video-based recognition of activities of daily living (ADLs) is being used in ambient assisted living systems in order to support the independent living of older people. However, current systems based on cameras located in the environment present a number of problems, such as occlusions and a limited field of view. Recently, wearable cameras have begun to be exploited. This paper presents a review of the state of the art of egocentric vision systems for the recognition of ADLs following a hierarchical structure: motion, action and activity levels, where each level provides higher semantic information and involves a longer time frame. The current egocentric vision literature suggests that ADLs recognition is mainly driven by the objects present in the scene, especially those associated with specific tasks. However, although object-based approaches have proven popular, object recognition remains a challenge due to the intra-class variations found in unconstrained scenarios. As a consequence, the performance of current systems is far from satisfactory. <s> BIB037 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> Studies of visual attention of patients with Dementia such as Parkinson’s Disease Dementia and Alzheimer Disease is a promising way for non-invasive diagnostics. Past research showed, that people suffering from dementia are not reactive with regard to degradations on still images. Attempts are being made to study their visual attention relatively to the video content. Here the delays in their reactions on novelty and “unusual” novelty of the visual scene are expected. Nevertheless, large-scale screening of population is possible only if sufficiently robust automatic prediction models can be built. In the medical protocols the detection of Dementia behavior in visual content observation is always performed in comparison with healthy, “normal control” subjects. Hence, it is a research question per see as to develop an automatic prediction models for specific visual content to use in psycho-visual experience involving Patients with Dementia (PwD). The difficulty of such a prediction resides in a very small amount of training data. In this paper the reaction of healthy normal control subjects on degraded areas in videos was studied. Furthermore, in order to build an automatic prediction model for salient areas in intentionally degraded videos for PwD studies, a deep learning architecture was designed. Optimal transfer learning strategy for training the model in case of very small amount of training data was deployed. The comparison with gaze fixation maps and classical visual attention prediction models was performed. Results are interesting regarding the reaction of normal control subjects against degraded areas in videos. <s> BIB038 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> We introduce SalGAN, a deep convolutional neural network for visual saliency prediction trained with adversarial examples. The first stage of the network consists of a generator model whose weights are learned by back-propagation computed from a binary cross entropy (BCE) loss over downsampled versions of the saliency maps. The resulting prediction is processed by a discriminator network trained to solve a binary classification task between the saliency maps generated by the generative stage and the ground truth ones. Our experiments show how adversarial training allows reaching state-of-the-art performance across different metrics when combined with a widely-used loss function like BCE. Our results can be reproduced with the source code and trained models available at https://imatge-upc.github. io/saliency-salgan-2017/. <s> BIB039 </s> Computational Techniques for Eye Movements Analysis towards Supporting Early Diagnosis of Alzheimer's Disease: A Review <s> Towards Early Detection Leveraging on Computational Attention Modelling <s> Inherent in visual scene analysis is a bottleneck associated with the need to sequentially sample locations with foveating eye movements. The concept of a 'saliency map' topographically encoding stimulus conspicuity over the visual scene has proven to be an efficient predictor of eye movements. Our work reviews insights into the neurobiological implementation of visual salience computation. We start by summarizing the role that different visual brain areas play in salience computation, whether at the level of feature analysis for bottom-up salience or at the level of goal-directed priority maps for output behaviour. We then delve into how a subcortical structure, the superior colliculus (SC), participates in salience computation. The SC represents a visual saliency map via a centre-surround inhibition mechanism in the superficial layers, which feeds into priority selection mechanisms in the deeper layers, thereby affecting saccadic and microsaccadic eye movements. Lateral interactions in the local SC circuit are particularly important for controlling active populations of neurons. This, in turn, might help explain long-range effects, such as those of peripheral cues on tiny microsaccades. Finally, we show how a combination of in vitro neurophysiology and large-scale computational modelling is able to clarify how salience computation is implemented in the local circuit of the SC.This article is part of the themed issue 'Auditory and visual scene analysis'. <s> BIB040
As we mentioned before, the identification of abnormalities in eye movements to support the early diagnosis or progression of an eventual dementia disease in elderly population during performance of ADL is a real scientific challenge. In this sense, some interesting approaches propose to predict human visual attention emulating the Human Visual Systems performance  . Indeed, computational visual attention models (CVSM) attempt to explain and describe the process of perceptual behavior and are compared with ground truth measured by eye trackers in psychovisual experiments BIB011 BIB004  . Several exciting CVSM such as visual saliency techniques on egocentric video are useful for the use in naturalistic scenarios to estimate the areas from the video that are more likely to become the focus of human visual attention  . The egocentric video provides a first-person view from the individual who "wears" an egocentric camera giving visual information about objects, locations, and interactions. Visual saliency techniques have been already combined with eye tracking in the field of Autism Spectrum Disorder (ASD) to screen differences between people with ASD and controls BIB024 . Researchers compare eye movements from both groups when freely viewing natural scenes images. In the analysis fixations towards visually salient regions, such as color, intensity, orientation, objects, and faces, are considered. To the best of our knowledge, research regarding the combination of data from eye trackers and visual saliency modeling to analyze eye movements abnormalities from PwAD is scarce. As we show earlier in Section 3.2 the work in BIB038 approaches with the analysis of fixations to degraded images, but more research is missing. Suitable devices identified to be appropriated for monitoring ADLs with egocentric vision capabilities are mobile eye trackers BIB025 , such as Tobbi© or SMI©. Additionally, egocentric cameras such as GoPro, Samsung Glass, and Microsoft Sense Cam BIB028 would allow scene understanding. Certainly, they record egocentric video giving a first-person view or in other words, what the camera wearer sees BIB020 . This captured information can be useful to analyze or predict some or all of visual attentive behavior through visual saliency computation. In this section, the main characteristics from the research on visual saliency are described, the techniques used and how this research field can be applied in the context of eye movements analysis. (1) Computational Visual Saliency Models. The research field that analyzes video computationally to estimate the image regions that attract visual attention is called visual saliency detection BIB039 BIB029 . This research field matches the areas of neuroscience, psychology, and computer vision BIB030 . Early work on visual saliency modelling uses handcrafted low-level features such as contrast BIB002 , color BIB006 , edges BIB007 , and orientation. It is funded on feature integration theory by Treisman and Gelade BIB001 . In addition, there is work that performs higher-level features extraction such as objects BIB021 and faces BIB008 in order to incorporate, into the low-level features, semantic elements of the observed scenes. Moreover, since the boom of deep learning, there are different proposals of configuration and arrangements of these supervised classification tools such as the convolutional neural networks BIB031 BIB032 BIB033 that report increased results for saliency estimation tasks. However, as deep learning requires a huge amount of data, more annotated information on diverse scenarios is still required. The works in  BIB014 BIB015 review techniques used for visual saliency detection. According to the method used for modeling attention, there are two main categories of methods followed in the research of visual saliency: bottom-up modelling and topdown modelling. Bottom-up methods use information such as color, contrast, orientation, and texture BIB034 ; they predict stimuli driven attention. And top-down models require prior knowledge on the visual search task and the context  . Currently, most of the work in visual saliency enters in the category of bottom-up methods. The visual saliency modelling outputs a saliency map S, which is a two-dimensional topographically arranged map that encodes stimulus conspicuity of the visual scene BIB040 . The pixel values in the map indicate the saliency degree of the corresponding regions in the visual scene  . The maps are compared against ground truth maps built upon gaze fixations recorded by eye trackers when subjects are performing visual tasks. The ground truth might include synthetic stimuli or come from natural scenes including still images and video BIB016 . Pioneer research on visual saliency studied saliency from a third-person perspective, for instance, using still images BIB017 or video BIB018 coming from nonegocentric cameras. In the scenario of understanding scene for AD studies, the interest is nevertheless in saliency from the point of view of the subject performing activities. Hence, building saliency maps in egocentric video content and from the point of view of the subjects wearing recording device is required. However, the analysis of egocentric video brings new challenges. For instance, motion cues are significant in third-person video but camera motion is inherent to egocentric recordings BIB035 . Therefore, a residual motion in image plane of egocentric video has to be computed after motion of camera wearer has been compensated  . Egocentric video allows better introducing contextual knowledge from the subject because egocentric video follows the field of view of the subject's action. In this sense, this paradigm can support top-down attention modeling BIB026 in naturalistic scenarios such as in ADLs execution. Egocentric video, beside the bottom-up image cues, provides context about the manipulation of objects BIB012 BIB009 , about hands positions BIB026 BIB019 , ego-motion BIB022 , actions  BIB036 , and activities  BIB037 . The egocentric video supports top-down attention modelling by the ensemble of these diverse contextual information. For example, a model assumes that gaze goes towards a given object currently held by the subject's hands. In addition, the ego-motion that occurs in visual exploration towards locating a specific object that is required in a given activity also serves for gaze estimation BIB013 . Top-down and bottom-up computational visual attention modelling show correlation between human fixations and predicted saliency maps. However, most of the results arise from experiments including healthy participants while scarce studies involve PwAD. In this review, we focus on the scenario for early AD detection. Thus, we address the relevant work relating computational attention modelling applied to AD in the next section. (2) Computational Visual Saliency Models and Diagnosis. Progress in prediction of visual saliency including in egocentric content makes it possible to build robust prediction models for regions of high expectancy for the fixations of the test subject. Therefore, if a subject executing an ADL does not fixate the predicted areas properly, then it can be supposed that this subject has to undergo further tests to diagnose AD or not. For healthy subjects, visual saliency techniques assume that the subject will execute the activity fixating toward relevant objects or with coherent visual exploration. However, as mentioned in the work by Forde et al. BIB010 , the participant with AD had lower proportion of relevant fixations compared to healthy subjects in ADL settings. Another relevant feature about gaze measuring in patients with AD is the sensitivity. For example, the diagnosis performed with SEM tasks requires saccadic sensitivity. However, current visual saliency techniques addressing saccadic estimation are in early stages BIB023 . Although top-down mechanisms dominate the attention modelling from healthy subjects, it has been suggested that the visual behavior from persons with cognitive problems might rely on bottom-up mechanisms and saliency driven with less fixations on objects relevant to the task BIB027 . The work in BIB003 suggests that visual attention problems from AD patients are more notorious when the target item is not salient and shares common features with the background. The study in BIB005 analyzes the visual search task performance from AD patients by conducting experiments using salient and not salient search conditions. The researchers measure the reaction times when PwAD and control participants search for target elements. The PwAD show longer reaction times than control participants. However, the gap between both groups is bigger when searching for nonsalient target items. This suggests that salient elements attract PwAD. The research in  on egocentric video acknowledges the potential to use visual saliency techniques to develop a tool for medical practitioners in realistic ADL scenarios. The researchers perform gaze comparison from an actor and a viewer. The actor is a person performing an activity (potentially a PwAD) while the viewer is a person watching egocentric video recordings from the actor (the medical practitioner). The paper suggests a relation between the gaze from the actor and the viewer. This relation consists in a time shift between the points of attention from the actor and the viewer. In other words, the viewer looks at the same place in the visual scene compared to the (healthy) actor but a few milliseconds later. The potential of this tool relies on the ability of a system to determine if the gaze from the actor is normal or abnormal according to the perspective from a medical practitioner. Additionally, the settings from the tool, like the use of egocentric camera, allow the use of computational attention-modelling techniques. Computational attention-modelling techniques can be complemented with other contextual information in order to know more about the subject oculomotor behavior. For example, the amount of time that a participant takes to complete an activity can be explored. As literature shows, people with cognitive deficits take longer to complete tasks.