Visual Attention for Robotic Cognition: A Survey <s> B. Robotic Visual Attention: Issues and Challenges <s> This paper argues that the object- and space-based modes of visual attention can be naturally integrated in a common mathematical framework. In an earlier work we have proposed a mathematical model of visual attention for robotic system exploiting the knowledge of visual attention mechanism of the primates. This paper investigates on the validity of the proposed model for robotic systems through experimentation on a real robot. The paper sheds light on a number of real world issues involved with the design of visual attention system for physically embodied robots and explains how the proposed Bayesian model of visual attention addresses these issues. The object- and space-based modes of visual attention are naturally integrated in the model and is reflected in the sequential Monte Carlo implementation of the model on a real robot. <s> BIB001 </s> Visual Attention for Robotic Cognition: A Survey <s> B. Robotic Visual Attention: Issues and Challenges <s> ‘Saliency map’, a scalar, two-dimensional representation of visual saliency, is at the center of almost all of the existing models of visual attention. Attention is directed toward the most salient location in the saliency map. The mechanism of ‘inhibition of return’ (IOR) suppresses the saliency of recently attended location and thereby enabling the shifts of attention toward different locations in the saliency map in order of decreasing saliency. This process performs well as longs as the attention is directed covertly. For overt attention with head movements, which is practically the case in robotic applications, the visual saliency as well as the frame of reference in which the IOR is expressed change after every head movement. These pose a set of computational challenges in implementing attention behavior and IOR. This paper argues that the re-mapping of visual saliency and dynamic shift of IOR emerge naturally in a particle filter based framework of visual attention. Experiments on a real camera head validate the arguments. <s> BIB002
The computer vision models of attention perform remarkably well in most of the computer vision applications where static images or images from a video stream are manually fed to the model in order to identify the most salient/task-relevant stimuli. In case of some real-time applications where the current visual input of the attention model is to be determined by the decision output of the model (i.e., the focus of attention) at the immediate past, the traditional computer vision models of visual attention faces severe limitations in a number of aspects BIB001 - BIB002 . Using a visual attention model as a component of robotic cognition is an example of such applications. In this case, the attention model should be able to locate the behaviorally relevant stimuli in an ongoing stream of visual input and respond to it, perform learning in an on-line fashion and with minimal human supervision, and apply the learned knowledge for guiding the attention behavior in arbitrary environmental settings. The issues and challenges involved with robotic visual attention are summarized below. Issue 1: Overt Shift of Attention: In majority of robotic applications (e.g., social robots, assistive robots, and entertainment robots) it is desired that selective attention will be accompanied by a saccadic movement of the camera head of the robot. Such movement is necessary to place the object of attention at the center of the camera frame and facilitates the learning of the focused object. A computational model of attention for the robots, therefore, requires an integration of the covert and overt modes in a common framework, much like the same way the primates integrate covert and overt shift of attention. The overt shift of attention leads to the following issues that has to be solved to design a model of attention for the robot.