Visual Attention for Robotic Cognition: A Survey <s> Issue 1.2: Dynamic IOR: <s> A visual attention system, inspired by the behavior and the neuronal architecture of the early primate visual system, is presented. Multiscale image features are combined into a single topographical saliency map. A dynamical neural network then selects attended locations in order of decreasing saliency. The system breaks down the complex problem of scene understanding by rapidly selecting, in a computationally efficient manner, conspicuous locations to be analyzed in detail. Index Terms—Visual attention, scene analysis, feature extraction, target detection, visual search. ———————— F ———————— <s> BIB001 </s> Visual Attention for Robotic Cognition: A Survey <s> Issue 1.2: Dynamic IOR: <s> This paper describes a multi-disciplinary initiative to promote collaborative research in enactive artificial cognitive systems by developing the iCub : a open-systems 53 degree-of-freedom cognitive humanoid robot. At 94 cm tall, the iCub is the same size as a three year-old child. It will be able to crawl on all fours and sit up, its hands will allow dexterous manipulation, and its head and eyes are fully articulated. It has visual, vestibular, auditory, and haptic sensory capabilities. As an open system, the design and documentation of all hardware and software is licensed under the Free Software Foundation GNU licences so that the system can be freely replicated and customized. We begin this paper by outlining the enactive approach to cognition, drawing out the implications for phylogenetic configuration, the necessity for ontogenetic development, and the importance of humanoid embodiment. This is followed by a short discussion of our motivation for adopting an open-systems approach. We proceed to describe the iCub's mechanical and electronic specifications, its software architecture, its cognitive architecture. We conclude by discussing the iCub phylogeny, i.e. the robot's intended innate abilities, and an scenario for ontogenesis based on human neonatal development. <s> BIB002 </s> Visual Attention for Robotic Cognition: A Survey <s> Issue 1.2: Dynamic IOR: <s> Visual attention is one of the important phenomena in biological vision which can be followed to achieve more efficiency, intelligence, and robustness in artificial vision systems. This paper investigates a region-based approach that performs pixel clustering prior to the processes of attention in contrast to late clustering as done by contemporary methods. The foundation steps of feature map construction for the region-based attention model are proposed here. The color contrast map is generated based upon the extended findings from the color theory, the symmetry map is constructed using a novel scanning-based method, and a new algorithm is proposed to compute a size contrast map as a formal feature channel. Eccentricity and orientation are computed using the moments of obtained regions and then saliency is evaluated using the rarity criteria. The efficient design of the proposed algorithms allows incorporating five feature channels while maintaining a processing rate of multiple frames per second. Another salient advantage over the existing techniques is the reusability of the salient regions in the high-level machine vision procedures due to preservation of their shapes and precise locations. The results indicate that the proposed model has the potential to efficiently integrate the phenomenon of attention into the main stream of machine vision and systems with restricted computing resources such as mobile robots can benefit from its advantages. <s> BIB003
The role of IOR in robotic attention is the same as that in the biological attention system: allowing the shift of attention toward fresh stimuli  . Failure to implement the IOR properly might cause a robot to oscillate between two stimuli. In overt attention, camera movement causes the location of a stimulus to shift in the image coordinate. It is, therefore, required to design a dynamic IOR strategy where (b) The inhibited region is mapped to the new image coordinate system at frame k. A "sough for" object appears within the inhibited region and the robot ignores its presence. (c) A random head movement in search of the "sought for" object causes it to go out of the VF. As a result, the robot requires longer time to find the inquired object. the location of the recently attended object will me mapped in the new image coordinate in order to inhibit its candidacy as the next focus of attention. The space-based dynamic IOR introduces the complexity that if, between two successive frame capture, a new object appears at the inhibited location of a recently attended object, the robot completely ignores its presence. Fig. 7 demonstrates one instance of this problem. This incurs a longer time to identify a "sought for" object during visual search. It is, therefore, beneficial to integrate space-based IOR with its object-based counterpart. The object-based IOR, however, introduces the problem of object correspondence. In order to inhibit a recently attended object from being attended again, the robot needs to identify it in the shifted image coordinate. This is generally a challenging task due to change in camera perspective, lighting, image blurring due to camera motion, and partial appearance of objects. Issue 1.3: Partial Appearance: Due to head movement, it is highly likely that a number of objects will partially/completely go out of the camera frame. The probability of this increases when the robot uses a narrow angle optics for the camera or when the objects are located either very close to the camera or near the periphery of the frame. Due to partial appearance, the robot might fail to identify a recently attended object. This, in turn, results in a failure to apply the IOR on it and the robot might reattend the same object. In the worst case, the attention of the robot will start oscillating among a set of objects. The same trapped situation might also occur if the robot always finds a set of objects "attention worthy" (e.g., because of their novelty) as it can not match the partially perceived features of these objects with its memory database of previously attended and learned features. Application of space-based dynamic IOR on all of the previously attended locations might relax this problem at the expense of worsening the problem stated under Issue 1.2. Issue 2: Integrated Space-and Object-Based Analysis: The space-based analysis, commonly used in majority of the computational models of attention, does not practically fulfill the interest of most robotic applications. Instead of a single pixel reported as the focus of attention, the information about the object underlying that salient pixel is of greater interest in robotic applications. The space-based model of attention, which generally relies on a traditional saliency map, does not preserve the information of the underlying objects. Another serious problem is the common practice of using coarse scales of the input image for space-based saliency map construction BIB001 ,  ,  . This causes the fading of many attention worthy small regions which do not get chance to be highlighted in the saliency map BIB003 . The problem of space-based dynamic IOR as stated under Issue 1.2 is another consequence of space-based analysis. It is, therefore, beneficial to integrate space-and object-based analysis within the same framework. Issue 3: Optimal Learning Strategy: This issues is particularly related to visual search. To perform a search for an object the robot needs to know the visual features of the target object. Because of the extended number of sensors and actuators, the modern-day robots are blessed with higher degrees of freedom in their visual perception. Even an static object in the environment can be perceived by the robot from arbitrary viewing angle. For a dynamic object the possibilities are even higher. To the best of our knowledge, there is no such image feature which is invariant to arbitrary affine transformation, change in viewing angle and lighting condition. Consequently, in order to identify an object in an arbitrary setting the robot requires to learn "several" views of the object. The precise number to quantify the term "several," however, is not known. The visual attention model of a robot must have a strategy to learn the visual features of an object from different view angles and with minimum human supervision. Issue 4: Generality: As shown in Fig. 4 , in majority of the computer vision models of visual attention, visual search, and visual exploration run in mutual exclusion of each other. The desired loop of attention (visual search or visual exploration) is manually activated by the programmer. Such a manual selection of visual attention mode significantly reduces the generality of an attention model and makes it unsuitable for robotic systems. A robotic visual attention model must be able to switch back-and-forth autonomously between the two modes of attention depending on the behavioral requirement. Issue 5: Prior Training: The robotic applications can not afford to have a separate off-line training phase for visual search. A robot has a very little use as a task-assistant of human if it requires a precise training to learn every possible object prior to performing a search for it. Rather, it is generally expected in the cognitive robots that they will learn while working, much like the same way we human learn BIB002 . But unfortunately, the majority of current models of visual attention for the robot use a prior separate training phase to enhance the recognition performance. Many of the research issues stated above have strong mutual dependency on each other. For instance, a strategy to deal with the changing reference frame (Issue 1.1) will inherently provide a solution to implement the dynamic IOR (Issue 1.2) . Again, for the sake of generality (Issue 4) if we integrate visual search and visual exploration in the same framework such that the model can switch back-and-forth between the two modes, there will be no room for prior training (Issue 5). In other words, the learning has to be performed on-line in an integrated framework of visual search and exploration. Again, if the target-learning is performed on-line, an intelligent strategy must be devised for learning to ensure that the robot obtains enough information about the target for identification in arbitrary settings (Issue 3). Addressing the Issues 1-5 is a crucial requirement to design a sound model of visual attention for the robots. In order to meet this requirement, we observe the rise of a separate group of visual attention models dedicated solely for robotic systems. There is no doubt that this new group of models are heavily inspired by the computer vision models of visual attention, specially when it comes to the detail of visual feature processing, but they attempt to address at least some of the research issues stated above.