Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Summary and Future Work <s> This paper reports on a reactive docking behavior which uses a vision algorithm that grows linearly with the number of image pixels. The docking robot imprints (initializes) on a two-colored docking fiducial upon departing from the dock, then uses region statistics to adapt the color segmentation in changing lighting conditions. The docking behavior was implemented on a marsupial team of robots, where a daughter micro-rover had to reenter the mother robot from an approach zone with a 2 m radius and 140/spl deg/ angular width with a tolerance of /spl plusmn/5 and /spl plusmn/2 cm. Testing during outdoor conditions (noon, dusk) and challenging indoor scenarios (flashing lights) showed that using adaptation and imprinting was more robust than using imprinting atone. <s> BIB001 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Summary and Future Work <s> Abstract This paper describes a mobile robotic assistant, developed to assist elderly individuals with mild cognitive and physical impairments, as well as support nurses in their daily activities. We present three software modules relevant to ensure successful human–robot interaction: an automated reminder system; a people tracking and detection system; and finally a high-level robot controller that performs planning under uncertainty by incorporating knowledge from low-level modules, and selecting appropriate courses of actions. During the course of experiments conducted in an assisted living facility, the robot successfully demonstrated that it could autonomously provide reminders and guidance for elderly residents. <s> BIB002 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Summary and Future Work <s> We present a method to learn and recognize object class models from unlabeled and unsegmented cluttered scenes in a scale invariant manner. Objects are modeled as flexible constellations of parts. A probabilistic representation is used for all aspects of the object: shape, appearance, occlusion and relative scale. An entropy-based feature detector is used to select regions and their scale within the image. In learning the parameters of the scale-invariant object model are estimated. This is done using expectation-maximization in a maximum-likelihood setting. In recognition, this model is used in a Bayesian manner to classify images. The flexible nature of the model is demonstrated by excellent results over a range of datasets including geometrically constrained classes (e.g. faces, cars) and flexible objects (such as animals). <s> BIB003 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Summary and Future Work <s> This paper presents a vision system for robotic soccer which was tested on Sony's four legged robot Aibo. The input for the vision system are images of the camera and the sensor readings of the robot's head joints, the output are the positions of all recognized objects in relation to the robot. The object recognition is based on the colors of the objects and uses a color look-up table. The vision system creates the color look-up table on its own during a soccer game. Thus no pre-run calibration is needed and the robot can cope with inhomogeneous or changing light on the soccer field. It is shown, how different layers of color representation can be used to refine the results of color classification. However, the self-calibrated color look-up table is not as accurate as a hand-made. Together with the introduced object recognition which is very robust relating to the quality of the color table, the self-calibrating vision works very well. This robustness is achieved using the detection of edges on scan lines. <s> BIB004 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Summary and Future Work <s> In this article we introduce a hierarchical Bayesian model to estimate a set of colors with a mobile robot. Estimating colors is particularly important if objects in an environment can only be distinguished by their color. Since the appearance of colors can change due to variations in the lighting condition, a robot needs to adapt its color model to such changes. We propose a two level Gaussian model in which the lighting conditions are estimated at the upper level using a switching Kalman filter. A hierarchical Bayesian technique learns Gaussian priors from data collected in other environments. Furthermore, since estimation of the color model depends on knowledge of the robot's location, we employ a Rao-Blackwellised particle filter to maintain a joint posterior over robot positions and lighting conditions. We evaluate the technique in the context of the RoboCup AIBO league, where a legged AIBO robot has to localize itself in an environment similar to a soccer field. Our experiments show that the robot can localize under different lighting conditions and adapt to changes in the lighting condition, for example, due to a light being turned on or off. <s> BIB005 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Summary and Future Work <s> We present a general-purpose framework for updating a robot’s observation model within the context of planning and execution. Traditional plan execution relies on monitoring plan step transitions through accurate state observations obtained from sensory data. In order to gather meaningful state data from sensors, tedious and time-consuming calibration methods are often required. To address this problem we introduce Reverse Monitoring, a process of learning an observation model through the use of plans composed of scripted actions. The automatically acquired observation models allow the robot to adapt to changes in the environment and robustly execute arbitrary plans. We have fully implemented the method in our AIBO robots, and our empirical results demonstrate its effectiveness. <s> BIB006 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Summary and Future Work <s> Color segmentation is a challenging subtask in computer vision. Most popular approaches are computationally expensive, involve an extensive off-line training phase and/or rely on a stationary camera. This paper presents an approach for color learning on-board a legged robot with limited computational and memory resources. A key defining feature of the approach is that it works without any labeled training data. Rather, it trains autonomously from a color-coded model of its environment. The process is fully implemented, completely autonomous, and provides high degree of segmentation accuracy. <s> BIB007 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Summary and Future Work <s> This paper proposes a new hierarchical formulation of POMDPs for autonomous robot navigation that can be solved in real-time, and is memory efficient. It will be referred to in this paper as the Robot Navigation-Hierarchical POMDP (RN-HPOMDP). The RN-HPOMDP is utilized as a unified framework for autonomous robot navigation in dynamic environments. As such, it is used for localization, planning and local obstacle avoidance. Hence, the RN-HPOMDP decides at each time step the actions the robot should execute, without the intervention of any other external module for obstacle avoidance or localization. Our approach employs state space and action space hierarchy, and can effectively model large environments at a fine resolution. Finally, the notion of the reference POMDP is introduced. The latter holds all the information regarding motion and sensor uncertainty, which makes the proposed hierarchical structure memory efficient and enables fast learning. The RN-HPOMDP has been experimentally validated in real dynamic environments. <s> BIB008 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Summary and Future Work <s> This article describes the robot Stanley, which won the 2005 DARPA Grand Challenge. Stanley was developed for high-speed desert driving without human intervention. The robot’s software system relied predominately on state-of-the-art AI technologies, such as machine learning and probabilistic reasoning. This article describes the major components of this architecture, and discusses the results of the Grand Challenge race. <s> BIB009 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Summary and Future Work <s> Simultaneous localization and mapping (SLAM) is an important research area in robotics. Lately, systems that use a single bearing-only sensors have received significant attention and the use of visual sensors have been strongly advocated. In this paper, we present a framework for 3D bearing only SLAM using a single camera. We concentrate on image feature selection in order to achieve precise localization and thus good reconstruction in 3D. In addition, we demonstrate how these features can be managed to provide real-time performance and fast matching, to detect loop-closing situations. The proposed vision system has been combined with an extended Kalman Filter (EKF) based SLAM method. A number of experiments have been performed in indoor environments which demonstrate the validity and effectiveness of the approach. We also show how the SLAM generated map can be used for robot localization. The use of vision features which are distinguishable allows a straightforward solution to the "kidnapped-robot" scenario. <s> BIB010 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Summary and Future Work <s> The Spatial Semantic Hierarchy and its predecessor the TOUR model are theories of robot and human commonsense knowledge of large-scale space: the cognitive map. The focus of these theories is on how spatial knowledge is acquired from experience in the environment, and how it can be used effectively in spite of being incomplete and sometimes incorrect. <s> BIB011 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Summary and Future Work <s> A major challenge in the path of widespread use of mobile robots is the ability to function autonomously, learning useful features from the environment and using them to adapt to environmental changes. We propose an algorithm for mobile robots equipped with color cameras that allows for smooth operation under illumination changes. The robot uses image statistics and the environmental structure to autonomously detect and adapt to both major and minor illumination changes. Furthermore, the robot autonomously plans an action sequence that maximizes color learning opportunities while minimizing localization errors. Our approach is fully implemented and tested on the Sony AIBO robots. <s> BIB012 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Summary and Future Work <s> The paper presents an HRI architecture for human-augmented mapping, which has been implemented and tested on an autonomous mobile robotic platform. Through interaction with a human, the robot can augment its autonomously acquired metric map with qualitative information about locations and objects in the environment. The system implements various interaction strategies observed in independently performed Wizard-of-Oz studies. The paper discusses an ontology-based approach to multi-layered conceptual spatial mapping that provides a common ground for human-robot dialogue. This is achieved by combining acquired knowledge with innate conceptual commonsense knowledge in order to infer new knowledge. The architecture bridges the gap between the rich semantic representations of the meaning expressed by verbal utterances on the one hand and the robot's internal sensor-based world representation on the other. It is thus possible to establish references to spatial areas in a situated dialogue between a human and a r... <s> BIB013 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Summary and Future Work <s> Flexible general purpose robots need to tailor their visual processing to their task, on the fly. We propose a new approach to this within a planning framework, where the goal is to plan a sequence of visual operators to apply to the regions of interest (ROIs) in a scene. We pose the visual processing problem as a Partially Observable Markov Decision Process (POMDP). This requires probabilistic models of operator effects to quantitatively capture the unreliability of the processing actions, and thus reason precisely about trade-offs between plan execution time and plan reliability. Since planning in practical sized POMDPs is intractable we show how to ameliorate this intractability somewhat for our domain by defining a hierarchical POMDP. We compare the hierarchical POMDP approach with a Continual Planning (CP) approach. On a real robot visual domain, we show empirically that all the planning methods outperform naive application of all visual operators. The key result is that the POMDP methods produce more robust plans than either naive visual processing or the CP approach. In summary, we believe that visual processing problems represent a challenging and worthwhile domain for planning techniques, and that our hierarchical POMDP based approach to them opens up a promising new line of research. <s> BIB014 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Summary and Future Work <s> A major challenge to the deployment of mobile robots in a wide range of tasks is the ability to function autonomously, learning appropriate models for environmental features and adapting these models in response to environmental changes. Such autonomous operation is feasible iff the robot is able to plan an appropriate action sequence. In this paper, we focus on the task of color modeling/learning, and present two algorithms that enable a mobile robot to plan action sequences that facilitate color learning. We propose a long-term action-selection approach that maximizes color learning opportunities while minimizing localization errors over an entire action sequence, and compare it with a greedy/heuristic action-selection approach that plans incrementally, to maximize the utility based on the current state information. We show that long-term action-selection provides a more principled solution that requires minimal human supervision. All algorithms are fully implemented and tested on the Sony AIBO robots. <s> BIB015 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Summary and Future Work <s> The Scale-Invariant Feature Transform (or SIFT) algorithm is a highly robust method to extract and consequently match distinctive invariant features from images. These features can then be used to reliably match objects in diering images. The algorithm was rst proposed by Lowe [12] and further developed to increase performance resulting in the classic paper [13] that served as foundation for SIFT which has played an important role in robotic and machine vision in the past decade. <s> BIB016 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Summary and Future Work <s> Planning can often be simplified by decomposing the task into smaller tasks arranged hierarchically. Charlin et al. [4] recently showed that the hierarchy discovery problem can be framed as a non-convex optimization problem. However, the inherent computational difficulty of solving such an optimization problem makes it hard to scale to real-world problems. In another line of research, Toussaint et al. [18] developed a method to solve planning problems by maximum-likelihood estimation. In this paper, we show how the hierarchy discovery problem in partially observable domains can be tackled using a similar maximum likelihood approach. Our technique first transforms the problem into a dynamic Bayesian network through which a hierarchical structure can naturally be discovered while optimizing the policy. Experimental results demonstrate that this approach scales better than previous techniques based on non-convex optimization. <s> BIB017
Mobile robots possess immense potential in terms of their applicability to a wide range of tasks BIB001 BIB002 BIB009 since high-fidelity sensors such as color cameras are becoming available at moderate costs. The sensors however typically require frequent and extensive manual calibration in response to environmen-tal changes. For instance, though vision is a rich source of information, color cameras require extensive manual calibration that is sensitive to illumination changes. The major challenge to the widespread use of mobile robots is hence the ability to learn, adapt and operate autonomously. In this article we have looked at such learning and adaptation in the context of color-based mobile robots operating in dynamic environments. We have especially focused on surveying a set of representative approaches for color segmentation, color learning, illumination invariance and action-planning. A common observation in all the surveyed modules is that many state-of-theart computer vision and robot vision algorithms fall short of satisfying the desired characteristics that would facilitate autonomous color-based mobile robot operation: real-time operation, deployment without extensive manual training or prior information, and adaptation to previously unseen environmental conditions. The assumptions that are made while solving just the vision problems, predominantly cease to hold in the case of physical agents that interact with the real-world -this is true of several robot tasks in addition to those reviewed in this article. Instead of just modifying existing computer vision algorithms to make them work on mobile robots, robot vision and autonomous mobile robot operation in general, needs to be tackled as a separate problem by clearly identifying the feasible assumptions, limiting constraints, and the information available to the robot. It is necessary to focus more on creating complete, integrated systems that sense and interact with their surroundings, rather than looking at just subsets of the problem in isolation. The DARPA initiatives [21,  22] , the EU Cognitive Systems (CoSy) project  , and the RoboCup initiative  are all excellent examples of tasks that aim to create such integrated systems. These initiatives force the researchers to identify and tackle key problems that would not arise in the absence of such practical challenges. In the case of mobile robots operating in the dynamic environments, the availability of high-quality visual sensors at moderate costs presents an alluring alternative to range sensors that offer relatively low-bandwidth information. There are several challenges on the path of completely autonomous operation using visual features such as color because of the inherent sensitivity of visual information to environmental factors such as illumination. However, there exist factors in robot application domains that can be exploited to offset these challenges. Our work, for instance, exploits the fact that many robot environments inherently have a moderate amount of structure. The robot uses the structure to learn models for color distributions, and to detect and adapt to illumination changes. Current algorithms that learn models for color distributions based on structure (known positions, shapes and other characteristics of the objects)  BIB006 BIB004 BIB007 require the structure to be pre-specified. Existing algorithms can however be used by the robot to learn a significant portion of this structure. Algorithms for SLAM (Simultaneous Localization and Map-ping) are able to use the visual input to simultaneously localize and build a map of the robot's surroundings BIB010 . In parallel, existing computer vision methods can learn object properties (shape, geometric arrangement etc) from visual input BIB003 . One significant step towards autonomous color-based mobile robot operation would therefore be to enable the mobile robot to autonomously learn the required structure and use it to learn models for other environmental features such as color. We observe from the survey that bootstrapping off of the learned or manually specified information leads to much more autonomous and robust performance than tackling the individual problems separately. As described in Section 7, it is possible for a mobile robot to plan action sequences that facilitate the learning of models for color distributions, and to use the learned color models to monitor and modify the plan to learn models for other colors robustly. In addition, the robot is able to use the learned models to recognize and adapt to illumination changes, which in turn leads to an appropriate modification of the learned models. Incorporating such bootstrapping between planning, learning and adaptation is therefore an appealing approach to achieve efficient and robust solutions to a range of robot vision problems. The sensitivity to illumination is a major challenge to the use to color on mobile robots. As seen in Section 5, illumination invariance (i.e. color constancy) has been a major research focus for years, but the problem is still far from being solved. Instead of making unrealistic assumptions, such as being able to measure the properties of the operating environment and/or knowing all possible illuminations ahead of time, robot vision algorithms that have modeled the effects of illumination on color on-board the robot BIB005 BIB012 have led to robust performance under a range of illuminations. It would be interesting to use the experimental data obtained over years of vision research to model the effects of illuminations on the color distributions in the form of a joint color-illumination mathematical model whose parameters can then be learned autonomously by the robot. In this article we have primarily looked at color-based mobile robot vision because it is a low-dimensional feature well understood under static conditions, and is easier to analyze under dynamic conditions. However, the lessons learned here are also applicable to challenges faced when working with other visual features and other sensory inputs (e.g. range sensors). Similar algorithms that bootstrap and learn feature-models in real-time can be devised for other visual features such as texture or local gradients BIB016 . An even more challenging problem is to fuse the information from a set of sensors, each with a varying degree of unreliability, which may have complementary properties. A combination of a laser range finder and a color camera is an example of such a complementary set of sensors. Such sensor fusion requires a probabilistic representation of the performance of the sensory sources, which can be learned by the robot. The learned representations can then be used to learn how the information from the sensors can be combined to obtain a single robust representation of the current state of the world. In the DARPA challenges  BIB009 for instance, sensor fusion continues to be a major challenge. When there are multiple sources of information, multiple actions that can be performed by the robot, and multiple information-processing operators that can be applied on the visual input, it is useful for the robot to plan a sequence of actions to achieve the desired goal reliably and optimally. Optimal use of resources is important for robots that interact with the real world through noisy sensors and actuators, because they need to respond to dynamic changes. Classical planning methods  require prior knowledge of the states and action outcomes, and they do not model uncertainty effectively. Posing this planning problem as a POMDP has yielded the most promising results because POMDPs elegantly capture the features of mobile robot domains: partially observable states and non-deterministic actions. POMDPs also provide the capability to simultaneously maximize reliability and minimize resource usage. The inherent intractability of modeling practical problems as POMDPs has been partially overcome by imposing a structure or hierarchy on the problem BIB008 BIB002 BIB014 based on domain knowledge. Unlike existing work on probabilistic planning, our work in the robot soccer domain BIB015 and the human-robot interaction domain BIB014 shows the feasibility of using autonomously learned models of action outcomes to generate robust plans to achieve the desired goal. In the future even the hierarchical decomposition may be learned BIB017 , and a range of state and action hierarchies BIB002 may be used to formulate complex real-world problems as POMDPs. Furthermore, a hybrid combination of deterministic and probabilistic planning schemes may be used to analyze complex human-robot interaction scenarios. In the long term, enabling robots to operate in the real-world would require autonomous operation at a much larger scale. The robot would need capabilities such as spatial and semantic reasoning BIB011 , analysis of 3D geometric structure, and natural language processing during human-robot interactions BIB013 . As the focus shifts towards developing such "cognitive" abilities on a mobile robot, including the ability to reason and bind information across different sensing modalities, it becomes necessary to design architectures that enable autonomous learning and adaptation in complex systems  . Ultimately, the goal is to enable widespread mobile robot deployment in a range of applications, with the robots operating autonomously and interacting with humans under completely uncontrolled natural conditions.