Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Illumination Invariance <s> A slit trenching and cable laying device includes: a frame held above the ground by a support, preferably adjustable to various levels above the ground, a plow for digging a slit trench and laying a cable or line therein, the plow being mounted so as to be relatively movable with respect to the frame, a mechanism for imparting a digging motion to the plow, preferably, adjustable to impart a variable motion and being adapted to be driven by a vehicle towing the device, and another mechanism for imparting a motion to the plow for reducing surface contact and resultant drag between the plow and the sides of the trench, preferably adjustable for varying the magnitude of the motion as conditions warrant. <s> BIB001 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Illumination Invariance <s> If color appearance is to be a useful feature in identifying an object, then color appearance must remain roughly constant when the object is viewed in different contexts. People maintain approximate color constancy despite variation in the color of nearby objects and despite variation in the spectral power distribution of the ambient light. Land’s retinex algorithm is a model of human color constancy. We analyze the retinex algorithm and discuss its general properties. We show that the algorithm is too sensitive to changes in the color of nearby objects to serve as an adequate model of human color constancy. <s> BIB002 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Illumination Invariance <s> Human and machine visual sensing is enhanced when surface properties of objects in scenes, including color, can be reliably estimated despite changes in the ambient lighting conditions. We describe a computational method for estimating surface spectral reflectance when the spectral power distribution of the ambient light is not known. <s> BIB003 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Illumination Invariance <s> Simple constraints on the sets of possible surface reflectance and illuminants are exploited in a new color constancy algorithm that builds upon Forsyth's (1990) theory of color constancy. Forsyth's method invokes the constraint that the surface colors under a canonical illuminant all fall within an established maximal convex gamut of possible colors. However, the method works only when restrictive conditions are imposed on the world: the illumination must be uniform, the surfaces must be planar, and there can be no specularities. To overcome these restrictions, we modify Forsyth's algorithm so that it works with the colors under a perspective projection (in a chromaticity space). The new algorithm working in perspective is simpler than Forsyth's method and more importantly the restrictions on the illuminant, surface shape and specularities can be relaxed. The algorithm is then extended to include a maximal gamut constraint on a set of illuminants that is analogous to the gamut constraint on surface colors. Tests on real images show that the algorithm provides good color constancy. <s> BIB004 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Illumination Invariance <s> The color constancy problem, that is, estimating the color of the scene illuminant from a set of image data recorded under an unknown light, is an important problem in computer vision and digital photography. The gamut mapping approach to color constancy is, to date, one of the most successful solutions to this problem. In this algorithm the set of mappings taking the image colors recorded under an unknown illuminant to the gamut of all colors observed under a standard illuminant is characterized. Then, at a second stage, a single mapping is selected from this feasible set. In the first version of this algorithm Forsyth (1990) mapped sensor values recorded under one illuminant to those recorded under a second, using a three-dimensional (3-D) diagonal matrix. However because the intensity of the scene illuminant cannot be recovered Finlayson (see IEEE Trans. Pattern Anal. Machine Intell. vol.18, no.10, p.1034-38, 1996) modified Forsyth's algorithm to work in a two-dimensional (2-D) chromaticity space and set out to recover only 2-D chromaticity mappings. While the chromaticity mapping overcomes the intensity problem it is not clear that something has not been lost in the process. The first result of this paper is to show that only intensity information is lost. Formally, we prove that the feasible set calculated by Forsyth's original algorithm, projected into 2-D, is the same as the feasible set calculated by the 2-D algorithm. Thus, there is no advantage in using the 3-D algorithm and we can use the simpler, 2-D version of the algorithm to characterize the set of feasible illuminants. Another problem with the chromaticity mapping is that it is perspective in nature and so chromaticities and chromaticity maps are perspectively distorted. Previous work demonstrated that the effects of perspective distortion were serious for the 2-D algorithm. Indeed, in order to select a sensible single mapping from the feasible set this set must first be mapped back up to 3-D. We extend this work to the case where a constraint on the possible color of the illuminant is factored into the gamut mapping algorithm. We show here that the illumination constraint can be enforced during selection without explicitly intersecting the two constraint sets. In the final part of this paper we reappraise the selection task. Gamut mapping returns the set of feasible illuminant maps. Our new algorithm is tested using real and synthetic images. The results of these tests show that the algorithm presented delivers excellent color constancy. <s> BIB005 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Illumination Invariance <s> The paper considers the problem of illuminant estimation: how, given an image of a scene, recorded under an unknown light, we can recover an estimate of that light. Obtaining such an estimate is a central part of solving the color constancy problem. Thus, the work presented will have applications in fields such as color-based object recognition and digital photography. Rather than attempting to recover a single estimate of the illuminant, we instead set out to recover a measure of the likelihood that each of a set of possible illuminants was the scene illuminant. We begin by determining which image colors can occur (and how these colors are distributed) under each of a set of possible lights. We discuss how, for a given camera, we can obtain this knowledge. We then correlate this information with the colors in a particular image to obtain a measure of the likelihood that each of the possible lights was the scene illuminant. Finally, we use this likelihood information to choose a single light as an estimate of the scene illuminant. Computation is expressed and performed in a generic correlation framework which we develop. We propose a new probabilistic instantiation of this correlation framework and show that it delivers very good color constancy on both synthetic and real images. We further show that the proposed framework is rich enough to allow many existing algorithms to be expressed within it: the gray-world and gamut-mapping algorithms are presented in this framework and we also explore the relationship of these algorithms to other probabilistic and neural network approaches to color constancy. <s> BIB006 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Illumination Invariance <s> Mobile robot localization is the problem of determining a robot’s pose from sensor data. This article presents a family of probabilistic localization algorithms known as Monte Carlo Localization (MCL). MCL algorithms represent a robot’s belief by a set of weighted hypotheses (samples), which approximate the posterior under a common Bayesian formulation of the localization problem. Building on the basic MCL algorithm, this article develops a more robust algorithm called MixtureMCL, which integrates two complimentary ways of generating samples in the estimation. To apply this algorithm to mobile robots equipped with range finders, a kernel density tree is learned that permits fast sampling. Systematic empirical results illustrate the robustness and computational efficiency of the approach.  2001 Published by Elsevier Science B.V. <s> BIB007 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Illumination Invariance <s> This paper reports on a reactive docking behavior which uses a vision algorithm that grows linearly with the number of image pixels. The docking robot imprints (initializes) on a two-colored docking fiducial upon departing from the dock, then uses region statistics to adapt the color segmentation in changing lighting conditions. The docking behavior was implemented on a marsupial team of robots, where a daughter micro-rover had to reenter the mother robot from an approach zone with a 2 m radius and 140/spl deg/ angular width with a tolerance of /spl plusmn/5 and /spl plusmn/2 cm. Testing during outdoor conditions (noon, dusk) and challenging indoor scenarios (flashing lights) showed that using adaptation and imprinting was more robust than using imprinting atone. <s> BIB008 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Illumination Invariance <s> Robots typically have many sensors, which are underutilized. This is usually because no simple mathematical models of the sensors have been developed or the sensors are too noisy to use techniques, which require simple noise models. We propose to use these underutilized sensors to determine the state of the environment in which the robot is operating. Being able to identify the state of the environment allows the robot to adapt to current operating conditions and the actions of other agents. Adapting to current operating conditions makes robot robust to changes in the environment by constantly adapting to the current conditions. This is useful for adapting to different lighting conditions or different flooring conditions amongst many other possible desirable adaptations. The strategy we propose for utilizing these sensors is to group sensor readings into statistical probability distributions and then compare the probability distributions to detect repeated states of the environment. <s> BIB009 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Illumination Invariance <s> This thesis uses a physical, intrinsic reflection model to analyze the effects of shading and highlights on dielectric, non-uniform materials, such as plastics and paints. Since black-and-white images do not provide sufficient information to interpret these effects appropriately, this work uses color images. The colors seen on a dielectric object are an additive mixture of the object color and the illumination color, i.e.: the colors are a linear combination of those two color vectors. They therefore lie in a plane in the color space. Within the plane, they form a cluster that looks like a skewed T, due to the geometric properties of shading and highlight reflection. ::: When combined with a camera model, this description of physically possible color changes on one object can be used to guide an algorithm in interpreting color images. It is the basis of an algorithm that analyzes color variation in real color images. The algorithm looks for characteristic color clusters in the image and relates their shape to hypotheses on the object color and the amount of shading, as well as the position, strength and color of highlights on the object. It verifies the hypotheses by reapplying them back to the image, testing whether and how they apply to the pixels in the local image area from which the respective hypothesis was derived. In this way, it adapts the image interpretation process to local scene characteristics and reacts differently to color and intensity changes at different places in the image. ::: The result is an image segmentation that ignores color changes due to shading and highlights, as well as a set of intrinsic images, one describing the amount of shading at every pixel in the image, and the other one showing the amount highlight reflection at every pixel. Such intrinsic images can be a useful tool for other areas of computer vision that are currently disturbed by highlights in images, such as stereo vision and motion analysis. The approach can also be used to determine the illumination color from the highlights and as a preprocessor for methods to determine object shape from shading or highlights. This line of research may lead to physics-based image understanding methods that are both more reliable and more useful than traditional methods. (Abstract shortened with permission of author.) <s> BIB010 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Illumination Invariance <s> In this article we introduce a hierarchical Bayesian model to estimate a set of colors with a mobile robot. Estimating colors is particularly important if objects in an environment can only be distinguished by their color. Since the appearance of colors can change due to variations in the lighting condition, a robot needs to adapt its color model to such changes. We propose a two level Gaussian model in which the lighting conditions are estimated at the upper level using a switching Kalman filter. A hierarchical Bayesian technique learns Gaussian priors from data collected in other environments. Furthermore, since estimation of the color model depends on knowledge of the robot's location, we employ a Rao-Blackwellised particle filter to maintain a joint posterior over robot positions and lighting conditions. We evaluate the technique in the context of the RoboCup AIBO league, where a legged AIBO robot has to localize itself in an environment similar to a soccer field. Our experiments show that the robot can localize under different lighting conditions and adapt to changes in the lighting condition, for example, due to a light being turned on or off. <s> BIB011 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Illumination Invariance <s> In this paper we propose an approach to color classification and image segmentation in non-stationary environments. Our goal is to cope with changing illumination condition by on-line adapting both the parametric color model and its structure/complexity. Other authors used parametric statistics to model color distribution in segmentation and tracking problems, but with a fixed complexity model. Our approach is able to on-line adapt also the complexity of the model, to cope with large variations in the scene illumination and color temperature. <s> BIB012 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Illumination Invariance <s> This article describes the robot Stanley, which won the 2005 DARPA Grand Challenge. Stanley was developed for high-speed desert driving without human intervention. The robot’s software system relied predominately on state-of-the-art AI technologies, such as machine learning and probabilistic reasoning. This article describes the major components of this architecture, and discusses the results of the Grand Challenge race. <s> BIB013 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Illumination Invariance <s> Mobile robots are increasingly finding application in fields as diverse as medicine, surveillance and navigation. In order to operate in the real world, robots are primarily dependent on sensory information but the ability to accurately sense the real world is still missing. Though visual input in the form of color images from a camera is a rich source of information for mobile robots, until recently most people have focussed their attention on other sensors such as laser, sonar and tactile sensors. There are several reasons for this reliance on other relatively low-bandwidth sensors. Most sophisticated vision algorithms require substantial computational (and memory) resources and assume a stationary or slow moving camera, while many mobile robot systems and embedded systems are characterized by rapid camera motion and real-time operation within constrained computational resources. In addition, color cameras require time-consuming manual color calibration, which is sensitive to illumination changes, while mobile robots typically need to be deployed in a short period of time and often go into places with changing illumination. ::: It is commonly asserted that in order to achieve autonomous behavior, an agent must learn to deal with unexpected environmental conditions. However, for true extended autonomy, an agent must be able to recognize when to abandon its current model in favor of learning a new one, how to learn in its current situation, and also what features or representation to learn. This thesis is a fully implemented example of such autonomy in the context of color learning and segmentation, which primarily leverages the fact that many mobile robot applications involve a structured environment consisting of objects of unique shape(s) and color(s) - information that can be exploited to overcome the challenges mentioned above. The main contributions of this thesis are as follows. ::: First, the thesis presents a hybrid color representation that enables color learning both within constrained lab settings and in un-engineered indoor corridors, i.e. it enables the robot to decide what to learn. The second main contribution of the thesis is to enable a mobile robot to exploit the known structure of its environment to significantly reduce human involvement in the color calibration process. The known positions, shapes and color labels of the objects of interest are used by the robot to autonomously plan an action sequence to facilitate learning, i.e. it decides how to learn. The third main contribution is a novel representation for illumination, which enables the robot to detect and adapt smoothly to a range of illumination changes, without any prior knowledge of the different illuminations, i.e. the robot figures out when to learn. Fourth, as a means of testing the proposed algorithms, the thesis provides a real-time mobile robot vision system, which performs color segmentation, object recognition and line detection in the presence of rapid camera motion. In addition, a practical comparison is performed of the color spaces for robot vision – YCbCr, RGB and LAB are considered. The baseline system initially requires manual color calibration and constant illumination, but with the proposed innovations, it provides a self-contained mobile robot vision system that enables a robot to exploit the inherent structure and plan a motion sequence for learning the desired colors, and to detect and adapt to illumination changes, with minimal human supervision. ::: Keywords. Autonomous Color Learning, Illumination Invariance, Realtime Vision, Legged robots. <s> BIB014 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Illumination Invariance <s> A central goal of robotics and AI is to be able to deploy an agent to act autonomously in the real world over an extended period of time. To operate in the real world, autonomous robots rely on sensory information. Despite the potential richness of visual information from on-board cameras, many mobile robots continue to rely on non-visual sensors such as tactile sensors, sonar, and laser. This preference for relatively low-fidelity sensors can be attributed to, among other things, the characteristic requirement of real-time operation under limited computational resources. Illumination changes pose another big challenge. For true extended autonomy, an agent must be able to recognize for itself when to abandon its current model in favor of learning a new one; and how to learn in its current situation. We describe a self-contained vision system that works on-board a vision-based autonomous robot under varying illumination conditions. First, we present a baseline system capable of color segmentation and object recognition within the computational and memory constraints of the robot. This relies on manually labeled data and operates under constant and reasonably uniform illumination conditions. We then relax these limitations by introducing algorithms for (i) Autonomous planned color learning, where the robot uses the knowledge of its environment (position, size and shape of objects) to automatically generate a suitable motion sequence and learn the desired colors, and (ii) Illumination change detection and adaptation, where the robot recognizes for itself when the illumination conditions have changed sufficiently to warrant revising its knowledge of colors. Our algorithms are fully implemented and tested on the Sony ERS-7 Aibo robots. <s> BIB015 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Illumination Invariance <s> A major challenge in the path of widespread use of mobile robots is the ability to function autonomously, learning useful features from the environment and using them to adapt to environmental changes. We propose an algorithm for mobile robots equipped with color cameras that allows for smooth operation under illumination changes. The robot uses image statistics and the environmental structure to autonomously detect and adapt to both major and minor illumination changes. Furthermore, the robot autonomously plans an action sequence that maximizes color learning opportunities while minimizing localization errors. Our approach is fully implemented and tested on the Sony AIBO robots. <s> BIB016 </s> Color Learning and Illumination Invariance on Mobile Robots: A Survey <s> Illumination Invariance <s> The Scale-Invariant Feature Transform (or SIFT) algorithm is a highly robust method to extract and consequently match distinctive invariant features from images. These features can then be used to reliably match objects in diering images. The algorithm was rst proposed by Lowe [12] and further developed to increase performance resulting in the classic paper [13] that served as foundation for SIFT which has played an important role in robotic and machine vision in the past decade. <s> BIB017
Irrespective of the method used for color segmentation or color learning, one common problem is the sensitivity to illumination. Mobile robot application environments are characterized by illumination changes and it is essential for the robot to be able to map the image pixels to the correct color labels, i.e. we need color constancy. This section surveys some popular color constancy techniques proposed over the last few decades. The following equation illustrates the factors that influence color perception BIB006 BIB003 : where E(λ) is the illumination, S x (λ) is the surface reflectance at a 3D scene point x that projects to the image pixel p, while R j (λ) is the channel sensitivity of the imaging device's j th sensor. The sensor value of the j th sensor of the imaging device at pixel p, m p j , is the integral of the product of these three terms over the range of wavelengths of light. The robot needs to perceive the same color (l) of the object under different illuminations. Changing either the surface reflectance or the illumination can modify the sensor values, assuming the imaging device and hence the channel sensitivities remain unchanged. Figure 7 presents an example of the effect of illumination change on a trained color map. Most color constancy methods focus on static images, are computationally expensive, and predominantly estimate the current illumination E(λ) and surface reflectance S(λ) as a function of a known illumination. The Retinex theory BIB001 was one of the first attempts to explain human color constancy. Based on the assumption that white reflection induces maximal rgb camera responses (since light incident on a white patch is spectrally unchanged after reflection), Land suggested that measuring the maximum r, g, and b responses can serve as an estimate of the scene illuminant color. When it was determined that the maximum rgb in an image is not the correct estimate for white, the technique was modified to be based on global or local image color averages. The "Gray World" algorithm by Buchsbaum [11] is also based on the same principle. However the image average, either local or global, has been shown to correlate poorly with the actual scene illuminant BIB002 . In addition, this method excludes the possibility of distinguishing between the actual changes in illumination and those as a result of a change in the collection of surfaces in the environment. However, the environmental objects change dynamically in a mobile robot application domain. Another classic approach to color constancy was proposed by Maloney and Wandell BIB003 . They try to recover the surface reflectance functions of objects in the scene with incomplete knowledge of the spectral distribution of the illumination. They provide a mathematical analysis of color constancy and decompose both the illumination and the surface reflectance functions as a weighted linear combination of a small set of basis functions: leading to a relationship between sensor responses and surface reflectances: where Λ ǫ is a p (dimensions of m p , the sensor vector at a pixel) by n matrix whose (k,j)th entry corresponds to E(λ)S j (λ)R k (λ). They show that a unique solution is possible iff the number of sensory channels, three in a color camera, is at least one more than the number of basis functions that determine the surface reflectances. Assuming that the illumination is constant over an image region, they use the pixel values to estimate the illumination and the surface reflectance. The paper also presents some interesting conclusions about human color perception. However, neither surface reflectances nor illuminations can be modeled well with just two basis functions. In addition, the illumination does not have to remain constant over large image regions. Forsyth proposed the gamut mapping algorithm for color constancy  , which estimates the current illumination as a function of a known illumination. Since surfaces can reflect no more light than is cast on them, he concluded that the illuminant color is constrained by the colors observed in the image and can hence be estimated using image measurements alone. The algorithm generates a set of mappings that transform image colors (sensor values) under an unknown illuminant to the gamut of colors observed under a standard (canonical) illuminant, using 3D diagonal matrices. Then a single mapping is chosen from the feasible set of mappings in order to estimate the actual sensor values under the canonical illuminant. Though the method requires advance knowledge of the possible illuminants to generate the mappings, it emphasizes the point that image statistics can provide a good estimate of the illumination. Realizing that the scene illuminant intensity cannot be recovered in Forsyth's approach, Finlayson modified it to work in the 2D chromaticity space BIB004 . He proved that the feasible set calculated by his algorithm was the same as that calculated by Forsyth's original algorithm when projected into 2D BIB005 . He then proposed the median selection method, which includes a constraint on the possible color of the illuminant in the gamut mapping algorithm. More recently he proposed a correlation framework BIB006 , where the range of colors that can occur under each of a possible set of illuminants is computed in advance. Correlating this range with the colors in a test image estimates the likelihood that each of a possible set of illuminants is the scene illuminant. Instead of estimating the illumination intensity, Klinker et al. BIB010 focus on the surface reflectance properties of objects in the scene. They propose an approach for color image understanding based on the Dichromatic Reflection Model (DRM). The color of reflected light is described as a mixture of the light from the surface reflection (highlights) and body reflection (actual object color). DRM can be used to separate a color image into an image based on just the highlights, and the original image with the highlights removed. They present a color image understanding system that can describe the reflection processes occurring in the scene. Their work requires prior knowledge of the possible illuminations and properties of objects in the scene, while robots are typically deployed in environments where it is not feasible to measure these properties in advance. However, their technique provides a better understanding of how the illumination and surface reflectances interact to produce the sensor responses i.e. the camera image. In contrast to the classical approaches to color constancy, several Bayesian approaches have also been proposed. Brainard and Freeman [7] present a Bayesian decision theoretic framework which combines all available statistics such as gray world, subspace and physical realizability constraints. They model the relation among illuminations, surfaces and photosensor responses and generate a priori distributions to describe the probability of existence of certain illuminations and surfaces. They present a local mass loss function: wherex is the estimate of the parameter vector x, and |K The matrix K L is chosen with sufficiently small eigenvalues so that the local mass loss function rewards approximately correct estimates and assign an almost equal penalty for grossly incorrect estimates. The maximum local mass (MLM) estimator is obtained by minimizing the expected loss for the local mass loss function. The MLM estimator integrates local probabilities and uses Bayes' rule to compute the posterior distributions for surfaces and illuminants for a given set of photosensor responses. However, their approach assumes a priori knowledge of the various illuminations and surfaces, which is not feasible to obtain in many robot application domains. Along similar lines, Tsin et al  present a Bayesian MAP (maximum a posteriori ) approach to achieve color constancy for outdoor object recognition with a static surveillance camera. Static overhead high-definition color images are collected over several days, and are used to learn statistical distributions for surface reflectance and the illumination. Then a linear iterative updating scheme is used to converge to the classification of the illumination and hence the objects in the test images. Most mobile robots operate in a dynamically changing world with motion induced image distortion and a limited view of its surroundings. Though collecting statistics as proposed by their approach may not be feasible in most robot applications, defining the space of operation based on the task is applicable to robot domains as well. In contrast to the Bayesian methods, Rosenberg et al.  develop models for sensor noise, canonical (i.e. "true") color and illumination. The pixel values are considered to be proportional to the sensor values under the canonical illuminant, and are transformed to a log-space, normalized and discretized to obtain a 2D color space that factors out the pixel intensities. Based on the assumption that the global scene illuminant parameters are independent of the canonical color of an object, a single distribution is maintained over the canonical colors and used to compute the a priori likelihood of observing a particular color measurement given a set of illumination parameters. Given an input image under an unknown illumination within the set of illuminations the system is trained for, the a posteriori global scene illumination parameters are determined by an exhaustive search over the space defined by the learned models, using the KL-divergence measure. Experimental results show that the performance is better than that obtained by using a Maximum Likelihood Estimate approach. This approach assumes a priori knowledge of the illuminations and sensor noise, and involves a computationally expensive search process. Mobile robots however frequently move into regions with previously unknown illuminations and sensor noise. In the domain of mobile robots, the problem of color constancy has often been avoided by using non-visual sensors such as laser range finders and sonar BIB007 . Even when visual input is considered, the focus has been on recognizing just a few well-separated colors  BIB008 or to use intensity-based features BIB017 . Until recently, there has been relatively little work on illumination invariance with a moving camera in the presence of shadows and artifacts caused by rapid motion in real-world domains. Furthermore, even the few approaches that do function within the computational constraints of mobile robots (e.g. BIB012 BIB009 BIB013 ) do not autonomously model illuminations and overlapping colors to smoothly detect and adapt to a range of illuminations. Lenser and Veloso BIB009  present a tree-based state description and identification technique which they use for detecting illumination changes on Aibos (four-legged robots). They incorporate a time-series of average screen illuminance to distinguish between illuminations, using the absolute value distance metric to determine the similarity between distributions. Prior work has shown that average scene illuminant is not a good representation for illumination BIB002 . In addition, their method does not run in real-time on-board a robot. However, their work provides additional support to the hypothesis that temporal image statistics can be used to model illuminations -the goal is to enable mobile robots to collect these statistics autonomously. Since both sensing and actuation on mobile robots involve uncertainty, probabilistic (i.e. Bayesian) methods are becoming popular for mobile robot tasks. For instance, Schulz and Fox BIB011 estimate colors using a hierarchical Bayesian model with Gaussian priors and a joint posterior on position and environmental illumination. In their approach, the mean color vector obtained from an image captured at a known location is treated as a direct observation of the current illumination. The posterior over illuminations at time k, denoted by θ k is modeled as a Gaussian: p(θ k |ī 1:k ) = N (θ k ; µ k , Σ k ), whereī k is the mean color vector and the Gaussian parameters (mean, covariance) are estimated using a standard Kalman filter update [97] . The joint posterior over robot poses and illuminations is factorized: where z 1:k are input images up to time k and u 1:k−1 are the robot motion commands. This decomposition allows the use of Rao-Blackwellised Particle Filters [97] such that illumination (θ k ) can be determined based on the estimated robot pose. The method however requires significant prior information to construct the a priori Gaussian density functions for color distributions over known illuminations -several images are hand labeled. The results reported in the paper indicate that the algorithm requires significant human supervision for experiments with just two distinct illuminations and a small set of colors. The Bayesian framework can however be exploited if the models for color distributions and illuminations are learned autonomously. Anzani et al. BIB012 describe another Bayesian approach for illumination invariance on wheeled robots. They use a Mixture of Gaussians (MoG) to model the distribution of each color (Equation 4). The labeling of color classes and association with mixture components is done by human supervision, and the Bayesian decision rule is used during classification to determine the color label. In order to adapt the model parameters and number of mixture components to changing illuminations, the EM algorithm  is used. Their algorithm has been tested only over a few intensity variations in the lab and involves iterative computation for parameter estimation. In addition to the computationally expensive iterative process, color distributions are not always well-modeled with MoG and it may be infeasible to obtain a large set of training samples in robot domains -incremental learning may be necessary. The recent DARPA challenges have resulted in several approaches to illumination invariance on real-world mobile robot domains [21,  BIB013 101, 22] . Though the visual information was only used sparingly in these challenges, Thrun et al. BIB013 model colors as MoG (Equation 4 in Section 4) and add additional Gaussians or modify the parameters of the existing Gaussians in response to illumination changes. As mentioned above, not all colors are modeled well using MoG given limited training data. Furthermore, they were interested only in distinguishing safe regions on the ground from the unsafe regions and did not have to model colors whose distributions overlap in the color space. In their work on color learning on legged robots, Sridharan and Stone BIB014 BIB015 enable the robot to model illuminations using autonomously collected image statistics. While the robot is building models for the color distributions (Equation 5 in Section 4), it also collects sample environmental images to create histograms in the color space (normalized RGB). In addition, the robot computes the distance between every pair of these histograms to arrive at a distribution of distances between color histograms under a given illumination. The histograms and the distribution of distances characterize a particular illumination. Periodically, the robot captures images and computes the corresponding color space histogram. If the average distance between this histogram and the stored histograms corresponding to the current illumination is significantly different from the expected range of distances under the current illumination, a significant change in illumination is detected. The robot adapts to large illumination changes by re-learning the appropriate portions of the color map. Furthermore, the robot detects minor illumination changes by monitoring the color segmentation performance over known scene objects, and adapts by updating the color distributions in real-time using Gaussian and histogram-merging techniques. The combined strategy enables the robot to operate smoothly in a range of illuminations BIB016 . Figure 8 shows sample images as the robot tracks illumination changes. Table 3 Illumination invariance methods -adaptation to new conditions is very rare. Table 3 evaluates the illumination invariance algorithms based on the criteria in Section 2. Computer vision approaches are computationally expensive and/or require extensive prior knowledge of the possible illuminations and the scene objects. Algorithms implemented for mobile robots only tackle a small subset of the problem. Methods that can exploit domain knowledge to autonomously detect and adapt color models to new illuminations are very rare BIB016 , and even these schemes require some domain knowledge to be specified in advance (the "(Yes)" in the final row). As in the case of color modeling, providing minimal domain knowledge to enable adaptation to illumination changes is significantly easier than measuring all the properties of the environment. Illumination invariance is hence still an open problem, both on mobile robots and in computer vision in general.