To enable intelligent man-machine interaction, attention control and object recognition are widely recognized as important issues. Due to the difficulty of scene segmentation and object recognition in real-world scenes, most work in this area has concentrated on explicitly or implicitly constrained scenarios, e.g. uncluttered background, homogenous coloring of foreground objects, or predefined objects classes. It remains difficult, however, to bridge the gap between the low level perceptual cues and the symbolic levels of object representations.
Current approaches for object learning are based on probabilistic and Bayesian methods (Krishnapuram B., C. M. Bishop, and M. Szummer, Generative models and Bayesian model comparison for shape recognition, Proceedings Ninth International Workshop on Frontiers in Handwriting Recognition, 2004, which is incorporated by reference herein in its entirety. J. Winn and N. Joijic, Locus: Learning object classes with unsupervised segmentation, Intl. Conf. on Computer Vision, 2005 which is incorporated by reference herein in its entirety. These demonstrate learning prototypic object categories together with their varying shape from natural images, but their methods are computationally extremely demanding and are not suitable for online and interactive learning.
To facilitate visual processing and to reduce search spaces, cognitive vision systems can use attention based vision control to generate fixations. On the lower level, attention control can be based on topographically ordered maps to focus the system resources to certain points of interest. For example in Joseph A. Driscoll, Richard Alan Peters II, and Kyle R. Cave, A visual attention network for a humanoid robot, Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS-98), Victoria, B.C., Oct. 12-16 1998, which is incorporated by reference herein in its entirety. These maps primarily use simple stimuli like color, oriented edges, or intensity, although mechanisms to integrate higher level information have also been proposed, for example, in J. J. Steil, G. Heidemann, J. Jockusch, R. Rae, N. Jungclaus, and H. Ritter, Guiding attention for grasping tasks by gestural instruction: The gravis-robot architecture, Proc. IROS 2001, pages 1570-1577. IEEE, 2001, which is incorporated by reference herein in its entirety. One approach to reach the semantic level is to search for known objects at the current fixation point with a holistic object classification system (for example in J. J. Steil and H. Ritter, Learning issues in a multi-modal robot-instruction scenario, IEEE Int. Conf. Robotics, Intelligent Systems and Signal Processing, 2003, which is incorporated by reference herein in its entirety) and to store objects recognized in a symbolic memory (for example in G. Heidemann, A multi-purpose visual classification system, B. Reusch, Editor, Proc. 7th Fuzzy Days, Dortmund, 2001, pages 305-312, Springer-Verlag, 2001; and in G. Heidemann and H. Ritter, Combining multiple neural nets for visual feature selection and classification, Proceedings of ICANN 99, 1999, which are incorporated by reference herein in their entirety). Due to the need for a large amount of training images from different views, the object classification itself has to be trained offline beforehand.
It is generally believed that segmentation and recognition are closely connected and some authors try to solve both approaches concurrently (see, for example, Stella X. Yu, Ralph Gross, and Jianbo Shi, Concurrent object recognition and segmentation by graph partitioning, Online proceedings of the Neural Information Processing Systems conference, 2002, which is incorporated by reference herein in its entirety), which results in rather complex architectures without online capabilities. In more classical approaches, segmentation is treated as an independent preprocessing step towards recognition. However, in such learning contexts it is crucial to use unsupervised segmentation, because a priori knowledge about the object to segment is not available.
To enable unsupervised segmentation, several cluster based segmentation approaches use different color spaces and sometimes the pixel coordinates as feature space. Such approaches are found in: Guo Dong and Ming Xie, Color clustering and learning for image segmentation based on neural networks, IEEE Transactions on Neural Networks, 16(14):925-936, 2005; and Y. Jiang and Z.-H. Zhou, Some ensemble-based image segmentation, Neural Processing Letters, 20(3):171-178, 2004, which are incorporated by reference herein in their entirety. They apply a vector quantization method like k-means or self organizing maps (SOM) to partition this space and segment the image with respect to the codebook vectors. Similarly, some approaches index the colors, quantize this index space, and back project this quantization to segments. For example in Jung Kim Robert Li, Image compression using fast transformed vector quantization, Applied Imagery Pattern Recognition Workshop, page 141, 2000; and Dorin Comaniciu and Richard Grisel, Image coding using transform vector quantization with training set synthesis, Signal Process., 82(11): 1649-1663, 2002, which are incorporated by reference herein in their entirety. Though such quantization methods can potentially be fast, they assume that objects have to be homogeneously colored and can be covered by one segment. If stereo images are available, disparity information can be used as segmentation cue (see N. H. Kim and Jai Song Park, Segmentation of object regions using depth information, ICIP, pages 231-234, 2004 which is incorporated by reference herein in its entirety) and some approaches try to support unreliable disparity information by additional color segmentation (see, Hai Tao and Harpreet S. Sawhney, Global matching criterion and color segmentation based stereo, Workshop on the Application of Computer Vision, pages 246-253, 2000 which is incorporated by reference herein in its entirety). In these schemes color segmentation is not learned and uses strong underlying homogeneity assumptions. Implicitly it is also assumed in these approaches that the objects to segment are isolated from each other, which in real scenarios often not the case, in particular not if humans manipulate and present objects to be learned to the machine.
Some approaches have been made to combine unsupervised color clustering methods with top down information about the object derived from other sources (see E. Borenstein, E. Sharon, and S. Ullman, Combining top-down and bottom-up segmentation, 2004 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'04), 4:46, 2004; and M. J. Bravo and H. Farid, Object segmentation by top-down processes, Visual Cognition, 10(4):471-491, 2003 which are incorporated by reference herein in their entirety). This approach has the advantage that in the unsupervised step smaller segments can be generated which may over-segment the objects. Thus homogeneity assumptions can be relaxed, however, the top down information must be sufficient to resolve the resulting ambiguities.
In Borenstein (cited above) therefore, the unsupervised step consists of generating a hierarchy of segments ordered in a tree and a successive optimization procedure to label the segments as belonging to the object with respect to a cost function based on the top-level information.
The complexity of this method is linear in the number of pixels, but still not sufficiently fast to allow real-time performance processing with several frames per second.