An image recognition and classification system (a machine vision system) includes a preprocessor in which a "top-down" method is used to extract features from an image, an associative learning neural network system which groups the features into patterns and classifies the patterns, and an attentional mechanism which focuses additional preprocessing and a neural network on relevant parts of an image.
Attempts to recognize and classify images have led to construction of automated artificial machine vision systems and to development of strategies to learn patterns in images and to recognize and classify images by using the learned patterns. Those developing artificial systems have continually attempted to incorporate principles of biological systems into their strategies, because biological systems outperform all artificial systems, implemented or proposed, by a wide margin. For example, machine vision systems based on artificial neural networks have been implemented on digital parallel computers, but a parallel implementation only provides an increase in speed without an increase in performance. Thus, the goal of emulating the pattern recognition performance of biological systems still eludes computer scientists.
In order for a biological nervous system to discriminate objects two fundamental problems must be solved: object segmentation and binding. "Object segmentation" deals with distinguishing separate objects; "binding" deals with how specific attributes such as shape and depth, are linked to create an individual object. A question addressed by object segmentation mechanisms is to which overlapping object does a border belong? An image of an object may be occluded (divided) by an overlapping image, and will need to be reconstructed as a whole image. Models have been proposed to explain this process, for example, using artificial neural networks. (Sajda and Finkel, 1992)
During the past half century, the theoretical infrastructure of machine vision systems has developed both top-down (beginning with large features of the image) and bottom-up (beginning at the lowest level of resolution, usually a pixel) views. However, actual development has focused almost exclusively on bottom-up approaches as exemplified by the title of Pentland's illuminating book From Pixels to Predicates, and comments therein such as: "Processing is primarily data-driven (i.e., bottom-up), although it can be responsive to the goals and expectations at the higher levels." (Pentland, 1986, part 1, page 1).
Ongoing efforts have focused on the extraction of "features" in an image by local manipulations of small micro-features (often 3.times.3 rarely more than 9.times.9 pixel areas), with the intent of identifying larger features (macro-features) from their combination. The paucity of robust results from this approach may be attributed to several causes, two of the most important of which are (1) that the mathematical operations performed on the small areas are usually differential operators such as edge detectors that enhance rather than reduce noise; and (2) that not even humans are very good at visual recognition when allowed only a small instantaneous field of view. Similarity of tactual and visual picture recognition with limited field of view. Loomis et al. (1991).
During this same time period, cognitive psychologists and neurobiologists have made impressive advances in research on the processing mechanisms that are at work in the visual cortex of mammals, particularly cats and monkeys. Electrophysiological and psychophysical experiments on cats and monkeys demonstrate a wide variety of feature selective cells in the visual cortex. In the mammalian cortex, these include simple cells (Hubel and Weisel, 1962), whose shape is closely approximated by a Gabor function (Daugman, 1985; Jones and Palmer, 1987) or a difference of Gaussian functions; end-stopped cells (often called first order hypercomplex cells) (Hubel & Weisel, 1965; Gilbert, 1977); color sensitive cells; and even cells that respond only to faces. (Desimone, 1991). Face-selective cells in the temporal cortex of monkeys. Desimone (1991).
Complex cells and second order hypercomplex cells (Hubel and Weisel, 1962, 1965) are sensitive to the same features as simple and first order hypercomplex cells, respectively. One of the differences among these cells, of interest in the context of feature extraction from static images, is that the complex and second order hypercomplex cells have larger receptive fields than simple cells, and are insensitive to location of micro-features within their receptive fields.
In the development of artificial systems, preprocessing of data derived from an image has been used to extract features from an image and to select features for further processing by machine vision systems. Preprocessing generally proceeds in steps from the "bottom-up," although "top-down" preprocessing has been suggested as a model for human vision. Preprocessing is accomplished by preprocessors, which may be implemented in hardware or software. In some systems, preprocessors have served as the first layer of a two layered neural network. Preprocessing strategies have included subdividing a whole image to be processed into sub-images. Various filters have been suggested to operate on the data, converting the data to a different form or value distribution. Control masks have been used to focus a network on a specific domain of an image.
Prior approaches to the problem of modeling biological preprocessing have been addressed by Grossberg (1988) and Fukushima (1988). For example, the neocognitron neural network developed by Fukushima conceptually models simple, complex, first order and second order hypercomplex cells as well as layers of cells that are sensitive to higher order features. Second order hypercomplex cells are constructed from combinations of first order hypercomplex cells; complex cells are constructed from combinations of simple cells, and the like.
One means of making a complex cell insensitive to location, the approach used by Fukushima, is to design it to receive input from several adjacent simple cells, whose frequency and orientation tuning are similar. The complex cell is made sensitive enough to respond when only one of the simple cells responds to a stimulus. The result is a complex cell with the same frequency and orientation tuning as the simple cells, whose receptive field size is equivalent to the total receptive field size of all its input simple cells combined. Furthermore, the complex cell is insensitive to where in its receptive field the luminance pattern is located (i.e., the complex cell is insensitive to which simple cell has been activated.) Trying to apply biological principles to artificial vision systems, Porat and Zeevi (1989) determined from their work and the work of others, that "primitives of image representations in vision have a wavelet form similar to Gabor elementary functions (EF's)," and proposed a method for texture discrimination in images using a Gabor approach.
Although Porat and Zeevi (1989) proposed that "These localized operators (referring to Gabor functions) are also suitable for a pyramidal scheme of multiresolution which appears to be characteristic of vision, and can also serve as oriented-edge operators and in pattern recognition tasks," (p. 116), they adopted the prevailing approach to the process as a bottom-up hierarchy.
An alternative to extracting features using predefined, generally applicable fixed filters (detectors), such as generated by Gabor and end-stop filters, is to design a system that generates its own feature detectors. In biological systems, the feature detectors must be general enough to handle all possible inputs encountered during the life experiences of the animal. It has been shown that a linear neural network with a correlation rule, when stimulated by random noise, will develop feature detectors similar to the center-surround and Gabor filters found in some artificial visual systems. However, in most practical applications of artificial networks, the universe of possible inputs is more restricted. This suggests that a system for adaptive filter generation that can develop feature detectors specific to the range of images that are encountered in a practical application would be highly desirable. Self-modifying learning algorithms have been pursued wherein a learning algorithm learns about its own effectiveness and modifies itself so that it is the most effective algorithm for solving a certain class of problems.
Despite extensive efforts and much progress, "Forty years of research in artificial neural networks has yielded networks with the neural complexity of, perhaps, a sea slug." (Wenskay, 1991) Image recognition and classification remains a major frontier. The present invention advances toward this frontier.