information channels called interconnects. Where these PE's perform a simple function, such as "on" if the weighted inputs thereto exceed a prescribed threshold, and "off" if not, these PE's may be referred to as "neurons." Each PE or neuron may have multiple input signals, but typically only one output signal (although this output signal may be connected to several different locations within the ANS). The PE's are arranged in several different layers. A first or input layer of PE's is generally characterized by each PE having an input that comprises one of the input signals to the PDN. An output layer of PE's is characterized by each PE providing one of the output signals of the PDN. The output signals from each PE in the input layer comprise the input signals to the PE's of other layers, either in a feedforward or feedback interconnect, and these interconnections, as indicated, may be weighted by an appropriate factor. For simple PDN applications, two layers of PE's may be sufficient, in which case the PDN comprises only an input layer and an output layer. For more complex applications, one or more hidden layers of PE's may be required intermediate the input and output layers. The complexity of the PDN is largely governed by the number of layers of PE's. The general structure of a three-layer PDN or ANS, for example, is illustrated in FIG. 6. This particular three-layer ANS is further discussed below.
Parallel distributed networks are described in the art. See, e.g., Rumelhart, et al., Parallel Distributed Processing, Vol. I (MIT Press, 1986). In general terms, such systems are characterized as including: (1) a set of processing units or elements; (2) a state of activation (i.e., the current state of the element); (3) an output function for each element (i.e., a function that defines the output signal as some function of the input signals); (4) a pattern of connectivity among the processing elements; (5) a propagation rule for propagating patterns of activities (signals) through the pattern of connectivities; (6) an activation rule for combining the inputs impinging of a processing element with the current state of that element to produce a new level of activation for the element; (7) a learning rule whereby patterns of connectivity may be modified by experience; and (8) an environment within which the system operates.
One advantage of using an PDN or ANS is that it can adapt or self-adjust the strength of the interconnections between processing elements. Self-adaptation allows an ANS to "learn" and to eventually improve in overall system performance.
Learning results from application of a selected learning rule. Most learning rules comprise variants of the Hebbian learning rule which states that it two units (processing elements) are highly active, the strength of the interconnect between these two units should be strengthened. A variation of this learning rule is the delta rule. According to the generalized delta rule, the amount of learning (i.e., the strength or weight of the interconnect between two processing elements) is proportional to the difference (or delta) between the actual activation achieved and a target activation provided by a teacher. This delta rule is discussed at length in Chapters 8 and 11 of Rumelhart reference, cited above, and several variations or extensions of the rule exist. The delta rule, its variations and/or extensions, provides the primary learning rule utilized by the ANS of the present invention. Although other learning rules, known or yet to be developed, could of course be used with the explosive detection system, use of the delta rule advantageously provides an effective way to accurately teach the ANS how to recognize an object containing an explosive based on using objects having known explosives as the "teacher".
The particular learning rule selected for a given application is often referred to in the art as a "paradigm of learning." As indicated, numerous paradigms of learning exist and are documented in the art.
Two variations of an ANS-based explosive detection system are co