The present invention relates, generally, to artificial neural networks and, more particularly, to a method and circuit for performing a diagnostic which guarantees the integrity of the neurons forming the artificial neural network.
Artificial neural networks (ANNs) are used with increased frequency in applications where no mathematical algorithm can describe the problem to be solved. Moreover, they have proven to be highly successful in classifying and recognizing objects. ANNs give excellent results in such cases because of their ability to learn by example and their proficiency of generalizing in order to respond to an input vector that was never introduced before. Thus far, most ANNs have been implemented in software and only a few in hardware.
Several neurons and artificial neural network architectures implemented in semiconductor chips are described in the following related patents:
U.S. Pat. No. 5,621,863 xe2x80x9cNeuron Circuitxe2x80x9d, issued on Apr. 15, 1997 to Boulet et al.;
U.S. Pat. No. 5,701,397 xe2x80x9cCircuit for Pre-charging a Free Neuron Circuitxe2x80x9d, issued on Dec. 23, 1997 to Steimle, et al.;
U.S. Pat. No. 5,710,869 xe2x80x9cDaisy Chain Circuit for Serial Connection of Neuron Circuitsxe2x80x9d, issued on Jan. 20, 1998 to Godefroy, et al.;
U.S. Pat. No. 5,717,832 xe2x80x9cNeural Semiconductor Chip and Neural Networks Incorporated Thereinxe2x80x9d, issued on Feb. 18, 1998 to Boulet, et al.; and
U.S. Pat. No. 5,740,326 xe2x80x9cCircuit for Searching/Sorting Data in Neural Networksxe2x80x9d, issued on Apr. 14, 1998 to Boulet, et al.,
all of which are incorporated herein by reference.
The semiconductor chips can be advantageously be implemented using chips known under the name ZISC (ZISC is a registered Trademark of the IBM Corporation). In the aforementioned ZISC chips, the ANNs are based on mapping an input space, in accordance to a well known Region of Influence (ROI) algorithm.
With reference to U.S. Pat. No. 5,710,869, the base ZISC chip architecture is depicted in FIGS. 4A and 4B. As apparent from the drawings, a plurality of neuron circuits referenced 11-1 to 11-n are fed in parallel by four buses labeled DATA-BUS, CAT-BUS, MaxIF/MinIF-BUS and NO/CXT-BUS which respectively transmit data, category information, maximum/minimum values of the field of influence and xe2x80x98normal contextxe2x80x99 values. After being processed, the data is outputted to a common communication bus labeled COM*-BUS via OR gate 12. The detailed architecture of a neuron is shown in FIG. 5 of the above mentioned patent. During the recognition phase, the multi-norm distance evaluator circuit calculates the distance D between the input vector and the prototype stored therein. A xe2x80x98distance compare circuitxe2x80x99 compares D with the prototype actual influence field (AIF) to generate a first and second comparison signal. An identification circuit processes the comparison signals and the local category signal to generate signals that represent the neuron""s response to the input vector. A minimum distance determination circuit establishes the minimum distance Dmin between the distances calculated by each neuron of the ANN. This circuit is also subsequently used to search and sort categories. For each neuron, a daisy chain circuit is serially connected to the daisy chain circuit of two adjacent neurons, thereby linking the neurons forming the ANN. The daisy chain circuit is essential in determining the state of the neuron, i.e., whether free or engaged. Finally, a context circuit enables (or inhibits) the neuron from participating alongside other neurons during the recognition phase by comparing the local context of each neuron to the global context.
During the learning phase, a xe2x80x98write category operationxe2x80x99only engages the first free neuron of the ANN. As mentioned previously, the daisy chain circuit distinguishes between the two possible states of the neuron (engaged or free) and identifies the first free xe2x80x98or ready to learnxe2x80x99neuron of the ANN, based on the respective values of its input and output signals. In the following description, a neuron will be considered as having three potential states (first free, free and engaged). The free neuron is the only neuron having these signals in a complementary state. The daisy chain circuit is described in detail in U.S. Pat. No. 5,710,869 with reference to FIGS. 5 and 31. In order to facilitate the understanding of the present invention, this drawing has been duplicated herein as FIG. 1 but with different numerals.
Referring to the aforementioned FIG. 1, the conventional daisy chain circuit now referenced 10 includes a 1-bit register 11 controlled by a store enable signal ST which is activated at initialization or at the end of the learning phase in which a new neuron is engaged. The 1-bit register 11 input is connected to the output of a 2-way AND gate 12 receiving signals DCI and RESET-. The output of 1-bit register 11 is connected to the input of a second 2-way OR gate 13 while the ALL signal is applied to the other input. The ALL signal enables all the daisy chain circuits of the ANN at initialization. DCO is the signal which is outputted from OR gate 13. Daisy chain circuit 10 further includes a control logic circuit 14 which consists of a 2-way XOR (exclusive OR) gate 15 and a 2-way AND gate 16. In XOR gate 15, DCI and DCO signals are compared to generate control signal RS (Ready to Store) that will be subsequently used as a gating signal for different circuits of each neuron to identify the first free or xe2x80x98ready to learnxe2x80x99neuron in the ANN. On the other hand, AND gate 16 generates a CO signal from the DCO and NS signals. The NS signal is generated by the matching circuit of the neuron (referenced 150 in FIGS. 5 and 16 of U.S. Pat. No. 5,710,869).
At initialization, all the 1-bit registers are forced to xe2x80x980xe2x80x99 by signal RESET-. The DCI input of the daisy chain of the first neuron in the ANN is set to xe2x80x981xe2x80x99 so that this neuron is xe2x80x98ready to learnxe2x80x99 (i.e., the first free neuron).
During the learning phase, the 1-bit register 11 of this free neuron is set to xe2x80x981xe2x80x99 when the ST signal is activated, xe2x80x9cengagingxe2x80x9d this neuron (i.e., its output is at xe2x80x981xe2x80x99). Once engaged, the next neuron in the ANN becomes the first free neuron. The signal RS, which is a combination of both DCI and DCO signals, enables the loading of the input vector components into the R/W (read/write) memory circuit (referenced 250 in FIG. 5 of U.S. Pat. No. 5,710,869). It also determines the category specified by the user into the category register, the global context value into the local context register and, finally, the minimum distance Dmin determined by the determination circuit (referenced 500 in FIGS. 5 and 8 of the aforementioned patent) which is stored in the AIF register as the AIF value of the first free neuron. At the end of the learning process, the DCI signal is latched into the 1-bit register 11 for each neuron to define which neuron is the first free neuron of the ANN.
During the recognition mode, only the neurons which are engaged and for which the local context matches the global context of all the engaged neurons (when the global context equals xe2x80x980xe2x80x99) participate in the recognition process. It is the role of the CO signal to select the distance evaluator circuit (referenced 200 in FIGS. 5 and 19 of U.S. Pat. No. 5,710,869) and the minimum distance determination circuit of these neurons to make them operative.
Unfortunately, the daisy chain circuit in ZISC chips does not include an adequate on-chip testability capability which would easily allow to test the integrity of the neurons forming the ANN not only during the setup but also when an application is running, without complicating the existing software and negatively impacting the overall performance of the ANN.
It is therefore a primary object of the present invention to provide a method and a circuit that perform an integrity diagnostic of all the neurons forming an artificial neural network based upon mapping the input space.
It is another object of the present invention to provide a method and a circuit that perform an integrity diagnostic of all the neurons forming an artificial neural network based upon mapping the input space which can simultaneously test all the neurons in real time.
It is another object of the present invention to provide a method and a circuit that perform an integrity diagnostic of all the neurons forming an artificial neural network based upon mapping the input space at the cost of only a few logic gates.
The accomplishments of these and other related objects are achieved by the method of the present invention. The diagnostic method engages simultaneously all the neurons of the ANN network of the same prototype.
In a first aspect of the invention, the method includes the steps of xe2x80x98learningxe2x80x99all the neurons of the ANN of the same prototype (vector components, category, context and AIF). A first test vector having. e.g., the same context, is first loaded into the ANN. All the neurons compute their own distance with a first test vector (this distance should be the same if all the neurons were good). If at least one neuron fails, either the distance, the category, the context or the AIF value will be different from a corresponding expected value. The neuron giving at least one different answer can easily be isolated using the minimum distance determination circuit and flagged according to its particular context, i.e., a value identifying a fail. As long as this context is not selected by the global context, failing neurons do not contribute to the learning and recognition processes. The diagnostic method allows a fast integrity diagnostic of the neurons of an ANN based on mapping an input space. When implemented with ZISC chips, the diagnostic method also reveals itself to be inexpensive since it only requires a few additional logic gates.