BACKGROUND

The present invention generally relates to crossbar arrays that implement neural networks, and more specifically, to training neural networks using compact fixed bit length deterministic pulse generation for updating analog crossbar arrays.

Artificial neural networks (ANNs), also known as neuromorphic and synaptronic systems, are computational systems that permit electronic systems to essentially function in a manner analogous to that of biological neural systems such as a human brain. In this detailed description, any reference to an ANN is used in the broadest sense to cover a wide range of computer and electronic systems that model biological neural networks, including but not limited to pulsed neural networks (PNNs). Some implementations of ANNs do not generally utilize the traditional digital model of manipulating zeros and ones. Instead, some implementations of ANNs create connections between processing elements that are roughly functionally equivalent to neurons of a biological brain.

ANNs are often implemented as relatively large, parallel computing systems having many simple processors with many interconnections. Modeling a biological nervous system using ANNs (or PNNs) can contribute to a better understanding of biological functions. Many ANN models attempt to use some “organizational” principles believed to be used in the human brain. Contemporary computer hardware technology (such as VLSI and optical) has made such modeling feasible. ANNs incorporate knowledge from a variety of disciplines, including neurophysiology, cognitive science/psychology, physics (statistical mechanics), control theory, computer science, artificial intelligence, statistics/mathematics, pattern recognition, computer vision, parallel processing and hardware (e.g., digital/analog/VLSI/optical).

A contemporary ANN includes a network of artificial neurons (also known as “nodes”). These nodes are connected to each other, and the strength of their connections to one another is assigned a value. For example, connections can be identified as exhibiting inhibition (maximum inhibition being negative 1.0) or excitation (maximum excitation being plus 1.0). If the magnitude value of the connection is high, this indicates a strong connection. Within each node's design, a transfer function is also built in. There are three types of neurons in a typical ANN, namely input nodes, hidden nodes and output nodes.

Input nodes take in information that can be numerically expressed. The information is presented as activation values, wherein each node is given a number, and a number having a higher magnitude causes a greater activation. This information is then passed throughout the network. Based on the connection strengths (weights), inhibition (or excitation), and transfer functions, the activation value is passed from node to node. Each node sums the activation values it receives. Each node than modifies the value based on its transfer function. The activation flows through the input nodes and the hidden layers until it reaches the output nodes. The output nodes reflect the input in a meaningful way to the outside world.

There are many types of neural networks, but the two broadest categories are feed-forward and feed-back networks. A feed-forward network is a non-recurrent network having inputs, outputs, and hidden layers. The signals can only travel in one direction. Input data is passed onto a layer of processing elements that perform calculations. Each processing element makes its computation based upon a weighted sum of its inputs. The new calculated values then become the new input values that feed the next layer. This process continues until it has gone through all the layers and determined the output. A threshold transfer function is sometimes used to quantify the output of a neuron in the output layer.

A feed-back network includes feed-back paths, which means that their signals can travel in both directions using loops. All possible connections between neurons are allowed. Because loops are present in this type of network, under certain operations, it can become a non-linear dynamical system that changes continuously until it reaches a state of equilibrium. Feed-back networks are often used in associative memories and optimization problems, wherein the network looks for the best arrangement of interconnected factors.

In an ANN, a spike generation function can be modeled by a component known generally as a temporal encoder or a spike generator. Thus, the ability to gain greater knowledge of the previously describe biological neural system depends on the development of ANNs that model how neural information is encoded in recurrent networks of spiking temporal encoders. Temporal encoders provide an alternative to both digital and analog encoding of information by integrating received signals asynchronously and producing subsequent sets of asynchronous pulsed signals. Certain configurations of temporal encoder networks allow for extraction of information about sets of signals input to the network from sets of pulses or spikes output from the network.

SUMMARY

Embodiments of the present invention are directed to a computer-implemented method of pulse generation for updating crossbar arrays. A non-limiting example of the computer-implemented method includes receiving a number used in matrix multiplication to represent using pulse generation for a crossbar array, and receiving a bit-length to represent the number. The computer-implemented method also includes selecting pulse positions in a pulse sequence having the bit length to represent the number, performing a computation using the selected pulse positions in the pulse sequence, and updating the crossbar array using the computation.

Embodiments of the present invention are directed to a system for generating pulses for updating crossbar arrays. A non-limiting example of the system includes a crossbar array having one or more memristive devices and a processor. The processor is configured to receive a number used in matrix multiplication to represent using pulse generation for the crossbar array, and receive a bit-length to represent the number. The processor is also configured to select pulse positions in a pulse sequence having the bit length to represent the number, perform a computation using the selected pulse positions in the pulse sequence, and update the crossbar array using the computation, wherein updating the crossbar array changes weights of the one or more memristive devices.

Embodiments of the invention are directed to a computer program product for pulse generation for updating crossbar arrays, the computer program product including a computer readable storage medium having program instructions embodied therewith. The program instructions are executable by a processor to cause the processor to perform a method. A non-limiting example of the method includes receiving a number used in matrix multiplication to represent using pulse generation for a crossbar array, and receiving a bit-length to represent the number. The method also includes selecting pulse positions in a pulse sequence having the bit length to represent the number, performing a computation using the selected pulse positions in the pulse sequence, and updating the crossbar array using the computation.

Additional technical features and benefits are realized through the techniques of the present invention. Embodiments and aspects of the invention are described in detail herein and are considered a part of the claimed subject matter. For a better understanding, refer to the detailed description and to the drawings.

