BACKGROUND

This specification relates to determining outputs for neural networks.

Neural networks are machine learning models that employ one or more layers of nonlinear units to predict an output for a received input. Some neural networks include one or more hidden layers in addition to an output layer. The output of each hidden layer is used as input to the next layer in the network, i.e., the next hidden layer or the output layer. Each layer of the network generates an output from a received input in accordance with current values of a respective set of parameters.

SUMMARY

This specification describes a system implemented as computer programs on one or more computers in one or more locations that performs the multiplications required by a neural network using tables, i.e., look-up tables, instead of performing the multiplications in software or using hardware multipliers. This greatly reduces the amount of computational resources required to perform an inference for the neural network and allows inferences to be computed for the neural network with minimal latency. More particularly, the tables required to perform input-weight multiplications and to apply the activation function for a given neural network layer can be stored in on-chip memory, e.g., of a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC) on which the neural network computations are being performed, and can therefore be repeatedly accessed to generate outputs quickly and while consuming minimal computational resources.

Particular embodiments of the subject matter described in this specification can be implemented so as to realize one or more of the following advantages.

Processing neural networks using look-up tables as described in this specification greatly reduces the amount of computational resources required to perform an inference for the neural network and allows inferences to be computed for the neural network with minimal latency, i.e., because the system computes inferences using the neural network without performing any multiplications and without requiring any support for floating point numbers. In particular, the tables required to perform input-weight multiplications and to apply activation functions for any given layer can be stored in on-chip memory of the special-purpose hardware, e.g., on-chip memory of an FPGA or an ASIC on which the neural network computations are being performed, and can therefore be repeatedly accessed to generate outputs quickly and while consuming minimal computational resources. Additionally, because performing inference through the neural network using tables as described in this specification does not require circuitry for performing multiplication, for computing non-linearities or support for floating point numbers, the system can deploy the neural network on hardware, e.g., ASICS or FPGA, that has greatly reduced complexity, surface area, and power consumption relative to what would be required for deploying neural networks for conventional inference.

The details of one or more embodiments of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims.

