INTRODUCTION

The subject disclosure relates to controlling a vehicle and, more particularly, to activation zero-bypass and weight pruning in neural networks for vehicle perception systems.

Machine learning involves the generation and use of algorithms capable of learning from, and making predictions, on data. Such algorithms typically operate by building a model from example inputs in order to make data-driven predictions or decisions. A number of machine learning approaches have been developed. One such approach, known as an artificial neural network (ANN), or simply a neural network (NN), is a learning algorithm inspired by the structure and function of biological neural networks.

An NN includes hierarchal layers of interconnected groups of artificial neurons (nodes), where each layer of nodes receives as inputs, the outputs of a lower layer. Deep neural networks (DNNs) are a type of NN that includes one or more hidden layers of nodes. Feed-forward NNs are NNs in which connections between the nodes do not form a cycle. That is, feed-forward NNs are NNs in which the information moves in only one direction, forward, from the input nodes, through the nodes of one or more hidden layers (if any), and ultimately to the output nodes. Convolutional NNs constitute a class of deep, feed-forward NNs that include input and output layers as well as multiple hidden layers. The hidden layers typically include convolutional layers, pooling layers, fully-connected layers, and normalization layers. Technical improvements to conventional NNs are described herein.

SUMMARY

In one exemplary embodiment, a computer-implemented method includes capturing a plurality of images at a camera associated with a vehicle and storing image data associated with the plurality of images to a memory. The method further includes dispatching vehicle perception tasks to a plurality of processing elements of an accelerator in communication with the memory. The method further includes performing, by at least one of the plurality of processing elements, the vehicle perception tasks for the vehicle perception using a neural network, wherein performing the vehicle perception tasks comprises performing an activation bypass for values below a first threshold, and performing weight pruning of synapses and neurons of the neural network based at least in part on a second threshold. The method further includes controlling the vehicle based at least in part on a result of performing the vehicle perception tasks.

In one or more embodiments, performing the weight pruning further comprises, performing an analysis, performing weight pruning based on the analysis, and tuning the weight pruning. In one or more embodiments, performing the weight pruning further comprises determining whether to perform additional compression. In one or more embodiments, performing the weight pruning further comprises, responsive to determining to perform additional compressing, re-performing the weight pruning and the tuning. In one or more embodiments, performing the weight pruning further comprises, responsive to determining not to perform additional compressing, performing a transformation. In one or more embodiments, performing the activation bypass avoids performing a multiplication and accumulation (MAC) operation, and wherein the MAC operation is performed for values above the first threshold. In one or more embodiments, the first threshold is determined based at least in part on a percent of operations pruned and a prediction error percentage. In one or more embodiments, the accelerator further comprises a neural processing unit instruction fetcher, a dispatcher, a level 1 cache, and a level 2 cache. In one or more embodiments, the accelerator is communicatively coupled to a host central processing unit via the memory, wherein the accelerator is a special purpose processor and wherein the host central processing unit is a general purpose processor. In one or more embodiments, the host central processing unit receives the image data from the camera associated with the vehicle and stores the image data associated with the plurality of images to the memory.

In another exemplary embodiment, a system includes a memory having computer readable instructions and a processing device for executing the computer readable instructions for performing a method. The method includes capturing a plurality of images at a camera associated with a vehicle and storing image data associated with the plurality of images to a memory. The method further includes dispatching vehicle perception tasks to a plurality of processing elements of an accelerator in communication with the memory. The method further includes performing, by at least one of the plurality of processing elements, the vehicle perception tasks for the vehicle perception using a neural network, wherein performing the vehicle perception tasks comprises performing an activation bypass for values below a first threshold, and performing weight pruning of synapses and neurons of the neural network based at least in part on a second threshold. The method further includes controlling the vehicle based at least in part on a result of performing the vehicle perception tasks.

In one or more embodiments, performing the weight pruning further comprises: performing an analysis; performing weight pruning based on the analysis; and tuning the weight pruning. In one or more embodiments, performing the weight pruning further comprises determining whether to perform additional compression. In one or more embodiments, performing the weight pruning further comprises, responsive to determining to perform additional compressing, re-performing the weight pruning and the tuning. In one or more embodiments, performing the weight pruning further comprises, responsive to determining not to perform additional compressing, performing a transformation. In one or more embodiments, performing the activation bypass avoids performing a multiplication and accumulation (MAC) operation, and wherein the MAC operation is performed for values above the first threshold. In one or more embodiments, the first threshold is determined based at least in part on a percent of operations pruned and a prediction error percentage. In one or more embodiments, the accelerator further comprises a neural processing unit instruction fetcher, a dispatcher, a level 1 cache, and a level 2 cache. In one or more embodiments, the accelerator is communicatively coupled to a host central processing unit via the memory, wherein the accelerator is a special purpose processor and wherein the host central processing unit is a general purpose processor, and the host central processing unit receives the image data from the camera associated with the vehicle and stores the image data associated with the plurality of images to the memory.

In yet another exemplary embodiment a computer program product includes a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processing device to cause the processing device to perform a method. The method includes capturing a plurality of images at a camera associated with a vehicle and storing image data associated with the plurality of images to a memory. The method further includes dispatching vehicle perception tasks to a plurality of processing elements of an accelerator in communication with the memory. The method further includes performing, by at least one of the plurality of processing elements, the vehicle perception tasks for the vehicle perception using a neural network, wherein performing the vehicle perception tasks comprises performing an activation bypass for values below a first threshold, and performing weight pruning of synapses and neurons of the neural network based at least in part on a second threshold. The method further includes controlling the vehicle based at least in part on a result of performing the vehicle perception tasks.

The above features and advantages, and other features and advantages of the disclosure are readily apparent from the following detailed description when taken in connection with the accompanying drawings.

