Exploiting Errors for Efficiency: A Survey from Circuits to Algorithms <s> Example: digit recognition <s> In recent years, inexact computing has been increasingly regarded as one of the most promising approaches for reducing energy consumption in many applications that can tolerate a degree of inaccuracy. Driven by the principle of trading tolerable amounts of application accuracy in return for significant resource savings - the energy consumed, the (critical path) delay and the (silicon) area being the resources - this approach has been limited to certain application domains. In this paper, we propose to expand the application scope, error tolerance as well as the energy savings of inexact computing systems through neural network architectures. Such neural networks are fast emerging as popular candidate accelerators for future heterogeneous multi-core platforms, and have flexible error tolerance limits owing to their ability to be trained. Our results based on simulated 65nm technology designs demonstrate that the proposed inexact neural network accelerator could achieve 43.91%-62.49% savings in energy consumption (with corresponding delay and area savings being 18.79% and 31.44% respectively) when compared to existing baseline neural network implementation, at the cost of an accuracy loss (quantified as the Mean Square Error (MSE) which increases from 0.14 to 0.20 on average). <s> BIB001 </s> Exploiting Errors for Efficiency: A Survey from Circuits to Algorithms <s> Example: digit recognition <s> State-of-the-art deep neural networks (DNNs) have hundreds of millions of connections and are both computationally and memory intensive, making them difficult to deploy on embedded systems with limited hardware resources and power budgets. While custom hardware helps the computation, fetching weights from DRAM is two orders of magnitude more expensive than ALU operations, and dominates the required power. Previously proposed 'Deep Compression' makes it possible to fit large DNNs (AlexNet and VGGNet) fully in on-chip SRAM. This compression is achieved by pruning the redundant connections and having multiple connections share the same weight. We propose an energy efficient inference engine (EIE) that performs inference on this compressed network model and accelerates the resulting sparse matrix-vector multiplication with weight sharing. Going from DRAM to SRAM gives EIE 120× energy saving; Exploiting sparsity saves 10×; Weight sharing gives 8×; Skipping zero activations from ReLU saves another 3×. Evaluated on nine DNN benchmarks, EIE is 189× and 13× faster when compared to CPU and GPU implementations of the same DNN without compression. EIE has a processing power of 102 GOPS working directly on a compressed network, corresponding to 3 TOPS on an uncompressed network, and processes FC layers of AlexNet at 1.88×104 frames/sec with a power dissipation of only 600mW. It is 24,000× and 3,400× more energy efficient than a CPU and GPU respectively. Compared with DaDianNao, EIE has 2.9×, 19× and 3× better throughput, energy efficiency and area efficiency. <s> BIB002 </s> Exploiting Errors for Efficiency: A Survey from Circuits to Algorithms <s> Example: digit recognition <s> As a result of the increasing demand for deep neural network (DNN)-based services, efforts to develop dedicated hardware accelerators for DNNs are growing rapidly. However,while accelerators with high performance and efficiency on convolutional deep neural networks (Conv-DNNs) have been developed, less progress has been made with regards to fully-connected DNNs (FC-DNNs). In this paper, we propose MATIC (Memory Adaptive Training with In-situ Canaries), a methodology that enables aggressive voltage scaling of accelerator weight memories to improve the energy-efficiency of DNN accelerators. To enable accurate operation with voltage overscaling, MATIC combines the characteristics of destructive SRAM reads with the error resilience of neural networks in a memory-adaptive training process. Furthermore, PVT-related voltage margins are eliminated using bit-cells from synaptic weights as in-situ canaries to track runtime environmental variation. Demonstrated on a low-power DNN accelerator that we fabricate in 65 nm CMOS, MATIC enables up to 60-80 mV of voltage overscaling (3.3x total energy reduction versus the nominal voltage), or 18.6x application error reduction. <s> BIB003
Digit recognition is the computational task of determining the correct cardinal number corresponding to an image of a single handwritten digit. One popular approach to implementing digit recognition is using neural networks. In a neural network implementation of digit recognition, pixel values from an input image of a standard size (e.g., 28×28 pixels) are fed into a neural network in which the final layer encodes the digit value (a number between 0 and 9). Because of the compute-intensive nature of neural network operators combined with the resilience to errors thanks to re-training techniques BIB001 , neural networks are a compelling target for resource versus correctness tradeoffs. Neural networks for digit recognition are particularly interesting on devices where energy efficiency is critical. Figure 3 (a) shows a simple network architecture for performing handwritten digit recognition. The network consists of three fully-connected layers (labeled "fc" in b0 stuck at 0 b0 stuck at 1 b1 stuck at 0 b1 stuck at 1 b0 stuck at 0 b0 stuck at 1 b1 stuck at 0 b1 stuck at 1 Error-Aware Ternary Encoding Fig. 4 . Application aware codes can be critical in enabling robustness to errors in neural networks. In the case of ternary neural networks, we can craft codes that guarantee at a deviation of at most 1 from the original value as the result of single bit flip error. the Figure) . The input layer takes in a 28×28 image with one node for each of the 784 pixels and the final output layer has 10 nodes. Applying individual tradeoffs. Figure 3 (b) shows the results for accuracy of the neural network with quantization of weights starting with a 32-bit floating point baseline all the way down to a 1-bit weight. The network is trained on the MNIST data set with quantization of weights either during or after training. The results show that as long as re-training is applied, this neural network is extremely tolerant even to aggressive quantization. Quantization furthermore enables weight prunability and compressibility: weights represented with fewer bits lead to fewer distinct values, and more zero-valued weights. This creates opportunities for sparse matrix compression BIB002 , which can be directly implemented in hardware. Potential for end-to-end optimization. Once the network has been quantized and compressed, we can further leverage resource versus correctness tradeoffs by storing the weights in approximate SRAM  , which occasionally produces read errors. Recent work BIB003 shows that correct retraining and fault detection mechanisms can mitigate the negative effects of SRAM read upsets on classification tasks. In addition, in the extreme case where weights are ternarized to −1, 0 and 1, one could explore an encoding with a redundant representation of zero as Figure 4 shows. With this encoding, single bit-flip errors would cause, in the worst case, a deviation of one as opposed to a value polarity flip from −1 to 1 or vice-versa. The latter is allowed under the default 2's complement encoding, and could potentially lead to further accuracy degradation.