A Survey on Deep Learning Methods for Robot Vision <s> Deep Compression and Quantization <s> The focus of this paper is speeding up the application of convolutional neural networks. While delivering impressive results across a range of computer vision and machine learning tasks, these networks are computationally demanding, limiting their deployability. Convolutional layers generally consume the bulk of the processing time, and so in this work we present two simple schemes for drastically speeding up these layers. This is achieved by exploiting cross-channel or filter redundancy to construct a low rank basis of filters that are rank-1 in the spatial domain. Our methods are architecture agnostic, and can be easily applied to existing CPU and GPU convolutional frameworks for tuneable speedup performance. We demonstrate this with a real world network designed for scene text character recognition [15], showing a possible 2.5× speedup with no loss in accuracy, and 4.5× speedup with less than 1% drop in accuracy, still achieving state-of-the-art on standard benchmarks. <s> BIB001 </s> A Survey on Deep Learning Methods for Robot Vision <s> Deep Compression and Quantization <s> Neural networks are both computationally intensive and memory intensive, making them difficult to deploy on embedded systems with limited hardware resources. To address this limitation, we introduce "deep compression", a three stage pipeline: pruning, trained quantization and Huffman coding, that work together to reduce the storage requirement of neural networks by 35x to 49x without affecting their accuracy. Our method first prunes the network by learning only the important connections. Next, we quantize the weights to enforce weight sharing, finally, we apply Huffman coding. After the first two steps we retrain the network to fine tune the remaining connections and the quantized centroids. Pruning, reduces the number of connections by 9x to 13x; Quantization then reduces the number of bits that represent each connection from 32 to 5. On the ImageNet dataset, our method reduced the storage required by AlexNet by 35x, from 240MB to 6.9MB, without loss of accuracy. Our method reduced the size of VGG-16 by 49x from 552MB to 11.3MB, again with no loss of accuracy. This allows fitting the model into on-chip SRAM cache rather than off-chip DRAM memory. Our compression method also facilitates the use of complex neural networks in mobile applications where application size and download bandwidth are constrained. Benchmarked on CPU, GPU and mobile GPU, compressed network has 3x to 4x layerwise speedup and 3x to 7x better energy efficiency. <s> BIB002 </s> A Survey on Deep Learning Methods for Robot Vision <s> Deep Compression and Quantization <s> Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers—8× deeper than VGG nets [40] but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions1, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation. <s> BIB003 </s> A Survey on Deep Learning Methods for Robot Vision <s> Deep Compression and Quantization <s> We propose two efficient approximations to standard convolutional neural networks: Binary-Weight-Networks and XNOR-Networks. In Binary-Weight-Networks, the filters are approximated with binary values resulting in 32\(\times \) memory saving. In XNOR-Networks, both the filters and the input to convolutional layers are binary. XNOR-Networks approximate convolutions using primarily binary operations. This results in 58\(\times \) faster convolutional operations (in terms of number of the high precision operations) and 32\(\times \) memory savings. XNOR-Nets offer the possibility of running state-of-the-art networks on CPUs (rather than GPUs) in real-time. Our binary networks are simple, accurate, efficient, and work on challenging visual tasks. We evaluate our approach on the ImageNet classification task. The classification accuracy with a Binary-Weight-Network version of AlexNet is the same as the full-precision AlexNet. We compare our method with recent network binarization methods, BinaryConnect and BinaryNets, and outperform these methods by large margins on ImageNet, more than \(16\,\%\) in top-1 accuracy. Our code is available at: http://allenai.org/plato/xnornet. <s> BIB004 </s> A Survey on Deep Learning Methods for Robot Vision <s> Deep Compression and Quantization <s> We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0%, respectively, which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. To reduce overfitting in the fully connected layers we employed a recently developed regularization method called "dropout" that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry. <s> BIB005
State-of-the-art vision systems based on CNNs require large memory and computational resources, such as those provided by high-end GPUs. For this reason CNN-based methods are unable to run on devices with low resources, such as smartphones or mobile robots, limiting their use in real-world applications. Thus, the development of mechanisms that allow CNNs to work using less memory and fewer computational resources, such as compression and quantization of DNNs, is an important area to be addressed. Mathieu et al.  propose computing convolutions by using FFTs. This technique enables reusing transformed filters with multiple images. FFT is efficient when computing responses for shared convolutional layers. Also, inverse FFT is required only once for each output channel. However, the use of FFTs requires additional memory for storing the layers in the frequency domain. Libraries such as CuDNN [153] are able to use FFTs when training and testing the networks. FFT is more efficient than normal convolutions when filter masks are large. However, the state-of-the-art ResNet BIB003 uses small 3x3 masks. The use of sparse representations is a useful approach for network compression. In BIB001 , kernels are factorized into a set of column and row filters, by using an approach based on separable convolutions. A 2.5x speedup is obtained with no loss of accuracy, and a 4.5x speedup with a loss in accuracy of only 1%. A procedure for reducing the redundancy in a CNN by using sparse decomposition is proposed in  . Both the images and the kernels are transformed onto a different space by using matrix multiplications, where convolutions can be computed by using sparse matrix multiplication. This procedure is able to make more than 90% of the parameters zero in an AlexNet BIB005 , with a drop in accuracy that is less than 1%. In BIB002 , a methodology for compressing CNNs is proposed, which is based on a three-stage pipeline: pruning, trained quantization, and Huffman coding. In the first stage, the original network is pruned by learning only the important connections, and a compressed sparse row/column format is used for storing the non-zero components. In the second stage, weights are quantized by using a 256-word dictionary for convolutional layers, and a 32-word dictionary for fully connected layers, enabling weight sharing. In the third stage, the binary representation of the network is compressed by using Huffman coding. By using the proposed approach, AlexNet can be compressed from 240MB to 6.9MB, achieving a 35x compression rate, and VGG can be compressed from 552MB to 11.3MB, achieving a 49x compression rate, without loss of accuracy. In  , an EIE (Efficient Inference Engine) is proposed for processing images while using the compressed network representation. EIE is 189x faster, and 13x faster when compared to CPU and GPU implementations of AlexNet without compression, without loss of accuracy. Two binary-based network architectures are proposed in BIB004 : Binary-Weight-Networks and XNOR-Networks. In BinaryWeight-Networks, the filters are approximated with binary values in closed form, resulting in a 32x memory saving. In XNORNetworks, both the filters and the input to convolutional layers are binary, but non-binary non-linearities like ReLU can still be used. This results in 58x faster convolutional operations on a CPU, by using XNOR-Bitcounting operations. The classification accuracy with a Binary-Weight-Network version of AlexNet is only 2.9% less than the full-precision AlexNet (in top-1 measure),;while XNOR-Networks have a larger, 12.4%, drop in accuracy. The compression and quantization of DNNs is an area under development. It is expected that easier ways for reducing the size of DNNs will be developed in the near future, and that deep learning frameworks and tools will make them available for use by developers of DNN based applications.