We present a Deep Neural Network with Spike Assisted Feature Extraction (SAFE-DNN) to improve robustness of classification under stochastic perturbation of inputs. The proposed network augments a DNN with unsupervised learning of low-level features using spiking neuron network (SNN) with Spike-Time-Dependent-Plasticity (STDP). The complete network learns to ignore local perturbation while performing global feature detection and classification. The experimental results on CIFAR-10 and ImageNet subset demonstrate improved noise robustness for multiple DNN architectures without sacrificing accuracy on clean images. There is a growing interest in deploying DNNs in autonomous systems interacting with physical world such as autonomous vehicles and robotics. It is important that an autonomous systems make reliable classifications even with noisy data. However, in a deep convolutional neural networks (CNN) trained using stochastic gradient descent (SGD), pixel level perturbation can cause kernels to generate incorrect feature maps. Such errors can propagate through network and degrade the classification accuracy (Nazaré et al. (2017) ; Luo & Yang (2014) ). Approaches for improving robustness of a DNN to pixel perturbation can be broadly divided into two complementary categories. First, many research efforts have developed image de-noising (or filtering) networks that can pre-process an image before classification, but at the expense of additional latency in the processing pipeline (Ronneberger et al. (2015) ; Na et al. (2019) ; Xie et al. (2012) ; Zhussip & Chun (2018) ; Soltanayev & Chun (2018) ; Zhang et al. (2017) ). De-noising is an effective approach to improve accuracy under noise but can degrade accuracy for clean images (Na et al. (2019) ). Moreover, de-noising networks trained on a certain noise type do not perform well if the a different noise structure is experienced during inference (Zhussip & Chun (2018) ). Advanced de-noising networks are capable of generalizing to multiple levels of a type of noise and effective for different noise types (Zhussip & Chun (2018) ; Soltanayev & Chun (2018) ; Zhang et al. (2017) ). But high complexity of these network makes them less suitable for real-time applications and lightweight platforms with limited computational and memory resources. An orthogonal approach is to develop a classification network that is inherently robust to input perturbations. Example approaches include training with noisy data, introducing noise to network parameters during training, and using pixel level regularization (Milyaev & Laptev (2017) ; Nazaré et al. (2017) ; Luo & Yang (2014) ; Na et al. (2018) ; Long et al. (2019) ). These approaches do not change the processing pipeline or increase computational and memory demand during inference. However, training-based approaches to design robust DNNs also degrade classification accuracy for clean images, and more importantly, are effective only when noise structure (and magnitude) during training and inference closely match. Therefore, a new class of DNN architecture is necessary for autonomous system that is inherently resilient to input perturbations of different type and magnitude without requiring training on noisy data, as well as computationally efficient. Towards this end, this paper proposes a new class of DNN architecture that integrates features extracted via unsupervised neuro-inspired learning and supervised training. The neuro-inspired learning, in particular, spiking neural network (SNN) with spike-timing-dependent plasticity (STDP) is an alternative and unsupervised approach to learning features in input data (Hebb et al. (1950) ; (2019)). However, the classification accuracy of a STDP-learned SNN for complex datasets is much lower than a that of a DNN. The fundamental premise of this paper is that, augmenting the feature space of a supervised (trained) DNN with features extracted by an SNN via STDP-based learning increases robustness of the DNN to input perturbations. We argue that stochastic gradient descent (SGD) based back-propagation in a DNN enables global learning between low-level pixel-to-pixel interactions and high-level detection and classification. On the other hand, STDP performs unsupervised local learning and extracts low-level features under spatial correlation. By integrating features from global (supervised training) and local (STDP) learning, the hybrid network "learns to ignore" locally uncorrelated perturbations (noise) in pixels while extracting the correct feature representation from the overall image. Consequently, hybridization of SGD and STDP enables robust image classification under noisy input while preserving the accuracy of the baseline DNN for clean images. We present a hybrid network architecture, referred to as Spike Assisted Feature Extraction based Deep Neural Network (SAFE-DNN), to establish the preceding premise. We develop an integrated learning/training methodology to couple the features extracted via neuro-inspired learning and supervised training. In particular, this paper makes the following contributions: • We present a SAFE-DNN architecture ( Figure 1 ) that couples STDP-based robust learning of local features with SGD based supervised training. This is achieved by integrating a spiking convolutional module within a DNN pipeline. • We present a novel frequency-dependent stochastic STDP learning rule for the spiking convolutional demonstrating local competitive learning of low level features. The proposed learning method makes the feature extracted by the spiking convolutional module robust to local perturbations in the input image. • We develop a methodology to transform the STDP-based spiking convolution to an equivalent CNN. This is achieved by using a novel special neuron activation unit (SAU), a non-spiking activation function, that facilitates integration of the SNN extracted features within the DNN thereby creating a single fully-trainable deep network. The supervised (SGD-based) training is performed in that deep network after freezing the STDP-learnt weights in the spiking CNN module. We present implementations of SAFE-DNN based on different deep networks including MobileNet, ResNet and DenseNet (Sandler et al. (2018) , He et al. (2015) , Huang et al. (2016) ) to show the versatility of our network architecture. Experiment is conducted for CIFRA10 and ImageNet subset considering different types of noise, including Gaussian, Wald, Poisson, Salt&Paper, and adversarial noise demonstrating robust classification under input noise. Unlike training-based approaches, SAFE-DNN shows improved accuracy for a wide range of noise structure and magnitude without requiring any prior knowledge of the perturbation during training and inference and does not degrade the accuracy for clean images (even shows marginal improvement in many cases). SAFE-DNN complements, and can be integrated with, de-noising networks for input pre-processing. However, unlike de-noising networks, the SAFE-DNN has negligible computation and memory overhead, and does not introduce new stages in the processing pipeline. Hence, SAFE-DNN is an attractive architecture for resource-constrained autonomous platforms with real-time processing. We note that, SAFE-DNN differs from deep SNNs that convert a pre-trained DNN to SNN (Sengupta et al. (2019) , Hu et al. (2018) ). Such networks function as a spiking network during inference to reduce energy; however, the learning is still based on supervision and back-propagation. In contrast, SAFE-DNN hybridizes STDP and SGD during learning but creates a single hybrid network operating as a DNN during inference. In this paper we present SAFE-DNN as a deep learning architecture that integrates spiking convolutional network with STDP based learning into a conventional DNN for robust low level feature extraction. The experimental results show that SAFE-DNN improves robustness to different input perturbations without any prior knowledge of the noise during training/inference. SAFE-DNN is compatible with various DNN designs and incurs negligible computation/memory overhead. Hence, it is an attractive candidate for real-time autonomous systems operating in noisy environment.
A noise robust deep learning architecture.
{"src": "scitldr-AIC_train", "paper_id": "BJg1fgBYwH"}