A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> This paper introduces two algorithms for analog and digital modulations recognition. The first algorithm utilizes the decision-theoretic approach in which a set of decision criteria for identifying different types of modulations is developed. In the second algorithm the artificial neural network (ANN) is used as a new approach for the modulation recognition process. Computer simulations of different types of band-limited analog and digitally modulated signals corrupted by band-limited Gaussian noise sequences have been carried out to measure the performance of the developed algorithms. In the decision-theoretic algorithm it is found that the overall success rate is over 94% at the signal-to-noise ratio (SNR) of 15 dB, while in the ANN algorithm the overall success rate is over 96% at the SNR of 15 dB. <s> BIB001 </s> A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> This paper presents a new digital modulation recognition algorithm for classifying baseband signals in the presence of additive white Gaussian noise. An elaborated classification technique uses various statistical moments of the signal amplitude, phase, and frequency applied to the fuzzy classifier. Classification results are given and it is found that the technique performs well at low SNR. Benefits of this technique are that it is simple to implement, has a generalization property, and requires no a priori knowledge of the SNR, carrier phase, or baud rate of the signal for classification. <s> BIB002 </s> A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> Automatic recognition of digital modulation signals has seen increasing demand nowadays. The use of artificial neural networks for this purpose has been popular since the late 1990s. Here, we include a variety of modulation types for recognition, e.g. QAM16, V29, V32, QAM64 through the addition of a newly proposed statistical feature set. Two training algorithms for multi-layer perceptron (MLP) recogniser, namely Backpropagation with Momentum and Adaptive Learning Rate is investigated, while resilient backpropagation (RPROP) is proposed for this problem, are employed in this work. In particular, the RPROP algorithm is applied for the first time in this area. In conjunction with these algorithms, we use a separate data set as validation set during training cycle to improve generalisation. Genetic algorithm (GA) based feature selection is used to select the best feature subset from the combined statistical and spectral feature set. RPROP MLP recogniser achieves about 99% recognition performance on most SNR values with only six features selected using GA. <s> BIB003 </s> A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> High-dimensional data can be converted to low-dimensional codes by training a multilayer neural network with a small central layer to reconstruct high-dimensional input vectors. Gradient descent can be used for fine-tuning the weights in such “autoencoder” networks, but this works well only if the initial weights are close to a good solution. We describe an effective way of initializing the weights that allows deep autoencoder networks to learn low-dimensional codes that work much better than principal components analysis as a tool to reduce the dimensionality of data. <s> BIB004 </s> A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> Spectrum awareness is currently one of the most challenging problems in cognitive radio (CR) design. Detection and classification of very low SNR signals with relaxed information on the signal parameters being detected is critical for proper CR functionality as it enables the CR to react and adapt to the changes in its radio environment. In this work, the cycle frequency domain profile (CDP) is used for signal detection and preprocessing for signal classification. Signal features are extracted from CDP using a threshold-test method. For classification, a Hidden Markov Model (HMM) has been used to process extracted signal features due to its robust pattern-matching capability. We also investigate the effects of varied observation length on signal detection and classification. It is found that the CDP-based detector and the HMM-based classifier can detect and classify incoming signals at a range of low SNRs. <s> BIB005 </s> A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> This paper addresses the problem of automatic modulation recognition of digital signals. A classification method based on spectral correlation and Support Vector Machine (SVM) is developed. The spectral correlation theory is introduced and several characteristic parameters which can be used for modulation analysis are extracted. The parameters are used as the input feature vectors to SVM. SVM maps the vectors into a high dimensional feature space, so the problem of non-separable classification in low dimension is resolved and the decision threshold become unnecessary. The experiment results show that the algorithm is robust with high accuracy even at low SNR. <s> BIB006 </s> A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> Spectrum sensing is one of the most challenging functions in cognitive radio system. Detection of the presence of signals and distinction of the type of signals in a particular frequency band are critical for cognitive radios to adapt to the radio environment. In this paper, a novel approach to signal classification combining spectral correlation analysis and support vector machine (SVM) is introduced. Four spectral coherence characteristic parameters are chosen via spectral correlation analysis. By utilizing a nonlinear SVM, a significant amount of calculation is performed offline, thus the computational complexity is reduced. Simulations indicate that the overall success rate is above 92.8% with data length of 1000 when SNR is equal to 4 dB. Compared to the existing methods including the classifiers based on binary decision tree (BDT) and multilayer linear perceptron network (MLPN), the proposed approach is more effective in the case of low SNR and limited training numbers. <s> BIB007 </s> A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> This paper presents modulation classification method capable of classifying incident digital signals without a priori information using WT key features and SVM. These key features for modulation classification should have good properties of sensitive with modulation types and insensitive with SNR variation. In this paper, the 4 key features using WT coefficients, which have the property of insensitive to the changing of noise, are selected. The numerical simulations using these features are performed. We investigate the performance of the SVM-DDAG classifier for classifying 8 digitally modulated signals using only 4 WT key features (i.e., 4 level scale), and compare with that of decision tree classifier to adapt the modulation classification module in software radio. Results indicated an overall success rate of 95% at the SNR of 10dB in SVM-DDAG classifier on an AWGN channel. <s> BIB008 </s> A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> Adaptive modulation schemes have been proposed to optimize Shannon's channel capacity in recent orthogonal frequency division multiplexing (OFDM) based broadband wireless standard proposals. By adapting the modulation type (effectively changing the number of bits per symbol) at the transmitter end one can improve the bit error rate (BER) during transmission at designated SNR. Blind detection of the transmitted modulation type is desirable to optimise the bandwidth available at the receivers. Hence, there is a need for an intelligent modulation classification engine at the receiver end. In this work, we evaluate some higher order statistical measures coupled with a classical Naive Bayes classifier for fast identification of adaptive modulation schemes. We also benchmark the experimental results with the optimal Maximum Likelihood Classifier, and Support Vector Machine based Classifier using the same feature set. <s> BIB009 </s> A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> The paper presents a new algorithm of digital modulation signals based on high order cumulants (HOC) and support vector machines (SVM). The parameters which are picked up from the signals' fourth order and sixth order cumulants are used as the classification feature vectors. Using SVM based on binary tree as classifiers, recognition of the 2ASK, 4ASK, QPSK, 2FSK and 4FSK signals is efficient. The computer simulation results justify that the success rate is over 97.5% at SNR=10 dB. <s> BIB010 </s> A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> Optimizing the coexistence performance of a cognitive radio (CR) greatly depends on the extent of its awareness of the radio environment. The more knowledge a CR can acquire about primary user (PU) systems, the better it can plan its radio activities. Ideally, it would like to classify the PU systems with respect to ‘known standards’ so that the documented knowledge of PU systems can be usefully utilized. Intensive research has been done in the area of signal classification but merely with respect to modulations. We present a novel neuro-fuzzy signal classifier (NFSC) to classify signals with respect to known standards. The NFSC demonstrates excellent performance by using modern A/D based wideband data acquisition as well as traditional heterodyne narrowband transceivers in real-time coexistence environments. <s> BIB011 </s> A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> In this paper we consider the problem of Radar Specific Emitter Identification (SEI) with the aim of distinguishing among several transmitting sources of the same kind which is a very hot topic in the device forensic field. At the design stage, we introduce a classification technique based on some suitable features evaluated from the cumulants of the signal emitted by the radar system. The devised features share some invariance properties which make them very attractive for the SEI problem. Hence, we use them as the input to a K-Nearest Neighbor (KNN) classifier which performs the assignment of the emitter to a specific class. At the analysis stage, we assess the performance of the new system on a real dataset containing radar signals from three identical airborne emitters. The results highlight that a satisfactory classification performance is achieved. <s> BIB012 </s> A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> This work presents a novel method for automatic modulation classification based on discriminative learning. The features are the ordered magnitude and phase of the received symbols at the output of the matched filter. The results using the proposed front end and support vector machines are compared to other techniques. Frequency offset is also considered and the results show that in this condition the new method significantly outperforms two cumulant-based classifiers. <s> BIB013 </s> A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> Automatic Modulation Classification (AMC) is an intermediate step between signal detection and demodulation. It is a very important process for a receiver that has no, or limited, knowledge of received signals. It is important for many areas such as spectrum management, interference identification and for various other civilian and military applications. This paper explores the use of Genetic Programming (GP) in combination with K-nearest neighbor (KNN) for AMC. KNN has been used to evaluate fitness of GP individuals during the training phase. Additionally, in the testing phase, KNN has been used for deducing the classification performance of the best individual produced by GP. Four modulation types are considered here: BPSK, QPSK, QAM16 and QAM64. Cumulants have been used as input features for GP. The classification process has been divided into two-stages for improving the classification accuracy. Simulation results demonstrate that the proposed method provides better classification performance compared to other recent methods. <s> BIB014 </s> A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> In recent years, deep artificial neural networks (including recurrent ones) have won numerous contests in pattern recognition and machine learning. This historical survey compactly summarizes relevant work, much of it from the previous millennium. Shallow and Deep Learners are distinguished by the depth of their credit assignment paths, which are chains of possibly learnable, causal links between actions and effects. I review deep supervised learning (also recapitulating the history of backpropagation), unsupervised learning, reinforcement learning & evolutionary computation, and indirect search for short programs encoding deep and large networks. <s> BIB015 </s> A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> In this paper, a new automatic modulation classification system using a polynomial classifier is proposed. The feature set used in the proposed system is comprised of second, fourth and sixth higher order cumulants of the received signal. The polynomial classifier expands the feature vector into a higher dimensional space in which the different modulation types are linearly separable. The performance of the proposed system is tested against BPSK, QPSK, 8-PSK, 16-QAM and 64-QAM constellations. Results showed a relatively higher classification rate for all types of modulations compared to previously published results using fourth and sixth order cumulants with other classification schemes. Moreover, the proposed classification scheme has low computational complexity and is more suited for real time applications. <s> BIB016 </s> A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors. <s> BIB017 </s> A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> A technique for joint modulation format and bit-rate classification is proposed.The proposed technique can simultaneously enable non-data-aided SNR estimation.The technique uses asynchronous delay-tap plots with artificial neural networks.The signal classification accuracy is 99.12% and mean SNR estimation error is 0.88?dB.Due to its simplicity, it is attractive for future cognitive wireless networks. We propose a novel technique for automatic classification of modulation formats/bit-rates of digitally modulated signals as well as non-data-aided (NDA) estimation of signal-to-noise ratio (SNR) in wireless networks. The proposed technique exploits modulation format, bit-rate, and SNR sensitive features of asynchronous delay-tap plots (ADTPs) for the joint estimation of these parameters. Simulation results validate successful classification of three commonly-used modulation formats at two different bit-rates with an overall accuracy of 99.12%. Similarly, in-service estimation of SNR in the range of 0-30 dB is demonstrated with mean estimation error of 0.88 dB. The proposed technique requires low-speed asynchronous sampling of signal envelope and hence, it can enable simple and cost-effective joint modulation format/bit-rate classification and NDA SNR estimation in future wireless networks. Display Omitted <s> BIB018 </s> A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> This paper presents a simplified automatic modulation classification system based on stepwise regression and Hierarchical Polynomial Classifier. The proposed system applies High Order Cumulants of the received signal to a polynomial network in order to calculate their second order polynomial expansion. Then, stepwise regression model is used as a feature selection system; whereas important features are identified and used for the classification problem and the insignificant features are discarded. With this optimization step, the complexity of the classifier is reduced significantly while maintaining the same classification accuracy as the original Hierarchical Polynomial Classifier with all the expanded features. <s> BIB019 </s> A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> Deep neural network (DNN) has recently received much attention due to its superior performance in classifying data with complex structure. In this paper, we investigate application of DNN technique to automatic classification of modulation classes for digitally modulated signals. First, we select twenty one statistical features which exhibit good separation in empirical distributions for all modulation formats considered (i.e., BPSK, QPSK, 8PSK, 16QAM, and 64QAM). These features are extracted from the received signal samples and used as the input to the fully connected DNN with three hidden layer. The training data containing 25,000 feature vectors is generated by the computer simulation under both additive Gaussian white noise (AWGN) and Rician fading channels. Our test results show that the proposed method brings dramatic performance improvement over the existing classifier especially for high Doppler fading channels. <s> BIB020 </s> A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> A novel algorithm for simultaneous modulation format/bit-rate classification and non-data-aided (NDA) signal-to-noise ratio (SNR) estimation in multipath fading channels by applying deep machine learning-based pattern recognition on signals’ asynchronous delay-tap plots (ADTPs) is proposed. The results for three widely-used modulation formats at two different bit-rates demonstrate classification accuracy of 99.8%. In addition, NDA SNR estimation over a wide range of 0−30 dB is shown with mean error of 1 dB. The proposed method requires low-speed, asynchronous sampling of signal and is thus ideal for low-cost multiparameter estimation under real-world channel conditions. <s> BIB021 </s> A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> Automated Modulation Classification (AMC) has been applied in various emerging areas such as cognitive radio (CR). In our paper, we propose a deep learning-based AMC method that employs Spectral Correlation Function (SCF). In our proposed method, one deep learning technology, Deep Belief Network (DBN), is applied for pattern recognition and classification. By using noise-resilient SCF signatures and DBN that is effective in learning complex patterns, we achieve high accuracy in modulation detection and classification even in the presence of environment noise. Our simulation results illustrate the efficiency of our proposed method in classifying 4FSK, 16QAM, BPSK, QPSK, and OFDM modulation techniques in various environments. <s> BIB022 </s> A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> Modulation identification shows great significance for any receiver that has little knowledge of the modulation scheme of the received signal. In this paper, we compare the performance of a deep autoencoder network and three shallow algorithms including SVM, Naive Bayes and BP neural network in the field of communication signal modulation recognition. Firstly, cyclic spectrum is used to pre-process the simulation communication signals, which are at various SNR (from −10dB to 10dB). Then, a deep autoencoder network is established to approximate the internal properties from great amount of data. A softmax regression model is used as a classifier to identify the five typical communication signals, which are FSK, PSK, ASK, MSK, QAM. The results for the experiment illustrate the excellent classification performance of the networks. At last, we discuss the comparison of these methods and three traditional shallow machine learning models. <s> BIB023 </s> A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> Abstract Recently, deep learning has received a lot of attention in many machine learning applications for its superior classification performance in speech recognition, natural language understanding and image processing. However, it still lacks attention in automatic modulation classification (AMC) until now. Here, we introduce the application of deep learning in AMC. We propose a fully connected 2 layer feed-forward deep neural network (DNN) with layerwise unsupervised pretraining for the classification of digitally modulated signals in various channel conditions. The system uses independent autoencoders (AEs) for feature learning with multiple hidden nodes. Signal information from the received samples is extracted and preprocessed via I and Q components, and formed into training input to 1st AE layer. A probabilistic based method is employed at the output layer to detect the correct modulation signal. Simulation results show that a significant improvement can be achieved compared to the other conventional machine learning methods in the literature. Moreover, we also show that our proposed method can extract the features from cyclic-stationary data samples. A good classification accuracy was achieved, even when the proposed deep network is trained and tested at different SNRs. This shows the future potential of the deep learning model for application to AMC. <s> BIB024 </s> A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> In this paper, we propose a low-complexity graphic constellation projection (GCP) algorithm for automatic modulation classification (AMC), where the recovered symbols are projected into artificial graphic constellations. Unlike the existing feature- based (FB) algorithms, we convert the AMC problem into an image recognition problem. Subsequently, the deep belief network (DBN) is adopted to learn the underlying features in these constellations and recognize their corresponding modulation schemes. Simulation results demonstrate that the proposed GCP-DBN based AMC system achieves better performance than other schemes. Specifically, the classification accuracy is beyond 95% at 0 dB, which is very close to the average likelihood ratio test upper bound (ALRT-Upper Bound). <s> BIB025 </s> A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> We survey the latest advances in machine learning with deep neural networks by applying them to the task of radio modulation recognition. Results show that radio modulation recognition is not limited by network depth and further work should focus on improving learned synchronization and equalization. Advances in these areas will likely come from novel architectures designed for these tasks or through novel training methods. <s> BIB026 </s> A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> The future wireless communication will come up with a strict requirement on high spectral efficiency, developing novel algorithms for spectrum sensing with deep sensing capability will be more challenging. However, traditional expert feature-based spectrum sensing algorithms are lack of sufficient capability of self-learning and adaptability to unknown environments and complex cognitive tasks. To address this problem, we propose to build up a deep learning network to learn short time-frequency transformation (STFT), a basic entity of traditional spectrum sensing algorithms. Spectrum sensing based on the learning to STFT network is supposed to automatically extract features for communication signals and makes decisions for complex cognitive tasks meanwhile. The feasibility and performances of the designed learning network are verified by classifying signal modulation types in deep spectrum sensing applications. <s> BIB027 </s> A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> Modulation identification of the transmitted signals remain a challenging area in modern intelligent communication systems like cognitive radios. The computation of the distinct features from input data set and applying machine learning algorithms has been a well-known method in the classification of such signals. However, recently, deep neural networks, a branch of machine learning, have gained significant attention in the pattern recognition of complex data due to its superior performance. Here, we test the application of deep neural networks to the automatic modulation classification in AWGN and flat-fading channel. Three training inputs were used; mainly 1) In-phase and quadrature (I-Q) constellation points, 2) the centroids of constellation points employing the fuzzy C-means algorithm to I-Q diagrams, and 3) the high-order cumulants of received samples. The unsupervised learning from these data sets was done using the sparse autoencoders and a supervised softmax classifier was employed for the classification. The designing parameters for training single and 2-layer sparse autoencoders are proposed and their performance compared with each other. The results show that a very good classification rate is achieved at a low SNR of 0 dB. This shows the potential of the deep learning model for the application of modulation classification. Method for AMC using powerful capability of deep networks.Comparison between a shallow and deep network in the application of AMC.2-layered deep neural network model outperforms other networks.The accuracy of this model converges to results obtained from conventional methods under flat-fading channels. <s> BIB028 </s> A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> Automated modulation classification plays a very important part in cognitive radio networks. Deep learning is also a powerful tool that we could not overlook its potential in addressing signal modulation recognition problem. In our last work, we propose a new data conversion algorithm in order to gain a better classification accuracy of communication signal modulation, but we still believe that the convolution neural network (CNN) can work better. However, its application to signal modulation recognition is often hampered by insufficient data and overfitting. Here, we propose a smart approach to programmatic data augmentation method by using the auxiliary classifier generative adversarial networks (ACGANs). The famous CNN model, AlexNet, has been utilized to be the classifier and ACGAN to be the generator, which will enlarge our data set. In order to alleviate the common issues in the traditional generative adversarial nets training, such as discriminator overfitting, generator disconverge, and mode collapse, we apply several training tricks in our training. With the result on original data set as our baseline, we will evaluate our result on enlarged data set to validate the ACGAN’s performance. The result shows that we can gain 0.1~6% increase in the classification accuracy in the ACGAN-based data set. <s> BIB029 </s> A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> Deep Learning (DL) is such a powerful tool that we have seen tremendous success in areas such as Computer Vision, Speech Recognition, and Natural Language Processing. Since Automated Modulation Classification (AMC) is an important part in Cognitive Radio Networks, we try to explore its potential in solving signal modulation recognition problem. It cannot be overlooked that DL model is a complex model, thus making them prone to over-fitting. DL model requires many training data to combat with over-fitting, but adding high quality labels to training data manually is not always cheap and accessible, especially in real-time system, which may counter unprecedented data in dataset. Semi-supervised Learning is a way to exploit unlabeled data effectively to reduce over-fitting in DL. In this paper, we extend Generative Adversarial Networks (GANs) to the semi-supervised learning will show it is a method can be used to create a more data-efficient classifier. <s> BIB030 </s> A Survey on Deep Learning Techniques in Wireless Signal Recognition <s> Definition of DL Problem. <s> Deep learning has recently attracted much attention due to its excellent performance in processing audio, image, and video data. However, few studies are devoted to the field of automatic modulation classification (AMC). It is one of the most well-known research topics in communication signal recognition and remains challenging for traditional methods due to complex disturbance from other sources. This paper proposes a heterogeneous deep model fusion (HDMF) method to solve the problem in a unified framework. The contributions include the following: (1) a convolutional neural network (CNN) and long short-term memory (LSTM) are combined by two different ways without prior knowledge involved; (2) a large database, including eleven types of single-carrier modulation signals with various noises as well as a fading channel, is collected with various signal-to-noise ratios (SNRs) based on a real geographical environment; and (3) experimental results demonstrate that HDMF is very capable of coping with the AMC problem, and achieves much better performance when compared with the independent network. <s> BIB031
The concept of DL originates from the research on artificial neural network BIB015 and the goal is to understand data by mimicking the mechanism of the human brain BIB017 . The basic neural network framework consists of three parts: input, hidden, and output layer and is shown in Figure 2 . Hidden layers maybe one layer or multilayer, and each layer consists of several nodes. The BIB001 BIB003 (ii) KNN BIB012 BIB014 (iii) SVM BIB010 BIB013 BIB006 BIB007 BIB008 (iv) Naïve Bayes BIB009 (v) HMM BIB005 (vi) Fuzzy classifier BIB002 (vii) Polynomial classifier BIB016 BIB019 (i) DNN BIB020 BIB018 BIB021 BIB024 (ii) DBN BIB022 BIB025 (iii) CNN [17, 19-21, 54, 64, 65, 70, 73-76, 79, 81, 82, 95, 96] (iv) LSTM  BIB026 (v) CRBM BIB027 (vi) Autoencoder network BIB023 BIB028 (vii) Generative adversarial networks BIB029 BIB030 (viii) HDMF BIB031  (ix) NFSC BIB011 Pros (i) works better on small data (ii) low implementation cost node presented in Figure 3 is the basic operational unit, in which the input vector is multiplied by a series of weights and the sum value is fed into the activation function . These operational units contribute to a powerful network, which could realize complex functions such as regression and classification. In fact, the study of DL makes substantial progress by Hinton in 2006 BIB004 , which causes a great sensation. DL architecture is comprised of many stacked layers of neural networks, emphasizing the learning from successive layers to obtain more meaningful information or high-level representations. The fundamental idea is to utilize feedback information to iteratively optimize weight value in multilayer neural networks. Moreover, the layers with a relatively large scale will bring about amazing effects. Due to the superior performance, DL has been employed on a wide variety of tasks including CV, MT and NLP.