Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> C. CNN for machine health monitoring <s> Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique. Given an appropriate network architecture, gradient-based learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional neural networks, which are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation recognition, and language modeling. A new learning paradigm, called graph transformer networks (GTN), allows such multimodule systems to be trained globally using gradient-based methods so as to minimize an overall performance measure. Two systems for online handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of graph transformer networks. A graph transformer network for reading a bank cheque is also described. It uses convolutional neural network character recognizers combined with global training techniques to provide record accuracy on business and personal cheques. It is deployed commercially and reads several million cheques per day. <s> BIB001 </s> Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> C. CNN for machine health monitoring <s> Abstract Vibration analysis is a well-established technique for condition monitoring of rotating machines as the vibration patterns differ depending on the fault or machine condition. Currently, mainly manually-engineered features, such as the ball pass frequencies of the raceway, RMS, kurtosis an crest, are used for automatic fault detection. Unfortunately, engineering and interpreting such features requires a significant level of human expertise. To enable non-experts in vibration analysis to perform condition monitoring, the overhead of feature engineering for specific faults needs to be reduced as much as possible. Therefore, in this article we propose a feature learning model for condition monitoring based on convolutional neural networks. The goal of this approach is to autonomously learn useful features for bearing fault detection from the data itself. Several types of bearing faults such as outer-raceway faults and lubrication degradation are considered, but also healthy bearings and rotor imbalance are included. For each condition, several bearings are tested to ensure generalization of the fault-detection system. Furthermore, the feature-learning based approach is compared to a feature-engineering based approach using the same data to objectively quantify their performance. The results indicate that the feature-learning system, based on convolutional neural networks, significantly outperforms the classical feature-engineering based approach which uses manually engineered features and a random forest classifier. The former achieves an accuracy of 93.61 percent and the latter an accuracy of 87.25 percent. <s> BIB002 </s> Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> C. CNN for machine health monitoring <s> Prognostics technique aims to accurately estimate the Remaining Useful Life (RUL) of a subsystem or a component using sensor data, which has many real world applications. However, many of the existing algorithms are based on linear models, which cannot capture the complex relationship between the sensor data and RUL. Although Multilayer Perceptron (MLP) has been applied to predict RUL, it cannot learn salient features automatically, because of its network structure. A novel deep Convolutional Neural Network (CNN) based regression approach for estimating the RUL is proposed in this paper. Although CNN has been applied on tasks such as computer vision, natural language processing, speech recognition etc., this is the first attempt to adopt CNN for RUL estimation in prognostics. Different from the existing CNN structure for computer vision, the convolution and pooling filters in our approach are applied along the temporal dimension over the multi-channel sensor data to incorporate automated feature learning from raw sensor signals in a systematic way. Through the deep architecture, the learned features are the higher-level abstract representation of low-level raw sensor signals. Furthermore, feature learning and RUL estimation are mutually enhanced by the supervised feedback. We compared with several state-of-the-art algorithms on two publicly available data sets to evaluate the effectiveness of this proposed approach. The encouraging results demonstrate that our proposed deep convolutional neural network based regression approach for RUL estimation is not only more efficient but also more accurate. <s> BIB003 </s> Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> C. CNN for machine health monitoring <s> Abstract Traditional artificial methods and intelligence-based methods of classifying and diagnosing various mechanical faults with high accuracy by extracting effective features from vibration data, such as support vector machines and back propagation neural networks, have been widely investigated. However, the problems of extracting features automatically without significantly increasing the demand for machinery expertise and maximizing accuracy without overcomplicating machine structure have to date remained unsolved. Therefore, a novel hierarchical learning rate adaptive deep convolution neural network based on an improved algorithm was proposed in this study, and its use to diagnose bearing faults and determine their severity was investigated. To test the effectiveness of the proposed method, an experiment was conducted with bearing-fault data samples obtained from a test rig. The method achieved a satisfactory performance in terms of both fault-pattern recognition and fault-size evaluation. In addition, comparison revealed that the improved algorithm is well suited to the fault-diagnosis model, and that the proposed method is superior to other existing methods. <s> BIB004 </s> Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> C. CNN for machine health monitoring <s> Feature extraction is an important step in conventional vibration-based fault diagnosis methods. However, the features are usually empirically extracted, leading to inconsistent performance. This paper presents a new automatic and intelligent fault diagnosis method based on convolution neural network. Firstly, the vibration signal is processed by wavelet transform into a multi-scale spectrogram image to manifest the fault characteristics. Next, the spectrogram image is directly fed into convolution neural network to learn the invariant representation for vibration signal and recognize the fault status for fault diagnosis. During model construction, rectifier neural activation function and dropout layer are incorporated into convolution neural network to improve the computational efficiency and model generalization. Training data is input into traditional convolutional neural network, ReLU network, Dropout network and enhanced convolutional neural network. The classification results are reached by inputting training data and test data. Then, comparison is made on the analytical results of the four networks to conclude that the preciseness of the classification result of the enhanced convolutional neural network achieves as high as 96%, 8% higher than traditional convolutional neural network. Through adjusting p, the holding probability of Dropout, 3 kinds of sparse neural networks are trained and the classification results are compared. It finds, when p=0.4, the enhanced convolutional neural network achieves the best classification performance, 5% and 4% higher than ReLU network and Dropout network respectively. <s> BIB005 </s> Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> C. CNN for machine health monitoring <s> Abstract Fast and reliable industrial inspection is a main challenge in manufacturing scenarios. However, the defect detection performance is heavily dependent on manually defined features for defect representation. In this contribution, we investigate a new paradigm from machine learning, namely deep machine learning by examining design configurations of deep Convolutional Neural Networks (CNN) and the impact of different hyper-parameter settings towards the accuracy of defect detection results. In contrast to manually designed image processing solutions, deep CNN automatically generate powerful features by hierarchical learning strategies from massive amounts of training data with a minimum of human interaction or expert process knowledge. An application of the proposed method demonstrates excellent defect detection results with low false alarm rates. <s> BIB006 </s> Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> C. CNN for machine health monitoring <s> Early detection of the motor faults is essential and artificial neural networks are widely used for this purpose. The typical systems usually encapsulate two distinct blocks: feature extraction and classification. Such fixed and hand-crafted features may be a suboptimal choice and require a significant computational cost that will prevent their usage for real-time applications. In this paper, we propose a fast and accurate motor condition monitoring and early fault-detection system using 1-D convolutional neural networks that has an inherent adaptive design to fuse the feature extraction and classification phases of the motor fault detection into a single learning body. The proposed approach is directly applicable to the raw data (signal), and, thus, eliminates the need for a separate feature extraction algorithm resulting in more efficient systems in terms of both speed and hardware. Experimental results obtained using real motor data demonstrate the effectiveness of the proposed method for real-time motor condition monitoring. <s> BIB007 </s> Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> C. CNN for machine health monitoring <s> Considering various health conditions under varying operational conditions, the mining sensitive feature from the measured signals is still a great challenge for intelligent fault diagnosis of spindle bearings. This paper proposed a novel energy-fluctuated multiscale feature mining approach based on wavelet packet energy (WPE) image and deep convolutional network (ConvNet) for spindle bearing fault diagnosis. Different from the vector characteristics applied in intelligent diagnosis of spindle bearings, wavelet packet transform is first combined with phase space reconstruction to rebuild a 2-D WPE image of the frequency subspaces. This special image can reconstruct the local relationship of the WP nodes and hold the energy fluctuation of the measured signal. Then, the identifiable characteristics can be further learned by a special architecture of the deep ConvNet. Other than the traditional neural network architecture, to maintain the global and local information simultaneously, deep ConvNet combines the skipping layer with the last convolutional layer as the input of the multiscale layer. The comparisons of clustering distribution and classification accuracy with six other features show that the proposed feature mining approach is quite suitable for spindle bearing fault diagnosis with multiclass classification regardless of the load fluctuation. <s> BIB008 </s> Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> C. CNN for machine health monitoring <s> Abstract Structural health monitoring (SHM) and vibration-based structural damage detection have been a continuous interest for civil, mechanical and aerospace engineers over the decades. Early and meticulous damage detection has always been one of the principal objectives of SHM applications. The performance of a classical damage detection system predominantly depends on the choice of the features and the classifier. While the fixed and hand-crafted features may either be a sub-optimal choice for a particular structure or fail to achieve the same level of performance on another structure, they usually require a large computation power which may hinder their usage for real-time structural damage detection. This paper presents a novel, fast and accurate structural damage detection system using 1D Convolutional Neural Networks (CNNs) that has an inherent adaptive design to fuse both feature extraction and classification blocks into a single and compact learning body. The proposed method performs vibration-based damage detection and localization of the damage in real-time. The advantage of this approach is its ability to extract optimal damage-sensitive features automatically from the raw acceleration signals. Large-scale experiments conducted on a grandstand simulator revealed an outstanding performance and verified the computational efficiency of the proposed real-time damage detection method. <s> BIB009
In some scenarios, machinery data can be presented in a 2D format such as time-frequency spectrum, while in some scenarios, they are in a 1D format, i.e., time-series. Therefore, CNNs models are able to learn complex and robust representation via its convolutional layer. Intuitively, filters in convolutional layers can extract local patterns in raw data and stacking these convolutional layers can further build complex patterns. Janssens et al. utilized a 2D-CNN model for four categories rotating machinery conditions recognition, whose input is DFT of two accelerometer signals from two sensors that are placed perpendicular to each other. Therefore, the height of input is the number of sensors. The adopted CNN model contains one convolutional layer and a fully connected layer. Then, the top softmax layer is adopted for classification BIB002 . In BIB003 , Babu et al. built a 2D deep convolution neural network to predict the RUL of system based on normalizedvariate time series from sensor signals, in which one dimension of the 2D input is number of sensors as the setting reported in BIB002 . In their model, average pooling is adopted instead of max pooling. Since RUL is a continuous value, the top layer was linear regression layer. Ding et al. proposed a deep Convolutional Network (ConvNet) where wavelet packet energy (WPE) image were used as input for spindle bearing fault diagnosis BIB008 . To fully discover the hierarchical representation, a multiscale layer was added after the last convolutional layer, which concatenates the outputs of the last convolutional layer and the ones of the previous pooling layer. Guo et al. proposed a hierarchical adaptive deep convolution neural network (ADCNN) BIB004 . Firstly, the input time series data as a signal-vector was transformed into a 32 × 32 matrix, which follows the typical input format adopted by LeNet BIB001 . In addition, they designed a hierarchical framework to recognize fault patterns and fault size. In the fault pattern decision module, the first ADCNN was adopted to recognize fault type. In the fault size evaluation layer, based on each fault type, ADCNN with the same structure was used to predict fault size. Here, the classification mechanism is still used. The predicted value f is defined as the probability summation of the typical fault sizes as follows: where [p 1 , . . . , p c ] is produced by the top softmax layer, which denote the probability score that each sample belongs to each class size and a j is the fault size corresponding to the j-th fault size. In BIB005 , an enhanced CNN was proposed for machinery fault diagnosis. To pre-process vibration data, morlet wavelet was used to decompose the vibration signal and obtain wavelet scaleogram. Then, bilinear interpolation was used to rescale the scaleogram into a grayscale image with a size of 32 × 32. BIB006 . In one specific application: industrial optical inspection, two directions of model configurations including depth (addition of conv-layer) and width (increase of number filters) were investigated. The optimal configuration verified empirically has been presented in Table II . In  , CNN was applied in the field of diagnosing the early small faults of front-end controlled wind generator (FSCWG) that the 784×784 input matrix consists of vibration data of generator input shaft (horizontal) and vibration data of generator output shaft (vertical) in time scale. As reviewed in our above section II-C, CNN can also be applied to 1D time series signal and the corresponding operations have been elaborated. In BIB007 , the 1D CNN was successfully developed on raw time series data for motor fault detection, in which feature extraction and classification were integrated together. The corresponding framework has been shown in Figure 8 . Abdeljaber et al. proposed 1D CNN on normalized vibration signal, which can perform vibrationbased damage detection and localization of the structural damage in real-time. The advantage of this approach is its ability to extract optimal damage-sensitive features automatically from the raw acceleration signals, which does not need any additional preporcessing or signal processing approaches BIB009 . To present an overview about all these above CNN models that have been successfully applied in the area of MHMS, their architectures have been summarized in Table II . To explain the used abbreviation, the structure of CNN applied in Weimer's work BIB006 is denoted as Input It means the input 2D data is 32 × 32 and the CNN firstly applied 2 convolutional layers with the same design that the filter number is 64 and the filter size is 3 × 3, then stacked one max-pooling layer whose pooling size is 2 × 2, then applied 3 convolutional layers with the same design that the filter number is 128 and the filer size is 3×3, then applied a pooling layer whose pooling size is 2 × 2, and finally adopted two fully-connected layers whose hidden neuron numbers are both 1024. It should be noted that the size of output layer is not given here, considering it is task-specific and usually set to be the number of categories.