Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> arXiv:1612.07640v1 [cs.LG] 16 Dec 2016 <s> The sensitivity of various features that are characteristic of a machine defect may vary considerably under different operating conditions. Hence it is critical to devise a systematic feature selection scheme that provides guidance on choosing the most representative features for defect classification. This paper presents a feature selection scheme based on the principal component analysis (PCA) method. The effectiveness of the scheme was verified experimentally on a bearing test bed, using both supervised and unsupervised defect classification approaches. The objective of the study was to identify the severity level of bearing defects, where no a priori knowledge on the defect conditions was available. The proposed scheme has shown to provide more accurate defect classification with fewer feature inputs than using all features initially considered relevant. The result confirms its utility as an effective tool for machine health assessment. <s> BIB001 </s> Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> arXiv:1612.07640v1 [cs.LG] 16 Dec 2016 <s> Real-time health monitoring of industrial components and systems that can detect, classify and predict impending faults is critical to reducing operating and maintenance cost. This paper presents a logistic regression based prognostic method for on-line performance degradation assessment and failure modes classification. System condition is evaluated by processing the information gathered from controllers or sensors mounted at different points in the system, and maintenance is performed only when the failure/ malfunction prognosis indicates instead of periodic maintenance inspections. The wavelet packet decomposition technique is used to extract features from non-stationary signals (such as current, vibrations), wavelet package energies are used as features and Fisher's criteria is used to select critical features. Selected features are input into logistic regression (LR) models to assess machine performance and identify possible failure modes. The maximum likelihood method is used to determine parameters of LR models. The effectiveness and feasibility of this methodology have been illustrated by applying the method to a real elevator door system. <s> BIB002 </s> Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> arXiv:1612.07640v1 [cs.LG] 16 Dec 2016 <s> We show how to use "complementary priors" to eliminate the explaining-away effects that make inference difficult in densely connected belief nets that have many hidden layers. Using complementary priors, we derive a fast, greedy algorithm that can learn deep, directed belief networks one layer at a time, provided the top two layers form an undirected associative memory. The fast, greedy algorithm is used to initialize a slower learning procedure that fine-tunes the weights using a contrastive version of the wake-sleep algorithm. After fine-tuning, a network with three hidden layers forms a very good generative model of the joint distribution of handwritten digit images and their labels. This generative model gives better digit classification than the best discriminative learning algorithms. The low-dimensional manifolds on which the digits lie are modeled by long ravines in the free-energy landscape of the top-level associative memory, and it is easy to explore these ravines by using the directed connections to display what the associative memory has in mind. <s> BIB003 </s> Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> arXiv:1612.07640v1 [cs.LG] 16 Dec 2016 <s> Recently, the issue of machine condition monitoring and fault diagnosis as a part of maintenance system became global due to the potential advantages to be gained from reduced maintenance costs, improved productivity and increased machine availability. This paper presents a survey of machine condition monitoring and fault diagnosis using support vector machine (SVM). It attempts to summarize and review the recent research and developments of SVM in machine condition monitoring and diagnosis. Numerous methods have been developed based on intelligent systems such as artificial neural network, fuzzy expert system, condition-based reasoning, random forest, etc. However, the use of SVM for machine condition monitoring and fault diagnosis is still rare. SVM has excellent performance in generalization so it can produce high accuracy in classification for machine condition monitoring and diagnosis. Until 2006, the use of SVM in machine condition monitoring and fault diagnosis is tending to develop towards expertise orientation and problem-oriented domain. Finally, the ability to continually change and obtain a novel idea for machine condition monitoring and fault diagnosis using SVM will be future works. <s> BIB004 </s> Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> arXiv:1612.07640v1 [cs.LG] 16 Dec 2016 <s> Previous work has shown that the difficulties in learning deep generative or discriminative models can be overcome by an initial unsupervised learning step that maps inputs to useful intermediate representations. We introduce and motivate a new training principle for unsupervised learning of a representation based on the idea of making the learned representations robust to partial corruption of the input pattern. This approach can be used to train autoencoders, and these denoising autoencoders can be stacked to initialize deep architectures. The algorithm can be motivated from a manifold learning and information theoretic perspective or from a generative model perspective. Comparative experiments clearly show the surprising advantage of corrupting the input of autoencoders on a pattern classification benchmark suite. <s> BIB005 </s> Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> arXiv:1612.07640v1 [cs.LG] 16 Dec 2016 <s> We present a new learning algorithm for Boltzmann machines that contain many layers of hidden variables. Data-dependent expectations are estimated using a variational approximation that tends to focus on a single mode, and dataindependent expectations are approximated using persistent Markov chains. The use of two quite different techniques for estimating the two types of expectation that enter into the gradient of the log-likelihood makes it practical to learn Boltzmann machines with multiple hidden layers and millions of parameters. The learning can be made more efficient by using a layer-by-layer “pre-training” phase that allows variational inference to be initialized with a single bottomup pass. We present results on the MNIST and NORB datasets showing that deep Boltzmann machines learn good generative models and perform well on handwritten digit and visual object recognition tasks. <s> BIB006 </s> Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> arXiv:1612.07640v1 [cs.LG] 16 Dec 2016 <s> In most of the industries related to mechanical engineering, the usage of pumps is high. Hence, the system which takes care of the continuous running of the pump becomes essential. In this paper, a vibration based condition monitoring system is presented for monoblock centrifugal pumps as it plays relatively critical role in most of the industries. This approach has mainly three steps namely feature extraction, classification and comparison of classification. In spite of availability of different efficient algorithms for fault detection, the wavelet analysis for feature extraction and Naive Bayes algorithm and Bayes net algorithm for classification is taken and compared. This paper presents the use of Naive Bayes algorithm and Bayes net algorithm for fault diagnosis through discrete wavelet features extracted from vibration signals of good and faulty conditions of the components of centrifugal pump. The classification accuracies of different discrete wavelet families were calculated and compared to find the best wavelet for the fault diagnosis of the centrifugal pump. <s> BIB007 </s> Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> arXiv:1612.07640v1 [cs.LG] 16 Dec 2016 <s> We classify digits of real-world house numbers using convolutional neural networks (ConvNets). Con-vNets are hierarchical feature learning neural networks whose structure is biologically inspired. Unlike many popular vision approaches that are hand-designed, ConvNets can automatically learn a unique set of features optimized for a given task. We augmented the traditional ConvNet architecture by learning multi-stage features and by using Lp pooling and establish a new state-of-the-art of 95.10% accuracy on the SVHN dataset (48% error improvement). Furthermore, we analyze the benefits of different pooling methods and multi-stage features in ConvNets. The source code and a tutorial are available at eblearn.sf.net. <s> BIB008 </s> Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> arXiv:1612.07640v1 [cs.LG] 16 Dec 2016 <s> The success of machine learning algorithms generally depends on data representation, and we hypothesize that this is because different representations can entangle and hide more or less the different explanatory factors of variation behind the data. Although specific domain knowledge can be used to help design representations, learning with generic priors can also be used, and the quest for AI is motivating the design of more powerful representation-learning algorithms implementing such priors. This paper reviews recent work in the area of unsupervised feature learning and deep learning, covering advances in probabilistic models, autoencoders, manifold learning, and deep networks. This motivates longer term unanswered questions about the appropriate objectives for learning good representations, for computing representations (i.e., inference), and the geometrical connections between representation learning, density estimation, and manifold learning. <s> BIB009 </s> Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> arXiv:1612.07640v1 [cs.LG] 16 Dec 2016 <s> This monograph provides an overview of general deep learning methodology and its applications to a variety of signal and information processing tasks. The application areas are chosen with the following three criteria in mind: (1) expertise or knowledge of the authors; (2) the application areas that have already been transformed by the successful use of deep learning technology, such as speech recognition and computer vision; and (3) the application areas that have the potential to be impacted significantly by deep learning and that have been experiencing research growth, including natural language and text processing, information retrieval, and multimodal information processing empowered by multi-task deep learning. <s> BIB010 </s> Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> arXiv:1612.07640v1 [cs.LG] 16 Dec 2016 <s> The features extracted from multisensory measurements can be used to characterize machinery conditions. However, the nonlinearity and uncertainty presented in machinery degradation process pose challenges on feature selection and fusion in machinery condition monitoring. To alleviate these issues, this paper presents a new probabilistic nonlinear feature selection and fusion method, named probabilistic kernel factor analysis (PKFA). First, the mathematical structure of the PKFA is formulated incorporating kernel techniques on the basis of conventional factor analysis (FA). Next, a PKFA-based machining tool condition monitoring model with support vector regression is presented. The effectiveness of the scheme is experimentally verified on a machining tool testbed. The experimental results show that the proposed PKFA method provides more accurate tool condition prediction than using all initially extracted features and other feature selection techniques (e.g., kernel principal component analysis and conventional FA), and thus confirms its utility as an effective tool for machining tool condition assessment. <s> BIB011 </s> Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> arXiv:1612.07640v1 [cs.LG] 16 Dec 2016 <s> Pervasiveness of ubiquitous computing advances the manufacturing scheme into a ubiquitous manufacturing era which poses significant challenges on sensing technology and system reliability. To improve manufacturing system reliability, this paper presents a new virtual tool wear sensing technique based on multisensory data fusion and artificial intelligence model for tool condition monitoring. It infers the difficult-to-measure tool wear parameters (e.g. tool wear width) by fusing in-process multisensory data (e.g. force, vibration, etc.) with dimension reduction technique and support vector regression model. Different state-of-the-art dimension reduction techniques including kernel principal component analysis, locally linear embedding, isometric feature mapping, and minimum redundancy maximum relevant method have been investigated for feature fusion in a virtual sensing model, and the kernel principal component analysis performs best in terms of sensing accuracy. The effectiveness of the developed virtual tool wear sensing technique is experimentally validated in a set of machining tool run-to-failure tests on a computer numerical control milling machine. The results show that the estimated tool wear width through virtual sensing is comparable to that measured offline by a microscope instrument in terms of accuracy, moreover, in a more cost-effective manner. A multi-sensory fusion based virtual tool wear sensing model is developed.KPCA performs best among the dimension reduction techniques investigated.The virtual tool wear sensing model is validated using a machining tool life test.The performance of the model is comparable to the costly offline instrumentation. <s> BIB012
Support Vector Machines (SVM), Naive Bayes (NB), logistic regression BIB004 , BIB002 , BIB007 . It is shown that the representation defines the upper-bound performances of machine learning algorithms BIB009 . However, it is difficult to know and determine what kind of good features should be designed. To alleviate this issue, feature extraction/selection methods, which can be regarded as a kind of information fusion, are performed between hand-crafted feature design and classification/regression models BIB001 , BIB012 , BIB011 . However, manually designing features for a complex domain requires a great deal of human labor and can not be updated on-line. At the same time, feature extraction/selection is another tricky problem, which involves prior selection of hyperparameters such as latent dimension. At last, the above three modules including feature design, feature extraction/selection and model training can not be jointly optimized which may hinder the final performance of the whole system. Deep learning based MHMS (DL-based MHMS) aim to extract hierarchical representations from input data by building deep neural networks with multiple layers of non-linear transformations. Intuitively, one layer operation can be regarded as a transformation from input values to output values. Therefore, the application of one layer can learn a new representation of the input data and then, the stacking structure of multiple layers can enable MHMS to learn complex concepts out of simpler concepts that can be constructed from raw input. In addition, DL-based MHMS achieve an end-to-end system, which can automatically learn internal representations from raw input and predict targets. Compared to conventional data driven MHMS, DL-based MHMS do not require extensive human labor and knowledge for handcrafted feature design. All model parameters including feature module and pattern classification/regression module can be trained jointly. Therefore, DL-based models can be applied to addressing machine health monitoring in a very general way. For example, it is possible that the model trained for fault diagnosis problem can be used for prognosis by only replacing the top softmax layer with a linear regression layer. The comparison between conventional data-driven MHMS and DL-based MHMS is given in Table I . A high-level illustration of the principles behind these three kinds of MHMS discussed above is shown in Figure 1 . Deep learning models have several variants such as AutoDncoders BIB005 , Deep Belief Network BIB003 , Deep Boltzmann Machines BIB006 , Convolutional Neural Networks BIB008 and Recurrent Neural Networks  . During recent years, various researchers have demonstrated success of these deep learning models in the application of machine health monitoring. This paper attempts to provide a wide overview on these latest DL-based MHMS works that impact the state-of-the art technologies. Compared to these frontiers of deep learning including Computer Vision and Natural Language Processing, machine health monitoring community is catching up and has witnessed an emerging research. Therefore, the purpose of this survey article is to present researchers and engineers in the area of machine health monitoring system, a global view of this hot and active topic, and help them to acquire basic knowledge, quickly apply deep learning models and develop novel DL-based MHMS. The remainder of this paper is organized as follows. The basic information on these above deep learning models are given in section II. Then, section III reviews applications of deep learning models on machine health monitoring. Finally, section IV gives a brief summary of the recent achievements of DL-based MHMS and discusses some potential trends of deep learning in machine health monitoring. II. DEEP LEARNING Originated from artificial neural network, deep learning is a branch of machine learning which is featured by multiple nonlinear processing layers. Deep learning aims to learn hierarchy representations of data. Up to date, there are various deep learning architectures and this research topic is fast-growing, in which new models are being developed even per week. And the community is quite open and there are a number of deep learning tutorials and books of good-quality BIB010 ,  . Therefore, only a brief introduction to some major deep learning techniques that have been applied in machine health monitoring is given. In the following, four deep architectures including Auto-encoders, RBM, CNN and RNN and their corresponding variants are reviewed, respectively.