Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> IV. SUMMARY AND FUTURE DIRECTIONS <s> The explosion of image data on the Internet has the potential to foster more sophisticated and robust models and algorithms to index, retrieve, organize and interact with images and multimedia data. But exactly how such data can be harnessed and organized remains a critical problem. We introduce here a new database called “ImageNet”, a large-scale ontology of images built upon the backbone of the WordNet structure. ImageNet aims to populate the majority of the 80,000 synsets of WordNet with an average of 500-1000 clean and full resolution images. This will result in tens of millions of annotated images organized by the semantic hierarchy of WordNet. This paper offers a detailed analysis of ImageNet in its current state: 12 subtrees with 5247 synsets and 3.2 million images in total. We show that ImageNet is much larger in scale and diversity and much more accurate than the current image datasets. Constructing such a large-scale database is a challenging task. We describe the data collection scheme with Amazon Mechanical Turk. Lastly, we illustrate the usefulness of ImageNet through three simple applications in object recognition, image classification and automatic object clustering. We hope that the scale, accuracy, diversity and hierarchical structure of ImageNet can offer unparalleled opportunities to researchers in the computer vision community and beyond. <s> BIB001 </s> Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> IV. SUMMARY AND FUTURE DIRECTIONS <s> A major assumption in many machine learning and data mining algorithms is that the training and future data must be in the same feature space and have the same distribution. However, in many real-world applications, this assumption may not hold. For example, we sometimes have a classification task in one domain of interest, but we only have sufficient training data in another domain of interest, where the latter data may be in a different feature space or follow a different data distribution. In such cases, knowledge transfer, if done successfully, would greatly improve the performance of learning by avoiding much expensive data-labeling efforts. In recent years, transfer learning has emerged as a new learning framework to address this problem. This survey focuses on categorizing and reviewing the current progress on transfer learning for classification, regression, and clustering problems. In this survey, we discuss the relationship between transfer learning and other related machine learning techniques such as domain adaptation, multitask learning and sample selection bias, as well as covariate shift. We also explore some potential future issues in transfer learning research. <s> BIB002 </s> Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> IV. SUMMARY AND FUTURE DIRECTIONS <s> Recent years have produced great advances in training large, deep neural networks (DNNs), including notable successes in training convolutional neural networks (convnets) to recognize natural images. However, our understanding of how these models work, especially what computations they perform at intermediate layers, has lagged behind. Progress in the field will be further accelerated by the development of better tools for visualizing and interpreting neural nets. We introduce two such tools here. The first is a tool that visualizes the activations produced on each layer of a trained convnet as it processes an image or video (e.g. a live webcam stream). We have found that looking at live activations that change in response to user input helps build valuable intuitions about how convnets work. The second tool enables visualizing features at each layer of a DNN via regularized optimization in image space. Because previous versions of this idea produced less recognizable images, here we introduce several new regularization methods that combine to produce qualitatively clearer, more interpretable visualizations. Both tools are open source and work on a pre-trained convnet with minimal setup. <s> BIB003 </s> Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> IV. SUMMARY AND FUTURE DIRECTIONS <s> This paper presents a transfer learning-based approach for bearing fault diagnosis, where the transfer strategy is proposed to improve diagnostic performance of the bearings under various operating conditions. The main idea of transfer learning is to utilize selective auxiliary data to assist target data classification, where a weight adjustment between them is involved in the TrAdaBoost algorithm for enhanced diagnostic capability. In addition, negative transfer is avoided through the similarity judgment, thus improving accuracy and relaxing computational load of the presented approach. Experimental comparison between transfer learning and traditional machine learning has verified the superiority of the proposed algorithm for bearing fault diagnosis. <s> BIB004 </s> Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> IV. SUMMARY AND FUTURE DIRECTIONS <s> Classification of imbalanced data is an important research problem as lots of real-world data sets have skewed class distributions in which the majority of data instances (examples) belong to one class and far fewer instances belong to others. While in many applications, the minority instances actually represent the concept of interest (e.g., fraud in banking operations, abnormal cell in medical data, etc.), a classifier induced from an imbalanced data set is more likely to be biased towards the majority class and show very poor classification accuracy on the minority class. Despite extensive research efforts, imbalanced data classification remains one of the most challenging problems in data mining and machine learning, especially for multimedia data. To tackle this challenge, in this paper, we propose an extended deep learning approach to achieve promising performance in classifying skewed multimedia data sets. Specifically, we investigate the integration of bootstrapping methods and a state-of-the-art deep learning approach, Convolutional Neural Networks (CNNs), with extensive empirical studies. Considering the fact that deep learning approaches such as CNNs are usually computationally expensive, we propose to feed low-level features to CNNs and prove its feasibility in achieving promising performance while saving a lot of training time. The experimental results show the effectiveness of our framework in classifying severely imbalanced data in the TRECVID data set. <s> BIB005 </s> Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> IV. SUMMARY AND FUTURE DIRECTIONS <s> Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers—8× deeper than VGG nets [40] but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions1, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation. <s> BIB006 </s> Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> IV. SUMMARY AND FUTURE DIRECTIONS <s> In the Internet-of-Things environment, it is critical to bridge the gap between business decision-making and real-time factory data to let companies transfer from condition-based maintenance service to predictive maintenance service. Condition monitoring systems have been widely applied to many industries to acquire operation and equipment related data, through which machine health state can be evaluated. One of the challenges of predicting future machine health lies in extracting the right features that are correlated well with the fault progression/degradation. We propose an enhanced restricted Boltzmann machine with a novel regularization term to automatically generate features that are suitable for remaining useful life prediction. The regularization term tries to maximize the trendability of the output features, which potentially better represent the degradation pattern of a system. The proposed method is benchmarked with regular restricted Boltzmann machine algorithm and principal component analysis. The generated features are used as input to a similarity-based method for life prediction. Run-to-failure datasets collected from two rotating systems are used for validation. <s> BIB007 </s> Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> IV. SUMMARY AND FUTURE DIRECTIONS <s> This paper addresses the cross-domain feature extraction and fusion from time-domain and frequency-domain with spectrum envelop preprocessing and time domain synchronization average principle using Transfer Component Analysis (TCA) for gearbox fault diagnosis. Considering TCA is developed based on kernel methods, the effects of different kernels including Gaussian kernel, Linear kernel, Polynomial kernel and PolyPlus kernel on the performance of TCA are investigated and evaluated in comprehensive experiments of gearbox testbed under various operating conditions. The experimental results show that the presented method can extract and fuse the cross-domain features of gearbox conditions by enhancing the reuse of historical data under various operating conditions efficiently, compared with other baseline dimension reduction methods. In addition, TCA with Gaussian kernel presents best performance, especially for low frequency levels of operation. <s> BIB008 </s> Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> IV. SUMMARY AND FUTURE DIRECTIONS <s> Binary Tree is integrated with Support Vector Data Description to address multi-classification issues with unbalanced datasets.Separability measure based on Mahalanobis distance is proposed to construct Binary Tree.The parameters of Support Vector Data Description are optimized using Particle Swarm Optimization to eliminate the error caused by manually selection. In machinery fault diagnosis area, the obtained data samples under faulty conditions are usually far less than those under normal condition, resulting in unbalanced dataset issue. The commonly used machine learning techniques including Neural Network, Support Vector Machine, and Fuzzy C-Means, etc. are subject to high misclassification with unbalanced datasets. On the other hand, Support Vector Data Description is suitable for unbalanced datasets, but it is limited for only two class classification. To address the aforementioned issues, Support Vector Data Description based machine learning model is formulated with Binary Tree for multi-classification problems (e.g. multi fault classification or fault severity recognition, etc.) in machinery fault diagnosis. The binary tree structure of multiple clusters is firstly drawn based on the order of cluster-to-cluster distances calculated by Mahalanobis distance. Support Vector Data Description model is then applied to Binary Tree structure from top to bottom for classification. The parameters of Support Vector Data Description are optimized by Particle Swarm Optimization algorithm taking the recognition accuracy as objective function. The effectiveness of presented method is validated in the rotor unbalance severity classification, and the presented method yields higher classification accuracy comparing with conventional models. <s> BIB009 </s> Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> IV. SUMMARY AND FUTURE DIRECTIONS <s> Data in vision domain often exhibit highly-skewed class distribution, i.e., most data belong to a few majority classes, while the minority classes only contain a scarce amount of instances. To mitigate this issue, contemporary classification methods based on deep convolutional neural network (CNN) typically follow classic strategies such as class re-sampling or cost-sensitive training. In this paper, we conduct extensive and systematic experiments to validate the effectiveness of these classic schemes for representation learning on class-imbalanced data. We further demonstrate that more discriminative deep representation can be learned by enforcing a deep network to maintain both intercluster and inter-class margins. This tighter constraint effectively reduces the class imbalance inherent in the local data neighborhood. We show that the margins can be easily deployed in standard deep learning framework through quintuplet instance sampling and the associated triple-header hinge loss. The representation learned by our approach, when combined with a simple k-nearest neighbor (kNN) algorithm, shows significant improvements over existing methods on both high-and low-level vision classification tasks that exhibit imbalanced class distribution. <s> BIB010 </s> Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> IV. SUMMARY AND FUTURE DIRECTIONS <s> In recent years, machine learning techniques have been widely used to solve many problems for fault diagnosis. However, in many real-world fault diagnosis applications, the distribution of the source domain data (on which the model is trained) is different from the distribution of the target domain data (where the learned model is actually deployed), which leads to performance degradation. In this paper, we introduce domain adaptation, which can find the solution to this problem by adapting the classifier or the regression model trained in a source domain for use in a different but related target domain. In particular, we proposed a novel deep neural network model with domain adaptation for fault diagnosis. Two main contributions are concluded by comparing to the previous works: first, the proposed model can utilize domain adaptation meanwhile strengthening the representative information of the original data, so that a high classification accuracy in the target domain can be achieved, and second, we proposed several strategies to explore the optimal hyperparameters of the model. Experimental results, on several real-world datasets, demonstrate the effectiveness and the reliability of both the proposed model and the exploring strategies for the parameters. <s> BIB011 </s> Deep Learning and Its Applications to Machine Health Monitoring: A Survey <s> IV. SUMMARY AND FUTURE DIRECTIONS <s> Abstract Diagnosis of bearings generally plays an important role in fault diagnosis of mechanical system, and machine learning has been a promising tool in this field. In many real applications of bearings fault diagnosis, the data tend to be online imbalanced, which means, the number of fault data is much less than the normal data while they are all collected in online sequential way. Suffering from this problem, many traditional diagnosis methods will get low accuracy of fault data which acts as the minority class in the collected bearing data. To address this problem, an online sequential prediction method for imbalanced fault diagnosis problem is proposed based on extreme learning machine. This method introduces the principal curve and granulation division to simulate the flow distribution and overall distribution characteristics of fault data, respectively. Then a confident over-sampling and under-sampling process is proposed to establish the initial offline diagnosis model. In online stage, the obtained granules and principal curves are rebuilt on the bearing data which are arrived in sequence, and after the over-sampling and under-sampling process, the balanced sample set is formed to update the diagnosis model dynamically. A theoretical analysis is provided and proves that, even existing information loss, the proposed method has lower bound of the model reliability. Simulation experiments are conducted on IMS bearing data and CWRU bearing data. The comparative results demonstrate that the proposed method can improve the fault diagnosis accuracy with better effectiveness and robustness than other algorithms. <s> BIB012
In this paper, we have provided a systematic overview of the state-of-the-art DL-based MHMS. Deep learning, as a subfield of machine learning, is serving as a bridge between big machinery data and data-driven MHMS. Therefore, within the past four years, they have been applied in various machine health monitoring tasks. These proposed DL-based MHMS are summarized according to four categories of DL architecture as: Auto-encoder models, Restricted Boltzmann Machines models, Convolutional Neural Networks and Recurrent Neural Networks. Since the momentum of the research of DL-based MHMS is growing fast, we hope the messages about the capabilities of these DL techniques, especially representation learning for complex machinery data and target prediction for various machine health monitoring tasks, can be conveyed to readers. Through these previous works, it can be found that DL-based MHMS do not require extensive human labor and expert knowledge, i.e., the end-to-end structure is able to map raw machinery data to targets. Therefore, the application of deep learning models are not restricted to specific kinds of machines, which can be a general solution to address the machine health monitoring problems. Besides, some research trends and potential future research directions are given as follows: * Open-source Large Dataset: Due to the huge model complexity behind DL methods, the performance of DL-based MHMS heavily depends on the scale and quality of datasets. On other hand, the depth of DL model is limited by the scale of datasets. As a result, the benchmark CNN model for image recognition has 152 layers, which can be supported by the large dataset ImageNet containing over ten million annotated images BIB006 , BIB001 . In contrast, the proposed DL models for MHMS may stack up to 5 hidden layers. And the model trained in such kind of large datasets can be the model initialization for the following specific task/dataset. Therefore, it is meaningful to design and publish large-scale machinery datasets. * Utilization of Domain Knowledge: deep learning is not a skeleton key to all machine health monitoring problems. Domain knowledge can contribute to the success of applying DL models on machine health monitoring. For example, extracting discriminative features can reduce the size of the followed DL models and appropriate task-specific regularization term can boost the final performance BIB007 . * Model and Data Visualization: deep learning techniques, especially deep neural networks, have been regarded as black boxes models, i.e., their inner computation mechanisms are unexplainable. Visualization of the learned representation and the applied model can offer some insights into these DL models, and then these insights achieved by this kind of interaction can facilitate the building and configuration of DL models for complex machine health monitoring problems. Some visualization techniques have been proposed including t-SNE model for high dimensional data visualization  and visualization of the activations produced by each layer and features at each layer of a DNN via regularized optimization BIB003 . * Transferred Deep Learning: Transfer learning tries to apply knowledge learned in one domain to a different but related domain BIB002 . This research direction is meaningful in machine health monitoring, since some machine health monitoring problems have sufficient training data while other areas lack training data. The machine learning models including DL models trained in one domain can be transferred to the other domain. Some previous works focusing on transferred feature extraction/dimensionality reduction have been done BIB004 , BIB008 . In BIB011 , a Maximum Mean Discrepancy (MMD) measure evaluating the discrepancy between source and target domains was added into the target function of deep neural networks. * Imbalanced Class: The class distribution of machinery data in real life normally follows a highly-skewed one, in which most data samples belong to few categories. For example, the number of fault data is much less than the one of health data in fault diagnosis. Some enhanced machine learning models including SVM and ELM have been proposed to address this imbalanced issue in machine health monitoring BIB012 , BIB009 . Recently, some interesting methods investigating the application of deep learning in imbalanced class problems have been developed, including CNN models with class resampling or cost-sensitive training BIB010 and the integration of boot strapping methods and CNN model BIB005 . It is believed that deep learning will have a more and more prospective future impacting machine health monitoring, especially in the age of big machinery data.