A Survey on the New Generation of Deep Learning in Image Processing <s> 3) PRIOR KNOWLEDGE FOR DEEP LEARNING <s> For the task of RGB-D object recognition, it is important to identify suitable representations of images, which can boost the performance of object recognition. In this work, we propose a novel representation learning method for RGB-D images by jointly incorporating the underlying data structure and the prior knowledge of the data. Specifically, the convolutional neural networks (CNN) are employed to learn image representation by exploiting the underlying data structure. To handle the problem of the limited RGB and depth images for object recognition, the multi-level hierarchies of features trained on ImageNet from the CNN are transferred to learn rich generic feature representation for RGB and depth images while the labeled images are leveraged. On the other hand, we propose a novel deep auto-encoders (DAE) to exploit the prior knowledge, which can overcome the expensive computational cost of optimization in feature encoding. The expected representations of images are obtained by integrating the two types of image representations. To verify the effectiveness of the proposed method, we thoroughly conduct extensive experiments on two publicly available RGB-D datasets. The encouraging experimental results compared with the state-of-the-art approaches demonstrate the advantages of the proposed method. <s> BIB001 </s> A Survey on the New Generation of Deep Learning in Image Processing <s> 3) PRIOR KNOWLEDGE FOR DEEP LEARNING <s> Wind speed prediction is a basic requirement of wind energy generation with large generation capacity for large-scale wind power penetration. The intermittency and stochastic quality of wind speed leads to a big challenge for high penetration of wind power in electricity systems due to error-prone wind speed prediction methods. There are many artificial neural network (ANN) approaches proposed in the recent literature in order to tackle this problem. However, hand engineering features and applying shallow architectures can lead to poor prediction performance in these methodologies. Deep neural networks are ANNs with great generalization capability that can automatically extract meaningful features from the data with as less prior knowledge as possible. In this paper we propose a stacked auto-encoder (SAE) neural network for ultra-short-term and short-term wind speed prediction. To the best of our knowledge this is the first paper that applies deep learning on wind speed prediction of a wind site. Moreover, a rough regression layer is applied at the top of this model in order to deal with uncertainty factors existing in the wind speed data. Experimental results show significant improvement compared to other ANN methods that applied shallow architectures in the literature and the traditional SAE. <s> BIB002 </s> A Survey on the New Generation of Deep Learning in Image Processing <s> 3) PRIOR KNOWLEDGE FOR DEEP LEARNING <s> As an interesting and emerging topic, cosaliency detection aims at simultaneously extracting common salient objects in multiple related images. It differs from the conventional saliency detection paradigm in which saliency detection for each image is determined one by one independently without taking advantage of the homogeneity in the data pool of multiple related images. In this paper, we propose a novel cosaliency detection approach using deep learning models. Two new concepts, called intrasaliency prior transfer and deep intersaliency mining, are introduced and explored in the proposed work. For the intrasaliency prior transfer, we build a stacked denoising autoencoder (SDAE) to learn the saliency prior knowledge from auxiliary annotated data sets and then transfer the learned knowledge to estimate the intrasaliency for each image in cosaliency data sets. For the deep intersaliency mining, we formulate it by using the deep reconstruction residual obtained in the highest hidden layer of a self-trained SDAE. The obtained deep intersaliency can extract more intrinsic and general hidden patterns to discover the homogeneity of cosalient objects in terms of some higher level concepts. Finally, the cosaliency maps are generated by weighted integration of the proposed intrasaliency prior, deep intersaliency, and traditional shallow intersaliency. Comprehensive experiments over diverse publicly available benchmark data sets demonstrate consistent performance gains of the proposed method over the state-of-the-art cosaliency detection methods. <s> BIB003 </s> A Survey on the New Generation of Deep Learning in Image Processing <s> 3) PRIOR KNOWLEDGE FOR DEEP LEARNING <s> Deep learning allows to develop feature representations and train classification models in a fully integrated way. However, learning deep networks is quite hard and it improves over shallow architectures only if a large number of training data is available. Injecting prior knowledge into the learner is a principled way to reduce the amount of required training data, as the learner does not need to induce the knowledge from the data itself. In this paper we propose a general and principled way to integrate prior knowledge when training deep networks. Semantic Based Regularization (SBR) is used as underlying framework to represent the prior knowledge, expressed as a collection of first-order logic clauses (FOL), and where each task to be learned corresponds to a predicate in the knowledge base. The knowledge base correlates the tasks to be learned and it is translated into a set of constraints which are integrated into the learning process via backpropagation. The experimental results show how the integration of the prior knowledge boosts the accuracy of a state-of-the-art deep network on an image classification task. <s> BIB004 </s> A Survey on the New Generation of Deep Learning in Image Processing <s> 3) PRIOR KNOWLEDGE FOR DEEP LEARNING <s> Developments in deep learning have seen the use of layerwise unsupervised learning combined with supervised learning for fine-tuning. With this layerwise approach, a deep network can be seen as a more modular system that lends itself well to learning representations. In this paper, we investigate whether such modularity can be useful to the insertion of background knowledge into deep networks, whether it can improve learning performance when it is available, and to the extraction of knowledge from trained deep networks, and whether it can offer a better understanding of the representations learned by such networks. To this end, we use a simple symbolic language—a set of logical rules that we call confidence rules —and show that it is suitable for the representation of quantitative reasoning in deep networks. We show by knowledge extraction that confidence rules can offer a low-cost representation for layerwise networks (or restricted Boltzmann machines). We also show that layerwise extraction can produce an improvement in the accuracy of deep belief networks. Furthermore, the proposed symbolic characterization of deep networks provides a novel method for the insertion of prior knowledge and training of deep networks. With the use of this method, a deep neural–symbolic system is proposed and evaluated, with the experimental results indicating that modularity through the use of confidence rules and knowledge insertion can be beneficial to network performance. <s> BIB005
With the disappearance of data dividends, deep learning has increasingly demonstrated its limitations, particularly in relying on large-scale image datasets with labels and the inability to use prior knowledge effectively, et., these limitations hinder the further development of deep learning BIB004 . How to effectively apply a large amount of prior knowledge/rules? How to make the result of the deep learning model consistent with prior knowledge BIB001 ? Based on the prior knowledge or rules is what bridges the gap between large neural networks and relatively small datasets, these problems have gradually become one of the mainstream research directions. Actually, there are many ways to incorporate prior knowledge into deep neural networks BIB005 , BIB003 . The simplest type of prior knowledge often used is weight decay, which assumes the weights come from a normal distribution with zero mean and some fixed variance. Then this type of prior knowledge is added as an additional term to the loss function. Especially, weight decay can view as the same as the Bayesian approach to modeling prior knowledge. Similar to weight decay, it is possible to construct other loss terms that penalize mappings contradicting our domain knowledge. However, integrating prior knowledge into deep learning is not always easy, the main reason is that the main description of knowledge representation is not an abstract quantitative feature, but a relationship between features BIB002 . Also, deep learning places too much emphasis on the independence of the system and exclude general prior knowledge. Therefore, deep learning combined with prior knowledge/rules of the image may become a breakthrough in solving small sample problems, but this promising direction still needs further research.