A Survey on the New Generation of Deep Learning in Image Processing <s> H. OTHER IMAGE PROCESSING TASKS <s> Learning effective feature representations and similarity measures are crucial to the retrieval performance of a content-based image retrieval (CBIR) system. Despite extensive research efforts for decades, it remains one of the most challenging open problems that considerably hinders the successes of real-world CBIR systems. The key challenge has been attributed to the well-known ``semantic gap'' issue that exists between low-level image pixels captured by machines and high-level semantic concepts perceived by human. Among various techniques, machine learning has been actively investigated as a possible direction to bridge the semantic gap in the long term. Inspired by recent successes of deep learning techniques for computer vision and other applications, in this paper, we attempt to address an open problem: if deep learning is a hope for bridging the semantic gap in CBIR and how much improvements in CBIR tasks can be achieved by exploring the state-of-the-art deep learning techniques for learning feature representations and similarity measures. Specifically, we investigate a framework of deep learning with application to CBIR tasks with an extensive set of empirical studies by examining a state-of-the-art deep learning method (Convolutional Neural Networks) for CBIR tasks under varied settings. From our empirical studies, we find some encouraging results and summarize some important insights for future research. <s> BIB001 </s> A Survey on the New Generation of Deep Learning in Image Processing <s> H. OTHER IMAGE PROCESSING TASKS <s> In this paper, we propose deformable deep convolutional neural networks for generic object detection. This new deep learning object detection framework has innovations in multiple aspects. In the proposed new deep architecture, a new deformation constrained pooling (def-pooling) layer models the deformation of object parts with geometric constraint and penalty. A new pre-training strategy is proposed to learn feature representations more suitable for the object detection task and with good generalization capability. By changing the net structures, training strategies, adding and removing some key components in the detection pipeline, a set of models with large diversity are obtained, which significantly improves the effectiveness of model averaging. The proposed approach improves the mean averaged precision obtained by RCNN [14], which was the state-of-the-art, from 31% to 50.3% on the ILSVRC2014 detection test set. It also outperforms the winner of ILSVRC2014, GoogLeNet, by 6.1%. Detailed component-wise analysis is also provided through extensive experimental evaluation, which provide a global view for people to understand the deep learning object detection pipeline. <s> BIB002 </s> A Survey on the New Generation of Deep Learning in Image Processing <s> H. OTHER IMAGE PROCESSING TASKS <s> Despite significant recent advances in the field of face recognition [10, 14, 15, 17], implementing face verification and recognition efficiently at scale presents serious challenges to current approaches. In this paper we present a system, called FaceNet, that directly learns a mapping from face images to a compact Euclidean space where distances directly correspond to a measure of face similarity. Once this space has been produced, tasks such as face recognition, verification and clustering can be easily implemented using standard techniques with FaceNet embeddings as feature vectors. <s> BIB003
In addition to the above image processing tasks, we will briefly introduce some of the successful applications of deep learning technology. First, let us start with the image retrieval task. CBIR is one of the fundamental research challenges extensively studied in the multimedia community for decades. However, the semantic gap issues that exist between low-level image pixels captured by machines and high-level semantic information perceived by a human is still one of the most challenging problems in current CBIR research. Among various techniques, deep CNN has been actively investigated as a promising direction to bridge the semantic gap in the long term  . On the one hand, deep CNN can obtain useful high-level feature representations of images using large-scale network architecture. On the other hand, a deep CNN model on classification or similarity loss function can be retrained easily since the features extracted by the pre-trained deep CNN model may not be better than the traditional handcrafted features BIB001 . Second, for the change detection, it means two images of the same area can be captured at two different time instances, and then these two images are processed to identify the changes part, the output is a binary change map, which indicates the location of the changes. Generally, the goal of change detection is to detect significant changes while rejecting unimportant ones by preprocessing operations (i.e., geometric and radiometric adjustments). In recently, Deep learning has a powerful ability to learn abstract features. In particular, as the unsupervised deep learning methods successively proposed, such as GAN, VAE, ML-CSC, therefore researching the application of deep learning techniques to change detection has become a promising research direction  ,  . Third, for face recognition, that is a relatively mature application direction for deep learning. In particular, CNN has achieved promising results in face recognition recently. From deep hidden identity feature (deepID BIB002 ) to faceNet BIB003 , CNN plays a very important role in feature extraction. On the widely used Labelled Faces in the Wild dataset, faceNet can even achieve a new record recognition accuracy of 99.63%.