Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> The newly-emerging sparse representation-based classifier (SRC) shows great potential for pattern classification but lacks theoretical justification. This paper gives an insight into SRC and seeks reasonable supports for its effectiveness. SRC uses L"1-optimizer instead of L"0-optimizer on account of computational convenience and efficiency. We re-examine the role of L"1-optimizer and find that for pattern recognition tasks, L"1-optimizer provides more classification meaningful information than L"0-optimizer does. L"0-optimizer can achieve sparsity only, whereas L"1-optimizer can achieve closeness as well as sparsity. Sparsity determines a small number of nonzero representation coefficients, while closeness makes the nonzero representation coefficients concentrate on the training samples with the same class label as the given test sample. Thus, it is closeness that guarantees the effectiveness of the L"1-optimizer based SRC. Based on the closeness prior, we further propose two kinds of class L"1-optimizer classifiers (CL"1C), the closeness rule based CL"1C (C-CL"1C) and its improved version: the Lasso rule based CL"1C (L-CL"1C). The proposed classifiers are evaluated on five databases and the experimental results demonstrate advantages of the proposed classifiers over SRC in classification performance and computational efficiency for large sample size problems. <s> BIB001 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> Training Deep Neural Networks is complicated by the fact that the distribution of each layer's inputs changes during training, as the parameters of the previous layers change. This slows down the training by requiring lower learning rates and careful parameter initialization, and makes it notoriously hard to train models with saturating nonlinearities. We refer to this phenomenon as internal covariate shift, and address the problem by normalizing layer inputs. Our method draws its strength from making normalization a part of the model architecture and performing the normalization for each training mini-batch. Batch Normalization allows us to use much higher learning rates and be less careful about initialization, and in some cases eliminates the need for Dropout. Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin. Using an ensemble of batch-normalized networks, we improve upon the best published result on ImageNet classification: reaching 4.82% top-5 test error, exceeding the accuracy of human raters. <s> BIB002 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> The discriminative model learning for image denoising has been recently attracting considerable attentions due to its favorable denoising performance. In this paper, we take one step forward by investigating the construction of feed-forward denoising convolutional neural networks (DnCNNs) to embrace the progress in very deep architecture, learning algorithm, and regularization method into image denoising. Specifically, residual learning and batch normalization are utilized to speed up the training process as well as boost the denoising performance. Different from the existing discriminative denoising models which usually train a specific model for additive white Gaussian noise at a certain noise level, our DnCNN model is able to handle Gaussian denoising with unknown noise level (i.e., blind Gaussian denoising). With the residual learning strategy, DnCNN implicitly removes the latent clean image in the hidden layers. This property motivates us to train a single DnCNN model to tackle with several general image denoising tasks, such as Gaussian denoising, single image super-resolution, and JPEG image deblocking. Our extensive experiments demonstrate that our DnCNN model can not only exhibit high effectiveness in several general image denoising tasks, but also be efficiently implemented by benefiting from GPU computing. <s> BIB003 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers—8× deeper than VGG nets [40] but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions1, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation. <s> BIB004 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> Conventionally, image denoising and high-level vision tasks are handled separately in computer vision. In this paper, we cope with the two jointly and explore the mutual influence between them. First we propose a convolutional neural network for image denoising which achieves the state-of-the-art performance. Second we propose a deep neural network solution that cascades two modules for image denoising and various high-level tasks, respectively, and use the joint loss for updating only the denoising network via back-propagation. We demonstrate that on one hand, the proposed denoiser has the generality to overcome the performance degradation of different high-level vision tasks. On the other hand, with the guidance of high-level vision information, the denoising network can generate more visually appealing results. To the best of our knowledge, this is the first work investigating the benefit of exploiting image semantics simultaneously for image denoising and high-level vision tasks via deep learning. The code is available online. <s> BIB005 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> Variations of deep neural networks such as convolutional neural network (CNN) have been successfully applied to image denoising. The goal is to automatically learn a mapping from a noisy image to a clean image given training data consisting of pairs of noisy and clean images. Most existing CNN models for image denoising have many layers. In such cases, the models involve a large amount of parameters and are computationally expensive to train. In this paper, we develop a dilated residual CNN for Gaussian image denoising. Compared with the recently proposed residual denoiser, our method can achieve comparable performance with less computational cost. Specifically, we enlarge receptive field by adopting dilated convolution in residual network, and the dilation factor is set to a certain value. We utilize appropriate zero padding to make the dimension of the output the same as the input. It has been proven that the expansion of receptive field can boost the CNN performance in image classification, and we further demonstrate that it can also lead to competitive performance for denoising problem. Moreover, we present a formula to calculate receptive field size when dilated convolution is incorporated. Thus, the change of receptive field can be interpreted mathematically. To validate the efficacy of our approach, we conduct extensive experiments for both gray and color image denoising with specific or randomized noise levels. Both of the quantitative measurements and the visual results of denoising are promising comparing with state-of-the-art baselines. <s> BIB006 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> The latest deep learning approaches perform better than the state-of-the-art signal processing approaches in various image restoration tasks. However, if an image contains many patterns and structures, the performance of these CNNs is still inferior. To address this issue, here we propose a novel feature space deep residual learning algorithm that outperforms the existing residual learning. The main idea is originated from the observation that the performance of a learning algorithm can be improved if the input and/or label manifolds can be made topologically simpler by an analytic mapping to a feature space. Our extensive numerical studies using denoising experiments and NTIRE single-image super-resolution (SISR) competition demonstrate that the proposed feature space residual learning outperforms the existing state-of-the-art approaches. Moreover, our algorithm was ranked high in the NTIRE competition with 5-10 times faster computational time compared to the top ranked teams. <s> BIB007 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> In medical imaging, denoising is very important for analysis of images, diagnosis and treatment of diseases. Currently, image denoising methods based on deep learning are effective, where the methods are however limited for the requirement of training sample size (i.e., not successful enough for small data size). Using small sample size, we design deep feed forward denoising convolutional neural networks by studying the model in deep framework, learning approach and regularization approach for medical image denoising. More specifically, we use residual learning as a learning approach and batch normalization as regularization in the deep model. Unlike most of the other image denoising approaches which directly learn the latent clean images, the residual learning approach learns the noise from the noisy images instead of the latent clean images where the denoised images are obtained by subtracting the learned residual from the noisy image. Moreover, batch normalization is integrated with residual learning to improve model learning accuracy and training time. We compute the quality of the reconstructed or denoised image in standard image quality metrics, peak signal to noise ratio and structural similarity and compare our model performance with some medical image denoising techniques. Experimental results reveal that our approach has better performance than some other methods. <s> BIB008 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> Recently, very deep convolutional neural networks (CNNs) have been attracting considerable attention in image restoration. However, as the depth grows, the long-term dependency problem is rarely realized for these very deep models, which results in the prior states/layers having little influence on the subsequent ones. Motivated by the fact that human thoughts have persistency, we propose a very deep persistent memory network (MemNet) that introduces a memory block, consisting of a recursive unit and a gate unit, to explicitly mine persistent memory through an adaptive learning process. The recursive unit learns multi-level representations of the current state under different receptive fields. The representations and the outputs from the previous memory blocks are concatenated and sent to the gate unit, which adaptively controls how much of the previous states should be reserved, and decides how much of the current state should be stored. We apply MemNet to three image restoration tasks, i.e., image denosing, super-resolution and JPEG deblocking. Comprehensive experiments demonstrate the necessity of the MemNet and its unanimous superiority on all three tasks over the state of the arts. Code is available at this https URL. <s> BIB009 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> We propose to learn a fully-convolutional network model that consists of a Chain of Identity Mapping Modules (CIMM) for image denoising. The CIMM structure possesses two distinctive features that are important for the noise removal task. Firstly, each residual unit employs identity mappings as the skip connections and receives pre-activated input in order to preserve the gradient magnitude propagated in both the forward and backward directions. Secondly, by utilizing dilated kernels for the convolution layers in the residual branch, in other words within an identity mapping module, each neuron in the last convolution layer can observe the full receptive field of the first layer. After being trained on the BSD400 dataset, the proposed network produces remarkably higher numerical accuracy and better visual image quality than the state-of-the-art when being evaluated on conventional benchmark images and the BSD68 dataset. <s> BIB010 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> Recently, deep learning approaches with various network architectures have achieved significant performance improvement over existing iterative reconstruction methods in various imaging problems. However, it is still unclear why these deep learning architectures work for specific inverse problems. To address these issues, here we show that the long-searched-for missing link is the convolution framelets for representing a signal by convolving local and non-local bases. The convolution framelets was originally developed to generalize the theory of low-rank Hankel matrix approaches for inverse problems, and this paper further extends the idea so that we can obtain a deep neural network using multilayer convolution framelets with perfect reconstruction (PR) under rectilinear linear unit nonlinearity (ReLU). Our analysis also shows that the popular deep network components such as residual block, redundant filter channels, and concatenated ReLU (CReLU) do indeed help to achieve the PR, while the pooling and unpooling layers should be augmented with high-pass branches to meet the PR condition. Moreover, by changing the number of filter channels and bias, we can control the shrinkage behaviors of the neural network. This discovery leads us to propose a novel theory for deep convolutional framelets neural network. Using numerical experiments with various inverse problems, we demonstrated that our deep convolution framelets network shows consistent improvement over existing deep architectures.This discovery suggests that the success of deep learning is not from a magical power of a black-box, but rather comes from the power of a novel signal representation using non-local basis combined with data-driven local basis, which is indeed a natural extension of classical signal processing theory. <s> BIB011 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> Abstract Low-dose CT has received increasing attention in the recent years and is considered a promising method to reduce the risk of cancer in patients. However, the reduction of the dosage leads to quantum noise in the raw data, which is carried on in the reconstructed images. Two different multilayer convolutional neural network (CNN) architectures for the denoising of CT images are investigated. ResFCN is based on a fully-convolutional network that consists of three blocks of 5×5 convolutions filters and a ResUNet that is trained with 10 convolutional blocks that are arranged in a multi-scale fashion. Both architectures feature a residual connection of the input image to ease learning. Training images are based on realistic simulations by using the XCAT phantom. The ResUNet approach shows the most promising results with a peak signal to noise ratio of 44.00 compared to ResFCN with 41.79. <s> BIB012 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> Optical coherence tomography (OCT) is widely used in the diagnosis of ophthalmic diseases. However, OCT is affected by ubiquitous speckle noise which make it difficult to analysis the retinal structures. To efficiently remove the noise as well as preserve clinical detail information contained in the images, we suggest to train a denoise generative adversarial network (DNGAN) jointly with a densely connected convolutional network to estimate clean OCT images from noisy OCT images. A generator convolutional neural network (CNN) with several dense connections, is trained to transform noisy OCT image into clean OCT image. At the same time, an adversarial CNN is trained to improve the denoising performance of the generator. The experimental results demonstrate the superior performance of our network on OCT images. <s> BIB013 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> Hyperspectral image (HSI) denoising is a crucial preprocessing procedure to improve the performance of the subsequent HSI interpretation and applications. In this paper, a novel deep learning-based method for this task is proposed, by learning a nonlinear end-to-end mapping between the noisy and clean HSIs with a combined spatial–spectral deep convolutional neural network (HSID-CNN). Both the spatial and spectral information are simultaneously assigned to the proposed network. In addition, multiscale feature extraction and multilevel feature representation are, respectively, employed to capture both the multiscale spatial–spectral feature and fuse different feature representations for the final restoration. The simulated and real-data experiments demonstrate that the proposed HSID-CNN outperforms many of the mainstream methods in both the quantitative evaluation indexes, visual effects, and HSI classification accuracy. <s> BIB014 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> Low-dose Computed Tomography (CT) is considered a solution for reducing the risk of X-ray radiation; however, lowering the X-ray current results in a degraded reconstructed image. To improve the quality of the image, different noise removal techniques have been proposed. Con- volutional neural networks also have shown promising results in denoising the low-dose CT images. In this paper, a deep residual network with dilated convolution is proposed. The identity mappings pass the signal to the higher layers and improve the performance of the network and its training time. Moreover, employing dilated convolution helps to increase the receptive field faster. Dilated convolution makes it possible to achieve good results with fewer layers and less computational costs. The proposed network learns end to end mapping from low-dose to normal-dose CT images. <s> BIB015 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> In this Letter, we propose a fast speckle noise reduction method with only a single reconstructed image based on convolutional neural networks. The proposed network has multi-sized kernels that can capture the speckle noise component effectively from digital holographic images. For robust noise reduction performance, the network is trained with a large noisy image dataset that has object-dependent noise and a wide range of noise levels. The experimental results show the fast, robust, and outstanding speckle noise reduction performance of the proposed approach. <s> BIB016 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> Aerial images are subject to various types of noise, which restricts the recognition and analysis of images, target monitoring, and search services. At present, deep learning is successful in image recognition. However, traditional convolutional neural networks (CNNs) extract the main features of an image to predict directly and are limited by the requirements of the training sample size (i.e., a small data size is not successful enough). In this paper, using a small sample size, we propose an aerial-image denoising recognition model based on CNNs with a multi-scale residual learning approach. The proposed model has the following three advantages: (1) Instead of directly learning latent clean images, the proposed model learns the noise from noisy images and then subtracts the learned residual from the noisy images to obtain reconstructed (denoised) images; (2) The developed image denoising recognition model is beneficial to small training datasets; (3) We use multi-scale residual learning as a learning approach, and dropout is introduced into the model architecture to force the network to learn to generalize well enough. Our experimental results on aerial-image denoising recognition reveal that the proposed approach is highly superior to the other state-of-the-art methods. <s> BIB017 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> The denoising of magnetic resonance (MR) images is a task of great importance for improving the acquired image quality. Many methods have been proposed in the literature to retrieve noise free images with good performances. Howerever, the state-of-the-art denoising methods, all needs a time-consuming optimization processes and their performance strongly depend on the estimated noise level parameter. Within this manuscript we propose the idea of denoising MRI Rician noise using a convolutional neural network. The advantage of the proposed methodology is that the learning based model can be directly used in the denosing process without optimization and even without the noise level parameter. Specifically, a ten convolutional layers neural network combined with residual learning and multi-channel strategy was proposed. Two training ways: training on a specific noise level and training on a general level were conducted to demonstrate the capability of our methods. Experimental results over synthetic and real 3D MR data demonstrate our proposed network can achieve superior performance compared with other methods in term of both of the peak signal to noise ratio and the global of structure similarity index. Without noise level parameter, our general noise-applicable model is also better than the other compared methods in two datasets. Furthermore, our training model shows good general applicability. <s> BIB018 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> Existing methods for image denoising mainly focused on noise and visual artifacts too much but rarely mentioned the loss of edge information. In this paper, we propose a deep denoising network based on the residual learning and perceptual loss to generate high-quality denoised results. Inspired by the deep residual network, two new strategies are used to modify the original structure, which can improve the learning process by compressing the mapping range. At first, the high-frequency layer of noisy image is used as the input by removing background information. The secondly, a residual mapping is trained to predict the difference between clean and noisy images as output instead of the final denoised image. Furtherly improve the denoised result, a joint loss function is defined as the weighted sum of pixel-to-pixel Euclidean loss and perceptual loss. A well-trained convolutional neural network is connected to learn the semantic information we would like to measure in our perceptual loss. It encourages the train process to learn similar feature representations rather than match each low-level pixel, which can guide front denoising network to reconstruct more edges and details. As opposed to the standard denoising models that concentrate on one specific noise level, our single model can deal with the noise of unknown levels (i.e., blind denoising). The experiments show that our proposed network achieves superior performances and recovers majority of missing details from low-quality observations. <s> BIB019 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> Blind image restoration processors based on convolutional neural network (CNN) are intensively researched because of their high performance. However, they are too sensitive to the perturbation of the degradation model. They easily fail to restore the image whose degradation model is slightly different from the trained degradation model. In this paper, we propose a non-blind CNN-based image restoration processor, aiming to be robust against a perturbation of the degradation model compared to the blind restoration processor. Experimental comparisons demonstrate that the proposed non-blind CNN-based image restoration processor can robustly restore images compared to existing blind CNN-based image restoration processors. <s> BIB020 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> Deep learning has recently proven extremely successful in many low-level image-processing tasks including natural image denoising. However, with regards to designing deep models for practical image processing, there are numerous essential viewpoints that one ought to consider. In this work, we first aimed to reply to probably the most critical design questions through theoretical analysis and extensive experiments: How deep and wide a deep denoiser should and can be? Does denoising performance get improved by using residual learning? Can and should we switch from the region-based to image-based models? And second, based on our analysis, we designed a deep neural network for natural image denoising which was hundred-layer deep, exploited both internal and external residual learning, and was trained in an image-based fashion. Our deep denoiser achieved the state-of-the-art results quantitatively and qualitatively on multiple datasets including one with more than 10,000 images. <s> BIB021 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> Image denoising is an urgent task that arises in image processing and transmission. Images are an integral part of content transmitted in infocommunication systems. Traditional statistical image filtering algorithms are not always effective for the random nature of the noise spectrum. Image denoising by convolutional neural networks is a modern and effective approach. The article is devoted to demonstrate the possibilities of using denoising convolutional neural networks to solve one of the most difficult tasks that developers face when performing the transfer of graphical information in infocommunication systems – denoising. Unlike traditional algorithms, denoising convolutional neural networks have architecture features that allow them to perform effectively image filtering with unknown noise level. It is suggested to use denoising convolutional neural networks to generate a correction signal in the infocommunication system, which transmits a noisy image. The article proposes to use denoising convolutional neural networks to generate a correction signal in the infocommunication system, which transmits a noisy image. Such pre-trained correction elements on a large and diverse image dataset can be easily fine-tuned for special filtering tasks. <s> BIB022 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> Abstract. Over the past couple of years, deep learning in convolutional neural networks (CNNs) has proven to be extremely powerful in image processing tasks including image denoising. We deal with digital image denoising based on deep learning. In this regard, dilated CNNs have been successful in achieving impressive denoising quality by obtaining a considerably effective trade-off between receptive field size and network depth, and thus reducing the computational intensity of the CNN architecture. Such networks in general use rectifier linear units as the activation function for introducing nonlinearity into the network. We propose a dilated CNN with exponential linear units for image denoising. Our aim is to enhance the performance of dilated networks in image denoising and develop an effective denoising CNN architecture by optimizing computational intensity. We experiment with a (conventional) 10-layer network, as well as a 5-layer version of the proposed model. It is observed that the proposed 5-layer network succeeds to achieve state-of-the-art denoised image quality as well as high computational efficiency. Our experimental results prove the efficiency of the proposed models in image denoising as well as complexity reduction. <s> BIB023 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> Image super-resolution and denoising are two important tasks in image processing that can lead to improvement in image quality. Image super-resolution is the task of mapping a low resolution image to a high resolution image whereas denoising is the task of learning a clean image from a noisy input. We propose and train a single deep learning network that we term as SuRDCNN (super-resolution and denoising convolutional neural network), to perform these two tasks simultaneously . Our model nearly replicates the architecture of existing state-of-the-art deep learning models for super-resolution and denoising. We use the proven strategy of residual learning, as supported by state-of-the-art networks in this domain. Our trained SuRDCNN is capable of super-resolving image in the presence of Gaussian noise, Poisson noise or any random combination of both of these noises. <s> BIB024 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> We propose a new deep network architecture for removing a stripe noise from a single meteorological satellite infrared cloud image. In the proposed framework, a residual learning is utilized to directly reduce the mapping range from input to output, which speeds up the training process as well as boosts the destriping performance. Inspired by the wide inference networks, we use wider CNNs with more convolutions in the first part of the proposed network, which is helpful for learning the similar pixel-distribution features from noisy images. To further improve the performance, we propose a local-global combination structure model, which combines the representations of different layers for recovering the rich details of infrared cloud images. Moreover, we extend our method to remove rain streaks from single images, which provides a new idea for rain-removal task. In addition, we provide a new meteorological satellite infrared cloud image dataset for training and validating the proposed network. Final extensive experiments demonstrate that the proposed method can achieve both comparable restoration quality and computational efficiency with several state-of-the-art approaches. <s> BIB025 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> Boosting is a classic algorithm which has been successfully applied to diverse computer vision tasks. In the scenario of image denoising, however, the existing boosting algorithms are surpassed by the emerging learning-based models. In this paper, we propose a novel deep boosting framework (DBF) for denoising, which integrates several convolutional networks in a feed-forward fashion. Along with the integrated networks, however, the depth of the boosting framework is substantially increased, which brings difficulty to training. To solve this problem, we introduce the concept of dense connection that overcomes the vanishing of gradients during training. Furthermore, we propose a path-widening fusion scheme cooperated with the dilated convolution to derive a lightweight yet efficient convolutional network as the boosting unit, named Dilated Dense Fusion Network (DDFN). Comprehensive experiments demonstrate that our DBF outperforms existing methods on widely used benchmarks, in terms of different denoising tasks. <s> BIB026 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> In many areas images can be corrupted by various types of noise and therefore image denoising is a prerequisite. For example, medical images like the 4D-CT or ultrasound ones, are prone to noise and artifacts that can affect diagnostic confidence. Remote sensing is another field for which image preprocessing is mandatory to improve the quality of source images. Synthetic Aperture Radar (SAR) images are typically corrupted by multiplicative speckle noise. In this paper, a deep neural network able to deal with both additive white Gaussian and multiplicative speckle noises is developed, showing also some blind denoising capacity. The experiments on noisy images show that the proposal, which consists in a encoder-decoder, is efficient and competitive in comparison with state-of-the-art methods. <s> BIB027 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> Camera sensors often fail to capture clear images or videos in a poorly lit environment. In this paper, we propose a trainable hybrid network to enhance the visibility of such degraded images. The proposed network consists of two distinct streams to simultaneously learn the global content and the salient structures of the clear image in a unified network. More specifically, the content stream estimates the global content of the low-light input through an encoder–decoder network. However, the encoder in the content stream tends to lose some structure details. To remedy this, we propose a novel spatially variant recurrent neural network (RNN) as an edge stream to model edge details, with the guidance of another auto-encoder. The experimental results show that the proposed network favorably performs against the state-of-the-art low-light image enhancement algorithms. <s> BIB028 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> 2-D neighborhood preserving projection (2DNPP) uses 2-D images as feature input instead of 1-D vectors used by neighborhood preserving projection (NPP). 2DNPP requires less computation time than NPP. However, both NPP and 2DNPP use the ${L}_{ {{2}}}$ norm as a metric, which is sensitive to noise in data. In this paper, we proposed a novel NPP method called low-rank 2DNPP (LR-2DNPP). This method divided the input data into a component part that encoded low-rank features, and an error part that ensured the noise was sparse. Then, a nearest neighbor graph was learned from the clean data using the same procedure as 2DNPP. To ensure that the features learned by LR-2DNPP were optimal for classification, we combined the structurally incoherent learning and low-rank learning with NPP to form a unified model called discriminative LR-2DNPP (DLR-2DNPP). By encoding the structural incoherence of the learned clean data, DLR-2DNPP could enhance the discriminative ability for feature extraction. Theoretical analyses on the convergence and computational complexity of LR-2DNPP and DLR-2DNPP were presented in details. We used seven public image databases to verify the performance of the proposed methods. The experimental results showed the effectiveness of our methods for robust image representation. <s> BIB029 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> The spectral and the spatial information in hyperspectral images (HSIs) are the two sides of the same coin. How to jointly model them is the key issue for HSIs’ noise removal, including random noise, structural stripe noise, and dead pixels/lines. In this paper, we introduce the deep convolutional neural network (CNN) to achieve this goal. The learned filters can well extract the spatial information within their local receptive filed. Meanwhile, the spectral correlation can be depicted by the multiple channels of the learned 2-D filters, namely, the number of filters in each layer. The consequent advantages of our CNN-based HSI denoising method (HSI-DeNet) over previous methods are threefold. First, the proposed HSI-DeNet can be regarded as a tensor-based method by directly learning the filters in each layer without damaging the spectral-spatial structures. Second, the HSI-DeNet can simultaneously accommodate various kinds of noise in HSIs. Moreover, our method is flexible for both single image and multiple images by slightly modifying the channels of the filters in the first and last layers. Last but not least, our method is extremely fast in the testing phase, which makes it more practical for real application. The proposed HSI-DeNet is extensively evaluated on several HSIs, and outperforms the state-of-the-art HSI-DeNets in terms of both speed and performance. <s> BIB030 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> Deep learning, especially a discriminative model for image denoising, has shown great potential in removing complex spectral–spatial noise in hyperspectral images (HSI). For HSI denoising, it is crucial to extract more context information around each pixel and to predict each pixel according to the surrounding context. Therefore, the effective receptive field plays an important role when performing denoising task. Generally, an HSI denoising model can achieve better performance by reserving the correlation of adjacent spectral bands and extracting more pixel features in the spatial domain. In this paper, 3-D atrous denoising convolution neural network (3DADCNN) is proposed for HSI. The model extracts feature maps along both spatial and spectral dimensions and enlarges the receptive field without significantly increasing the number of network parameters. Simultaneously, the multibranch and multiscale structure is utilized to reduce training difficulty, lessen overfitting risk, and preserve details in texture. The proposed model can be applied to the corrupted image with a mixed type of photon and thermal noise. Experimental results of the quantitative and qualitative evaluation show that 3DADCNN outperforms state-of-the-art HSI denoising methods. <s> BIB031 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> The signal degradation due to the Poisson noise is a common problem in the low-light imaging field. Recently, deep learning employing the convolution neural network for image denoising has drawn considerable attention owing to its favourable denoising performance. On the basis of the fact that the reconstruction of corrupted pixels can be facilitated by the context information in image denoising, the authors propose a deep multi-scale cross-path concatenation residual network (MC ::: 2 ::: RNet) which incorporates cross-path concatenation modules for Poisson denoising. Multiple paths are achieved by the cross-path concatenation operation and the skip connection. As a consequence, multi-scale context representations of images under different receptive fields can be learnt by MC ::: 2 ::: RNet. With the residual learning strategy, MC ::: 2 ::: RNet learns the residual between the noisy image and the latent clean image rather than the direct mapping to facilitate model training. Specially, unlike existing discriminative Poisson denoising algorithms that train a model only for the specific noise level, they aim to train a single model for handling Poisson noise with different levels, i.e. blind Poisson denoising. Quantitative experiments demonstrate that the proposed model is superior over the state-of-the-art Poisson denoising approaches in terms of peak signal-to-noise ratio and visual effect. <s> BIB032 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> Abstract This review article provides a comprehensive survey on state-of-the-art impulse and Gaussian denoising filters applied to images and summarizes the progress that has been made over the years in all applications involving image processing. The random noise model in this survey is assumed to be comprised of impulse (salt and pepper) and Gaussian noise. Different noise models are addressed, and different types of denoising filters are studied in terms of their performance on digital images and in their various practical implications and domains of application. A comprehensive comparison is performed to cover all the denoising methods in details and the results they yield. With this extensive review, researchers in image processing will be able to ascertain which of these denoising methods will be best applicable to their research needs and the application domain where such methods are contemplated for implementation. <s> BIB033 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> Image restoration problems are typically ill-posed requiring the design of suitable priors. These priors are typically hand-designed and are fully instantiated throughout the process. In this paper, we introduce a novel framework for handling inverse problems related to image restoration based on elements from the half quadratic splitting method and proximal operators. Modeling the proximal operator as a convolutional network, we defined an implicit prior on the image space as a function class during training. This is in contrast to the common practice in literature of having the prior to be fixed and fully instantiated even during training stages. Further, we allow this proximal operator to be tuned differently for each iteration which greatly increases modeling capacity and allows us to reduce the number of iterations by an order of magnitude as compared to other approaches. Our final network is an end-to-end one whose run time matches the previous fastest algorithms while outperforming them in recovery fidelity on two image restoration tasks. Indeed, we find our approach achieves state-of-the-art results on benchmarks in image denoising and image super resolution while recovering more complex and finer details. <s> BIB034 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN for AWNI denoising <s> Abstract A single-exposure image may lose details because of the imaging dynamic range limitations of single camera sensor. Multi-image fusion techniques are often used to improve the image quality, but if there are moving objects in the scene, the fused images may result in ghost artifacts. In order to avoid this problem and enhance single-exposure images, this paper proposes a dual network cascade model for single image enhancement, including exposure prediction network and exposure fusion network. First, the exposure prediction network generates two under-/over-exposure images that differ from the input normal-exposure image so as to recover the lost details of the under-exposed/over-exposed regions. Then, the exposure fusion network fuses the input image and the generated under-/over-exposure images to generate the final enhanced image. The loss function constructed by a structural dissimilarity index is used to alleviate chessboard artifacts in the generated image. Further, through three-phase training, the model robustly generates enhanced images without any post-processing. The experimental results demonstrate that the proposed method can effectively improve the image contrast and reconstruct details of under-exposed/over-exposed regions in the original image. <s> BIB035
Automatic feature extraction methods are important to reduce computational costs for image applications BIB001 BIB028 BIB029 . CNNs based on this reason are developed for image denoising  BIB005 . Zhang et al. BIB003 proposed a model as well as DnCNN to deal with multiple low-level BIB002 and residual learning techniques BIB004 . Wang et al. BIB006 , Bae et al. BIB007 and Jifara et al. BIB008 also presented a residual learning and deeper CNN for image denoising, respectively. However, deeper CNN technique depended on deeper layer rather than shallow layer, which resulted in long-term dependency problem. To tackle this problem, a lot of signal-base methods were proposed. Tai et al. BIB009 exploited recursive and gate units to adaptively mine more accurate features and recover clean images. Inspired by low-rank Hankel matrix in low-level vision, Ye et al. BIB011 provided convolution frames to explain the connection between signal processing and deep learning by convolving local and nonlocal bases. For solving insufficient noisy images (i.e. hyperspectral and medical images), a lot of recent works try to extract more useful information by improved CNN BIB030 BIB012 BIB013 BIB031 . For example, Yuan et al. BIB014 combined deep CNN, residual learning and multiscale knowledge to remove the noise from hyperspectral-noisy images. However, using these operations may increase computational costs and memory consumption, which was not optimistic for real applications. For addressing the phenomenon, Gholizadeh et al. BIB015 utilized dilated convolutions  to enlarge receptive field and reduce depth of network at no bring extra cost for CT image denoising. Lian et al. BIB032 proposed a residual network via multi-scale cross-path concatenation to suppress the noise. It is known from most of methods above relied on improved CNNs to deal with the noise. Thus, designing network architectures is important for image denoising   . Changing network architectures has the following ways in general  BIB033 : (1) fusing features from multiple inputs of a CNN. (2) Changing the loss function. (3) Increasing depth or width of the CNN. (4) Adding some auxiliary plug-ins into CNNs. 5) Using skip connections or cascade operations into CNNs. Specifically, the first way included three types: 1) different parts of one sample as multiple inputs from different networks  . 2) Different perspectives for the one sample as input, such as multiple scales BIB016 BIB017 . 3) Different channels of a CNN as input BIB018 . The second way mainly designed different loss function according to characteristic of nature images to extract more robust features BIB034 . For example, Chen et al. BIB019 jointed Euclidean and perceptual loss functions to mine more edge information for image denoising. The third way enlarged receptive field size to improve denoising performance via increasing the depth or width of network BIB020 BIB021 BIB022 . The fourth way applied plug-ins, i.e. activation function, dilated convolution, fully connected layer and pooling operations to enhance the expressive ability of the CNN BIB023  BIB024 . The final way utilized skip connections BIB025 BIB026 BIB027 BIB010 or cascade operations BIB032 BIB035 to provide complementary information for deep layer in CNN. Table 1 provides an overview of CNNs for AWNI denoising.