Deep Learning on Image Denoising: An overview <s> CNN/NN and common feature extraction methods for AWNI denoising <s> A sparse representation-based classifier (SRC) is developed and shows great potential for real-world face recognition. This paper presents a dimensionality reduction method that fits SRC well. SRC adopts a class reconstruction residual-based decision rule, we use it as a criterion to steer the design of a feature extraction method. The method is thus called the SRC steered discriminative projection (SRC-DP). SRC-DP maximizes the ratio of between-class reconstruction residual to within-class reconstruction residual in the projected space and thus enables SRC to achieve better performance. SRC-DP provides low-dimensional representation of human faces to make the SRC-based face recognition system more efficient. Experiments are done on the AR, the extended Yale B, and PIE face image databases, and results demonstrate the proposed method is more effective than other feature extraction methods based on the SRC. <s> BIB001 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN and common feature extraction methods for AWNI denoising <s> Despite their great success, there is still no comprehensive theoretical understanding of learning with Deep Neural Networks (DNNs) or their inner organization. Previous work proposed to analyze DNNs in the \textit{Information Plane}; i.e., the plane of the Mutual Information values that each layer preserves on the input and output variables. They suggested that the goal of the network is to optimize the Information Bottleneck (IB) tradeoff between compression and prediction, successively, for each layer. ::: In this work we follow up on this idea and demonstrate the effectiveness of the Information-Plane visualization of DNNs. Our main results are: (i) most of the training epochs in standard DL are spent on {\emph compression} of the input to efficient representation and not on fitting the training labels. (ii) The representation compression phase begins when the training errors becomes small and the Stochastic Gradient Decent (SGD) epochs change from a fast drift to smaller training error into a stochastic relaxation, or random diffusion, constrained by the training error value. (iii) The converged layers lie on or very close to the Information Bottleneck (IB) theoretical bound, and the maps from the input to any hidden layer and from this hidden layer to the output satisfy the IB self-consistent equations. This generalization through noise mechanism is unique to Deep Neural Networks and absent in one layer networks. (iv) The training time is dramatically reduced when adding more hidden layers. Thus the main advantage of the hidden layers is computational. This can be explained by the reduced relaxation time, as this it scales super-linearly (exponentially for simple diffusion) with the information compression from the previous layer. <s> BIB002 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN and common feature extraction methods for AWNI denoising <s> Regression-based algorithms have shown to be good at denoising Monte Carlo (MC) renderings by leveraging its inexpensive by-products (e.g., feature buffers). However, when using higher-order models to handle complex cases, these techniques often overfit to noise in the input. For this reason, supervised learning methods have been proposed that train on a large collection of reference examples, but they use explicit filters that limit their denoising ability. To address these problems, we propose a novel, supervised learning approach that allows the filtering kernel to be more complex and general by leveraging a deep convolutional neural network (CNN) architecture. In one embodiment of our framework, the CNN directly predicts the final denoised pixel value as a highly non-linear combination of the input features. In a second approach, we introduce a novel, kernel-prediction network which uses the CNN to estimate the local weighting kernels used to compute each denoised pixel from its neighbors. We train and evaluate our networks on production data and observe improvements over state-of-the-art MC denoisers, showing that our methods generalize well to a variety of scenes. We conclude by analyzing various components of our architecture and identify areas of further research in deep learning for MC denoising. <s> BIB003 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN and common feature extraction methods for AWNI denoising <s> This study presents a model to effectively recognise image noise of different types and levels: impulse, Gaussian, Speckle and Poisson noise, and a mixture of multiple types of the noise. To classify image noise type, the convolutional neural network (CNN) method with backpropagation algorithm and stochastic gradient descent optimisation techniques are implemented. In order to reduce the training time and computational cost of the algorithm, the principal components analysis (PCA) filters generating strategy is deployed to obtain data adaptive filter banks. The authors validated their designed CNN with PCA for noise types recognition model with degraded images containing noise of single and combination of multiple types, with a total of 11,000 and 1650 datasets for training and testing purposes, respectively. The variety and complexity of data have never been addressed before in any other research work. The capability of their intelligent system in handling images degraded under this complicated environment has surpassed human-eye performance in noise types recognition. The authors’ experiments have proven the reliability of the proposed noise types recognition model by having achieved an overall average accuracy of 99.3% while recognising eight classes of noise. <s> BIB004 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN and common feature extraction methods for AWNI denoising <s> There are two main streams in up-to-date image denoising algorithms: non-local self similarity (NSS) prior based methods and convolutional neural network (CNN) based methods. The NSS based methods are favorable on images with regular and repetitive patterns while the CNN based methods perform better on irregular structures. In this paper, we propose a block-matching convolutional neural network (BMCNN) method that combines NSS prior and CNN. Initially, similar local patches in the input image are integrated into a 3D block. In order to prevent the noise from messing up the block matching, we first apply an existing denoising algorithm on the noisy image. The denoised image is employed as a pilot signal for the block matching, and then denoising function for the block is learned by a CNN structure. Experimental results show that the proposed BMCNN algorithm achieves state-of-the-art performance. In detail, BMCNN can restore both repetitive and irregular structures. <s> BIB005 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN and common feature extraction methods for AWNI denoising <s> Image denoising is one of the most important directions in image processing. Medical images are often affected by noise and interference from the environment and equipment during acquisition, conversion, and transmission, resulting in degradation. This paper mainly introduces a new convolutional neural network structure for medical image denoising - deep neural network based on wavelet domain（deep wavelet denoising net ，DWDN ）. Our DWDN model exhibits high effectiveness in general medical image denoising tasks and is more excellent in the details of image <s> BIB006 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN and common feature extraction methods for AWNI denoising <s> The tradeoff between receptive field size and efficiency is a crucial issue in low level vision. Plain convolutional networks (CNNs) generally enlarge the receptive field at the expense of computational cost. Recently, dilated filtering has been adopted to address this issue. But it suffers from gridding effect, and the resulting receptive field is only a sparse sampling of input image with checkerboard patterns. In this paper, we present a novel multi-level wavelet CNN (MWCNN) model for better tradeoff between receptive field size and computational efficiency. With the modified U-Net architecture, wavelet transform is introduced to reduce the size of feature maps in the contracting subnetwork. Furthermore, another convolutional layer is further used to decrease the channels of feature maps. In the expanding subnetwork, inverse wavelet transform is then deployed to reconstruct the high resolution feature maps. Our MWCNN can also be explained as the generalization of dilated filtering and subsampling, and can be applied to many image restoration tasks. The experimental results clearly show the effectiveness of MWCNN for image denoising, single image super-resolution, and JPEG image artifacts removal. <s> BIB007 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN and common feature extraction methods for AWNI denoising <s> Magnetic Resonance Imaging (MRI) is considered one of the most effective imaging techniques used in the medical field for both clinical investigation and diagnosis. This is due to the fact that MRI provides many critical features of the tissue including both physiological and chemical information. Rician noise affects MR images during acquisition thereby reducing the quality of the image and complicating the accurate diagnosis. In this paper, we propose a novel technique for MR image denoising using Deep Convolutional Neural Network (Deep CNN) and anisotropic diffusion (AD) which we will refer to as Deep CNN-AD. Watershed transform is then used to segment the tumorous portion of the denoised image. The proposed method is tested on the BraTS MRI datasets. The proposed denoising method produced better results compared to previous methods. As denoising process affect the segmentation process therefore better denoised images by proposed technique produced more accurate segmentation with an average Specificity of 99.85% and dice coefficient of 90.46% thus indicating better performance of proposed technique. © 2018 Authors. <s> BIB008 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN and common feature extraction methods for AWNI denoising <s> Denoising is a fundamental task in image processing with wide applications for enhancing image qualities. BM3D is considered as an effective baseline for image denoising. Although learning-based methods have been dominant in this area recently, the traditional methods are still valuable to inspire new ideas by combining with learning-based approaches. In this letter, we propose a new convolutional neural network inspired by the classical BM3D algorithm, dubbed as BM3D-Net. We unroll the computational pipeline of BM3D algorithm into a convolutional neural network structure, with “extraction” and “aggregation” layers to model block matching stage in BM3D. We apply our network to three denoising tasks: gray-scale image denoising, color image denoising, and depth map denoising. Experiments show that BM3D-Net significantly outperforms the basic BM3D method, and achieves competitive results compared with state of the art on these tasks. <s> BIB009 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN and common feature extraction methods for AWNI denoising <s> We present a technique for jointly denoising bursts of images taken from a handheld camera. In particular, we propose a convolutional neural network architecture for predicting spatially varying kernels that can both align and denoise frames, a synthetic data generation approach based on a realistic noise formation model, and an optimization guided by an annealed loss function to avoid undesirable local minima. Our model matches or outperforms the state-of-the-art across a wide range of noise levels on both real and synthetic data. <s> BIB010 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN and common feature extraction methods for AWNI denoising <s> Abstract Although hyperspectral image (HSI) denoising has been studied for decades, preserving spectral data efficiently remains an open problem. In this paper, we present a powerful and trainable spectral difference mapping method based on convolutional networks with residual learning in an end-to-end fashion for preserving spectral profile while removing noise in HSIs. Our approach, called SSDRN (spectral-spatial denoising by residual network), blends a spectral difference mapping strategy with a denoised key band for an efficient complete set of HSI denoising. The key band is selected based on a principal component transformation matrix. Experiments have been conducted on both ground based HSIs and airborne data. Comparative analyses validate that the proposed method presents superior denoising performance as it preserves spectral information better, and requires less computational time. <s> BIB011 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN and common feature extraction methods for AWNI denoising <s> Abstract. Image denoising based on a convolution neural network (CNN) can be described as the problem of learning a mapping function from a noisy image to a clean image through an end-to-end training. We propose a multiscale parallel feature extraction module (MPFE) for CNN denoising, which integrates residual learning and dense connection. The MPFE uses convolution kernels of different sizes to adaptively extract multiple features in different scales from the input image. We introduce dense connection to connect each MPFE, which can make different features interact with each other and concatenate together, so as to fully exploit the image features. The dense connection can pass the features that carry many image details, which help reduce image distortion. Meanwhile, it can also reduce gradient disappearance and improve convergence speed. The MPFE uses residual learning to resolve the gradient loss caused by high network depth while still ensuring that the network learns the details of the noisy image. Simulation experiments show that our denoising method has the ability of suppressing Gaussian noises with different noise levels, it performs superior performance over the state-of-the-art denoising methods. <s> BIB012 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN and common feature extraction methods for AWNI denoising <s> There are two main streams in up-to-date image denoising algorithms: non-local self similarity (NSS) prior based methods and convolutional neural network (CNN) based methods. The NSS based methods are favorable on images with regular and repetitive patterns while the CNN based methods perform better on irregular structures. In this paper, we propose a block-matching convolutional neural network (BMCNN) method that combines NSS prior and CNN. Initially, similar local patches in the input image are integrated into a 3D block. In order to prevent the noise from messing up the block matching, we first apply an existing denoising algorithm on the noisy image. The denoised image is employed as a pilot signal for the block matching, and then denoising function for the block is learned by a CNN structure. Experimental results show that the proposed BMCNN algorithm achieves state-of-the-art performance. In detail, BMCNN can restore both repetitive and irregular structures. <s> BIB013 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN and common feature extraction methods for AWNI denoising <s> Neighborhood preserving embedding (NPE) has been proposed to encode overall geometry manifold embedding information. However, the class-special structure of the data is destroyed by noise or outliers existing in the data. To address this problem, in this article, we propose a novel embedding approach called robust flexible preserving embedding (RFPE). First, RFPE recovers the noisy data by low-rank learning and obtains clean data. Then, the clean data are used to learn the projection matrix. In this way, the projective learning is totally unaffected by noise or outliers. By encoding a flexible regularization term, RFPE can keep the property of the data points with a nonlinear manifold and be more flexible. RFPE searches the optimal projective subspace for feature extraction. In addition, we also extend the proposed RFPE to a kernel case and propose kernel RFPE (KRFPE). Extensive experiments on six public image databases show the superiority of the proposed methods over other state-of-the-art methods. <s> BIB014 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN and common feature extraction methods for AWNI denoising <s> Abstract Segmentation of the brain into gray matter, white matter, and cerebrospinal fluid (CSF) using magnetic resonance (MR) imaging plays a fundamental role in neuroimaging research and clinical settings. Due to the complexity of brain anatomy, low image quality, and insufficient training data, both traditional and deep learning segmentation methods have a limited performance. In this paper, we propose a multi-model, multi-size and multi-view deep neural network (M3Net) for brain MR image segmentation, which uses three identical modules to segment transaxial, coronal, and sagittal MR slices, respectively. Each module consists of multi-size U-Nets and multi-size back propagation neural networks. It also uses a probabilistic atlas to explore brain anatomy and a convolutional auto-encoder (CAE) to restore MR images. The proposed M3Net model was evaluated against widely used segmentation methods on both synthetic and clinical studies. Our results suggest that the proposed model is able to segment Brain MR Images more accurately. <s> BIB015 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN and common feature extraction methods for AWNI denoising <s> The stripe noise effects severely degrade the image quality in infrared imaging systems. The existing destriping algorithms still struggle to balance noise suppression, detail preservation, and real-time performance, which retards their application in spectral imaging and signal processing field. To solve this problem, an innovative wavelet deep neural network from the perspective of transform domain is presented in this paper, which takes the intrinsic characteristics of stripe noise and complementary information between the coefficients of different wavelet sub-bands into full consideration to accurately estimate the noise with the lower computational load. In addition, a special directional regularizer is further defined to separate the scene details from stripe noise more thoroughly and recover the details more accurately. The extensive experiments on simulated and real data demonstrate that our proposed method outperforms several classical destriping methods on both quantitative and qualitative assessments. <s> BIB016 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN and common feature extraction methods for AWNI denoising <s> Most of the classical denoising methods restore clear results by selecting and averaging pixels in the noisy input. Instead of relying on hand-crafted selecting and averaging strategies, we propose to explicitly learn this process with deep neural networks. Specifically, we propose deformable 2D kernels for image denoising where the sampling locations and kernel weights are both learned. The proposed kernel naturally adapts to image structures and could effectively reduce the oversmoothing artifacts. Furthermore, we develop 3D deformable kernels for video denoising to more efficiently sample pixels across the spatial-temporal space. Our method is able to solve the misalignment issues of large motion from dynamic scenes. For better training our video denoising model, we introduce the trilinear sampler and a new regularization term. We demonstrate that the proposed method performs favorably against the state-of-the-art image and video denoising approaches on both synthetic and real-world data. <s> BIB017 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN and common feature extraction methods for AWNI denoising <s> The low-dose computed tomography (CT) perfusion data has low signal-to-noise ratio resulting in derived perfusion maps being noisy. These low-quality maps typically requires a denoising step to improve their utility in real-time. The existing methods, including state-of-the-art online sparse perfusion deconvolution (SPD), largely relies on the convolutional model that may not be applicable in all cases of brain perfusion. In this paper, a denoising convolutional neural network (DCNN) was proposed that relies only on computed perfusion maps for performing the denoising step. The network was trained with a large number of low-dose digital brain phantom perfusion maps to provide an approximation to the corresponding high-dose perfusion maps. The batch normalization coupled with residual learning makes the trained model invariant to the dynamic range of the input low-dose perfusion maps. The denoising of the raw-data using the convolutional neural network was also attempted here and shown to have limited applicability in the low-dose CT perfusion cases. The digital perfusion phantom as well as in-vivo results indicate that the proposed DCNN applied in the derived map domain provides superior improvement compared to the online SPD with an added advantage of being computationally efficient. <s> BIB018 </s> Deep Learning on Image Denoising: An overview <s> CNN/NN and common feature extraction methods for AWNI denoising <s> Abstract Structure-preserved denoising of 3D magnetic resonance imaging (MRI) images is a critical step in medical image analysis. Over the past few years, many algorithms with impressive performances have been proposed. In this paper, inspired by the idea of deep learning, we introduce an MRI denoising method based on the residual encoder–decoder Wasserstein generative adversarial network (RED-WGAN). Specifically, to explore the structure similarity between neighboring slices, a 3D configuration is utilized as the basic processing unit. Residual autoencoders combined with deconvolution operations are introduced into the generator network. Furthermore, to alleviate the oversmoothing shortcoming of the traditional mean squared error (MSE) loss function, the perceptual similarity, which is implemented by calculating the distances in the feature space extracted by a pretrained VGG-19 network, is incorporated with the MSE and adversarial losses to form the new loss function. Extensive experiments are implemented to assess the performance of the proposed method. The experimental results show that the proposed RED-WGAN achieves performance superior to several state-of-the-art methods in both simulated and real clinical data. In particular, our method demonstrates powerful abilities in both noise suppression and structure preservation. <s> BIB019
Feature is used to represent the whole image in image processing, and it is important for machine learning  BIB014 BIB001 . However, deep learning technique is black box, and cannot choose features, which cannot guarantee obtained features are the most robust BIB002 BIB015 . Motivated by this reason, common feature extraction method embedded into CNN was conducted in image denoising. That can be divided into five categories: weak edge-information, non-linear, high dimensional and non-salient noisy images, and high computational costs. For weak edge-information noisy images, CNN with transformation domain method including Guan et al. BIB016 , Li et al. BIB006 , Liu et al. BIB007 , Latif et al. BIB008 and Yang et al. BIB009 was very popular to remove the noise. Specifically, in BIB007 , it used wavelet method and U-net to eliminate the gridding effect of dilated convolutions on enlarging receptive field for image restoration. For non-linear noisy images, CNN with kernel method was useful BIB003 BIB017 . These methods had three steps in general BIB010 . First step used CNN to extract features. Second step utilized kernel method to convert obtained non-linear features into linearity. Final step exploited the RL to construct the latent clean image. For high dimensional noisy images, the combination of CNN and dimensional reduction method were proposed BIB011  . For example, Khaw et al. BIB004 used CNN with principal component analysis (PCA) for image denoising. This had three phases. First phase used convolution operations to extract features. Second phase utilized the PCA to reduce the dimension of obtained features. Final phase employed convolutions to deal with obtained features from the PCA and reconstruct a clean image. For non-salient noisy images, signal processing idea can guide CNN to extract salient features BIB012 BIB018 BIB019  . Specifically, skip connection operation was a typically operation of signal processing BIB018 . For high computational cost tasks, CNN with nature of image was very effective to decrease complex  BIB013 BIB005 . For example, Ahn et al. BIB005 used CNN with NSS to filter the noise, where similar characteristics of the given noisy image can accelerate speed of extraction feature and reduce computational cost. More detailed information of these methods mentioned are summarized in Table 2 .