Deep Learning on Image Denoising: An overview <s> Deep learning techniques for real noisy image denoising <s> In this paper, we propose a CNN based method to perform low-light image enhancement. We design a special module to utilize multiscale feature maps, which can avoid gradient vanishing problem as well. In order to preserve image textures as much as possible, we use SSIM loss to train our model. The contrast of low-light images can be adaptively enhanced using our method. Results demonstrate that our CNN based method outperforms other contrast enhancement methods. <s> BIB001 </s> Deep Learning on Image Denoising: An overview <s> Deep learning techniques for real noisy image denoising <s> Batch Normalization is quite effective at accelerating and improving the training of deep models. However, its effectiveness diminishes when the training minibatches are small, or do not consist of independent samples. We hypothesize that this is due to the dependence of model layer inputs on all the examples in the minibatch, and different activations being produced between training and inference. We propose Batch Renormalization, a simple and effective extension to ensure that the training and inference models generate the same outputs that depend on individual examples rather than the entire minibatch. Models trained with Batch Renormalization perform substantially better than batchnorm when training with small or non-i.i.d. minibatches. At the same time, Batch Renormalization retains the benefits of batchnorm such as insensitivity to initialization and training efficiency. <s> BIB002 </s> Deep Learning on Image Denoising: An overview <s> Deep learning techniques for real noisy image denoising <s> Noise is an inherent issue of low-light image capture, one which is exacerbated on mobile devices due to their narrow apertures and small sensors. One strategy for mitigating noise in a low-light situation is to increase the shutter time of the camera, thus allowing each photosite to integrate more light and decrease noise variance. However, there are two downsides of long exposures: (a) bright regions can exceed the sensor range, and (b) camera and scene motion will result in blurred images. Another way of gathering more light is to capture multiple short (thus noisy) frames in a "burst" and intelligently integrate the content, thus avoiding the above downsides. In this paper, we use the burst-capture strategy and implement the intelligent integration via a recurrent fully convolutional deep neural net (CNN). We build our novel, multiframe architecture to be a simple addition to any single frame denoising model, and design to handle an arbitrary number of noisy input frames. We show that it achieves state of the art denoising results on our burst dataset, improving on the best published multi-frame techniques, such as VBM4D and FlexISP. Finally, we explore other applications of image enhancement by integrating content from multiple frames and demonstrate that our DNN architecture generalizes well to image super-resolution. <s> BIB003 </s> Deep Learning on Image Denoising: An overview <s> Deep learning techniques for real noisy image denoising <s> Model-based optimization methods and discriminative learning methods have been the two dominant strategies for solving various inverse problems in low-level vision. Typically, those two kinds of methods have their respective merits and drawbacks, e.g., model-based optimization methods are flexible for handling different inverse problems but are usually time-consuming with sophisticated priors for the purpose of good performance, in the meanwhile, discriminative learning methods have fast testing speed but their application range is greatly restricted by the specialized task. Recent works have revealed that, with the aid of variable splitting techniques, denoiser prior can be plugged in as a modular part of model-based optimization methods to solve other inverse problems (e.g., deblurring). Such an integration induces considerable advantage when the denoiser is obtained via discriminative learning. However, the study of integration with fast discriminative denoiser prior is still lacking. To this end, this paper aims to train a set of fast and effective CNN (convolutional neural network) denoisers and integrate them into model-based optimization method to solve other inverse problems. Experimental results demonstrate that the learned set of denoisers can not only achieve promising Gaussian denoising results but also can be used as prior to deliver good performance for various low-level vision applications. <s> BIB004 </s> Deep Learning on Image Denoising: An overview <s> Deep learning techniques for real noisy image denoising <s> In this paper, we propose a joint framework to enhance images under low-light conditions. First, a convolutional neural network (CNN) based architecture is proposed to denoise low-light images. Then, based on atmosphere scattering model, we introduce a low-light model to enhance image contrast. In our low-light model, we propose a simple but effective image prior, bright channel prior, to estimate the transmission parameter; besides, an effective filter is designed to adaptively estimate environment light in different image areas. Experimental results demonstrate that our method achieves superior performance over other methods. <s> BIB005 </s> Deep Learning on Image Denoising: An overview <s> Deep learning techniques for real noisy image denoising <s> X-ray computed tomography (CT) using sparse projection views is often used to reduce the radiation dose. However, due to the insufficient projection views, a reconstruction approach using the filtered back projection (FBP) produces severe streaking artifacts. Recently, deep learning approaches using large receptive field neural networks such as U-net have demonstrated impressive performance for sparse view CT reconstruction. However, theoretical justification is still lacking. The main goal of this paper is, therefore, to develop a mathematical theory and to discuss how to improve these algorithms. In particular, inspired by the recent theory of deep convolutional framelets, we show that the U-net relies on a sub-optimal non-local bases that overly emphasizes low frequency components. The discovery leads to a dual frame and a tight frame U-net architectures for effective recovery of directional image components. <s> BIB006 </s> Deep Learning on Image Denoising: An overview <s> Deep learning techniques for real noisy image denoising <s> Light field (LF) cameras provide perspective information of scenes by taking directional measurements of the focusing light rays. The raw outputs are usually dark with additive camera noise, which impedes subsequent processing and applications. We propose a novel LF denoising framework based on anisotropic parallax analysis (APA). Two convolutional neural networks are jointly designed for the task: first, the structural parallax synthesis network predicts the parallax details for the entire LF based on a set of anisotropic parallax features. These novel features can efficiently capture the high frequency perspective components of a LF from noisy observations. Second, the view-dependent detail compensation network restores non-Lambertian variation to each LF view by involving view-specific spatial energies. Extensive experiments show that the proposed APA LF denoiser provides a much better denoising performance than state-of-the-art methods in terms of visual quality and in preservation of parallax details. <s> BIB007 </s> Deep Learning on Image Denoising: An overview <s> Deep learning techniques for real noisy image denoising <s> Abstract Deep convolutional neural networks (CNNs) have attracted great attention in the field of image denoising. However, there are two drawbacks: (1) it is very difficult to train a deeper CNN for denoising tasks, and (2) most of deeper CNNs suffer from performance saturation. In this paper, we report the design of a novel network called a batch-renormalization denoising network (BRDNet). Specifically, we combine two networks to increase the width of the network, and thus obtain more features. Because batch renormalization is fused into BRDNet, we can address the internal covariate shift and small mini-batch problems. Residual learning is also adopted in a holistic way to facilitate the network training. Dilated convolutions are exploited to extract more information for denoising tasks. Extensive experimental results show that BRDNet outperforms state-of-the-art image-denoising methods. The code of BRDNet is accessible at http://www.yongxu.org/lunwen.html . <s> BIB008 </s> Deep Learning on Image Denoising: An overview <s> Deep learning techniques for real noisy image denoising <s> Low-light remote sensing is a powerful complement to daytime optical remote sensing and can greatly expand the time domain of high-resolution earth observations, and make day and night imaging possible. However, when a low-light sensor is used in the morning dusk and dawn, the captured images have characteristics of low contrast, low brightness, and low signal-to-noise ratio, which severely restrict the identification and interpretation of ground objects. Traditional low-light image enhancement algorithms such as histogram equalization, gamma conversion, and contrast-limited adaptive histogram equalization algorithm, and so on can enhance the low-light remote sensing image and solve the problem of contrast enhancement, but the noise amplification effect brought by the enhancement will degrade the signal-to-noise ratio of the enhanced image. Therefore, in this paper, a data-driven low-light remote sensing image enhancement algorithm is studied. First of all, lots of low-light raw image data pairs corresponding to very low illumination are captured. Then, these raw image data are used to train a deep fully convolutional neural network composed of an encoder–decoder structure. After that, the low-light remote sensing images could be enhanced by the pretrained net structure. The numerical results demonstrate that the fully convolutional neural network based on enhancement algorithm greatly improves the brightness and the contrast of low-light images compared with the traditional enhancement algorithms while a high enough signal-to-noise ratio could be preserved, which will make interpretation and identification much easier. <s> BIB009 </s> Deep Learning on Image Denoising: An overview <s> Deep learning techniques for real noisy image denoising <s> When taking photos in dim-light environments, due to the small amount of light entering, the shot images are usually extremely dark, with a great deal of noise, and the color cannot reflect real-world color. Under this condition, the traditional methods used for single image denoising have always failed to be effective. One common idea is to take multiple frames of the same scene to enhance the signal-to-noise ratio. This paper proposes a recurrent fully convolutional network (RFCN) to process burst photos taken under extremely low-light conditions, and to obtain denoised images with improved brightness. Our model maps raw burst images directly to sRGB outputs, either to produce a best image or to generate a multi-frame denoised image sequence. This process has proven to be capable of accomplishing the low-level task of denoising, as well as the high-level task of color correction and enhancement, all of which is end-to-end processing through our network. Our method has achieved better results than state-of-the-art methods. In addition, we have applied the model trained by one type of camera without fine-tuning on photos captured by different cameras and have obtained similar end-to-end enhancements. <s> BIB010 </s> Deep Learning on Image Denoising: An overview <s> Deep learning techniques for real noisy image denoising <s> Deep convolutional neural networks perform better on images containing spatially invariant noise (synthetic noise); however, their performance is limited on real-noisy photographs and requires multiple stage network modeling. To advance the practicability of denoising algorithms, this paper proposes a novel single-stage blind real image denoising network (RIDNet) by employing a modular architecture. We use a residual on the residual structure to ease the flow of low-frequency information and apply feature attention to exploit the channel dependencies. Furthermore, the evaluation in terms of quantitative metrics and visual quality on three synthetic and four real noisy datasets against 19 state-of-the-art algorithms demonstrate the superiority of our RIDNet. <s> BIB011 </s> Deep Learning on Image Denoising: An overview <s> Deep learning techniques for real noisy image denoising <s> Image reconstruction techniques such as denoising often need to be applied to the RGB output of cameras and cellphones. Unfortunately, the commonly used additive white noise (AWGN) models do not accurately reproduce the noise and the degradation encountered on these inputs. This is particularly important for learning-based techniques, because the mismatch between training and real world data will hurt their generalization. This paper aims to accurately simulate the degradation and noise transformation performed by camera pipelines. This allows us to generate realistic degradation in RGB images that can be used to train machine learning models. We use our simulation to study the importance of noise modeling for learning-based denoising. Our study shows that a realistic noise model is required for learning to denoise real JPEG images. A neural network trained on realistic noise outperforms the one trained with AWGN by 3 dB. An ablation study of our pipeline shows that simulating denoising and demosaicking is important to this improvement and that realistic demosaicking algorithms, which have been rarely considered, is needed. We believe this simulation will also be useful for other image reconstruction tasks, and we will distribute our code publicly. <s> BIB012 </s> Deep Learning on Image Denoising: An overview <s> Deep learning techniques for real noisy image denoising <s> While deep convolutional neural networks (CNNs) have achieved impressive success in image denoising with additive white Gaussian noise (AWGN), their performance remains limited on real-world noisy photographs. The main reason is that their learned models are easy to overfit on the simplified AWGN model which deviates severely from the complicated real-world noise model. In order to improve the generalization ability of deep CNN denoisers, we suggest training a convolutional blind denoising network (CBDNet) with more realistic noise model and real-world noisy-clean image pairs. On the one hand, both signal-dependent noise and in-camera signal processing pipeline is considered to synthesize realistic noisy images. On the other hand, real-world noisy photographs and their nearly noise-free counterparts are also included to train our CBDNet. To further provide an interactive strategy to rectify denoising result conveniently, a noise estimation subnetwork with asymmetric learning to suppress under-estimation of noise level is embedded into CBDNet. Extensive experimental results on three datasets of real-world noisy pho- tographs clearly demonstrate the superior performance of CBDNet over state-of-the-arts in terms of quantitative met- rics and visual quality. The code has been made available at https://github.com/GuoShi28/CBDNet. <s> BIB013 </s> Deep Learning on Image Denoising: An overview <s> Deep learning techniques for real noisy image denoising <s> Abstract Deep convolutional neural networks (CNNs) have attracted considerable interest in low-level computer vision. Researches are usually devoted to improving the performance via very deep CNNs. However, as the depth increases, influences of the shallow layers on deep layers are weakened. Inspired by the fact, we propose an attention-guided denoising convolutional neural network (ADNet), mainly including a sparse block (SB), a feature enhancement block (FEB), an attention block (AB) and a reconstruction block (RB) for image denoising. Specifically, the SB makes a tradeoff between performance and efficiency by using dilated and common convolutions to remove the noise. The FEB integrates global and local features information via a long path to enhance the expressive ability of the denoising model. The AB is used to finely extract the noise information hidden in the complex background, which is very effective for complex noisy images, especially real noisy images and bind denoising. Also, the FEB is integrated with the AB to improve the efficiency and reduce the complexity for training a denoising model. Finally, a RB aims to construct the clean image through the obtained noise mapping and the given noisy image. Additionally, comprehensive experiments show that the proposed ADNet performs very well in three tasks (i.e. synthetic and real noisy images, and blind denoising) in terms of both quantitative and qualitative evaluations. The code of ADNet is accessible at https://github.com/hellloxiaotian/ADNet . <s> BIB014
The main focus of real applications on deep learning for image denoising has two kinds: single end-to-end CNN and the combination of prior knowledge and CNN. For the first method, changing the network architecture is popular to remove the noise from the given real corrupted image. Multiscale is very effective for image denoising. For example, a CNN comprising of convolution, ReLU and RL employed different phase features to enhance the expressive ability of the low-light image denoising model BIB001 . To overcome the blurry and false image artifacts, a dual U-Net with skip connection was proposed for CT image reconstruction BIB006 . To address resource-constraint problem, Tian et al. BIB008 used a dual CNN with batch renormalization BIB002 , RL and dilated convolutions to deal with real noisy image. According to the nature of light image, two CNNs utilized anisotropic parallax analysis to generate structural parallax information for real noisy images BIB007 . Additionally, using CNN to resolve remote sense BIB009 and medical images  under low-light condition is very effective  . To extract more detailed information, recurrent connections were used to enhance the representative ability to deal with corrupted image in the real world BIB003 BIB010 . To deal with unknown real noisy images, a residual structure was utilized to facilitate low-frequency features, then, an attention mechanism BIB014 can be applied to extract more potential features from channels BIB011 . From the point of view of producing the noisy image, imitating cameral pipelines to construct the degradation model was very effective to filter the real noisy BIB012 . Detailed information of these researches can be shown in Table 4 . BIB014 CNN Gaussian image denoising, blind denoising and real noisy image denoising CNN with attention mechanism and sparse method for image denoising For the second method, combining CNN and prior can better deal with both speed and complex noise task in real noisy image. Zhang et al. BIB004 proposed to use half quadratic splitting (HQS) and CNN to estimate the noise from the given real noisy image. After that, Guo et al. BIB013 proposed a three-phase denoising method. The first phase used Gaussian noise and in-cameral processing pipeline to synthesize noisy image. The synthetic and real noisy images are merged to better represent real noisy images. The second phase used sub-network with asymmetric and total variation losses to estimate the noise of real noisy image. The third phase exploited original noisy image and estimated noise to recover the latent clean image. Additionally, CNN with channel prior was effective for low-light image enhancement BIB005 . To make readers easily observe, we use Table 5 to show the detailed information of these researches.  CNN Real noisy image dneoising GAN with attentive mechanism and noise domain for real noisy image denoising