A Survey on the New Generation of Deep Learning in Image Processing <s> G. IMAGE DENOISING <s> The discriminative model learning for image denoising has been recently attracting considerable attentions due to its favorable denoising performance. In this paper, we take one step forward by investigating the construction of feed-forward denoising convolutional neural networks (DnCNNs) to embrace the progress in very deep architecture, learning algorithm, and regularization method into image denoising. Specifically, residual learning and batch normalization are utilized to speed up the training process as well as boost the denoising performance. Different from the existing discriminative denoising models which usually train a specific model for additive white Gaussian noise at a certain noise level, our DnCNN model is able to handle Gaussian denoising with unknown noise level (i.e., blind Gaussian denoising). With the residual learning strategy, DnCNN implicitly removes the latent clean image in the hidden layers. This property motivates us to train a single DnCNN model to tackle with several general image denoising tasks, such as Gaussian denoising, single image super-resolution, and JPEG image deblocking. Our extensive experiments demonstrate that our DnCNN model can not only exhibit high effectiveness in several general image denoising tasks, but also be efficiently implemented by benefiting from GPU computing. <s> BIB001 </s> A Survey on the New Generation of Deep Learning in Image Processing <s> G. IMAGE DENOISING <s> Variations of deep neural networks such as convolutional neural network (CNN) have been successfully applied to image denoising. The goal is to automatically learn a mapping from a noisy image to a clean image given training data consisting of pairs of noisy and clean images. Most existing CNN models for image denoising have many layers. In such cases, the models involve a large amount of parameters and are computationally expensive to train. In this paper, we develop a dilated residual CNN for Gaussian image denoising. Compared with the recently proposed residual denoiser, our method can achieve comparable performance with less computational cost. Specifically, we enlarge receptive field by adopting dilated convolution in residual network, and the dilation factor is set to a certain value. We utilize appropriate zero padding to make the dimension of the output the same as the input. It has been proven that the expansion of receptive field can boost the CNN performance in image classification, and we further demonstrate that it can also lead to competitive performance for denoising problem. Moreover, we present a formula to calculate receptive field size when dilated convolution is incorporated. Thus, the change of receptive field can be interpreted mathematically. To validate the efficacy of our approach, we conduct extensive experiments for both gray and color image denoising with specific or randomized noise levels. Both of the quantitative measurements and the visual results of denoising are promising comparing with state-of-the-art baselines. <s> BIB002 </s> A Survey on the New Generation of Deep Learning in Image Processing <s> G. IMAGE DENOISING <s> Due to the fast inference and good performance, discriminative learning methods have been widely studied in image denoising. However, these methods mostly learn a specific model for each noise level, and require multiple models for denoising images with different noise levels. They also lack flexibility to deal with spatially variant noise, limiting their applications in practical denoising. To address these issues, we present a fast and flexible denoising convolutional neural network, namely FFDNet, with a tunable noise level map as the input. The proposed FFDNet works on downsampled sub-images to speed up the inference, and adopts orthogonal regularization to enhance the generalization ability. In contrast to the existing discriminative denoisers, FFDNet enjoys several desirable properties, including (i) the ability to handle a wide range of noise levels (i.e., [0, 75]) effectively with a single network, (ii) the ability to remove spatially variant noise by specifying a non-uniform noise level map, and (iii) faster speed than benchmark BM3D even on CPU without sacrificing denoising performance. Extensive experiments on synthetic and real noisy images are conducted to evaluate FFDNet in comparison with state-of-the-art denoisers. The results show that FFDNet is effective and efficient, making it highly attractive for practical denoising applications. <s> BIB003 </s> A Survey on the New Generation of Deep Learning in Image Processing <s> G. IMAGE DENOISING <s> Since the proposal of big data analysis and Graphic Processing Unit (GPU), the deep learning technology has received a great deal of attention and has been widely applied in the field of imaging processing. In this paper, we have an aim to completely review and summarize the deep learning technologies for image denoising proposed in recent years. Morever, we systematically analyze the conventional machine learning methods for image denoising. Finally, we point out some research directions for the deep learning technologies in image denoising. <s> BIB004 </s> A Survey on the New Generation of Deep Learning in Image Processing <s> G. IMAGE DENOISING <s> Recent years have witnessed the unprecedented success of deep convolutional neural networks (CNNs) in single image super-resolution (SISR). However, existing CNN-based SISR methods mostly assume that a low-resolution (LR) image is bicubicly downsampled from a high-resolution (HR) image, thus inevitably giving rise to poor performance when the true degradation does not follow this assumption. Moreover, they lack scalability in learning a single model to deal with multiple degradations. To address these issues, we propose a dimensionality stretching strategy that enables a single convolutional super-resolution network to take two key factors of the SISR degradation process, i.e., blur kernel and noise level, as input. Consequently, the proposed super-resolver can handle multiple and even spatially variant degradations, which significantly improves the practicability. Extensive experimental results on synthetic and real LR images show that the proposed convolutional super-resolution network not only can produce favorable results on multiple degradations but also is computationally efficient, providing a highly effective and scalable solution to practical SISR applications. <s> BIB005
Image denoising can be described as the problem of mapping from a noisy image to a noise-free image. The best currently available denoising methods approximate this mapping with cleverly deep learning algorithms. Deep learning technologies can be chosen for image denoising based on the following three reasons. First, deep network architecture can learn more extractions. Second, BN and ReLU can accelerate the training speed of deep networks. Third, deep learning models can train more samples and improve efficiency employing GPU BIB004 . DnCNNs can use BN and ResNet to perform image denoising BIB001 . This framework not only deals with blind image denoising, but also addresses image super-resolution task, and JPEG image deblocking. Fig.31 illustrates the architecture of the DnCNN. The input of our DnCNN is a noisy observation of y = x + z, where z is additive noise, and x is a clean image. One can adopt the residual learning formulation to train a residual mapping R(y) ≈ z, and then we have a clean image of x = y − R(y). For N noisy-clean training image (patch) pairs (x i , y i ) N i=1 , then we have the following optimization object function, where θ denotes the trainable parameters in DnCNNs, experiments demonstrate that DnCNN can exhibit high effectiveness in several general image denoising tasks. Many other typical methods also obtain excellent performance for image denoising. For example, the fusion of the dilated convolution and ResNet is used for image denoising, and this framework is fit for combing disparate sources of experts of image denoising BIB002 . FFDNet can uses noise level map and noisy image as an input to deal with different noise levels BIB003 . IRCNN can fuses the model-based optimization method and CNN to perform image denoising, which can deal with different inverse problems and multiple tasks with one single-mode BIB005 .