Demosaicing, denoising and super-resolution (SR) are of practical importance in digital image processing and have been studied independently in the passed decades. Despite the recent improvement of learning-based image processing methods in image quality, there lacks enough analysis into their interactions and characteristics under a realistic setting of the mixture problem of demosaicing, denoising and SR. In existing solutions, these tasks are simply combined to obtain a high-resolution image from a low-resolution raw mosaic image, resulting in a performance drop of the final image quality. In this paper, we first rethink the mixture problem from a holistic perspective and then propose the Trinity Enhancement Network (TENet), a specially designed learning-based method for the mixture problem, which adopts a novel image processing pipeline order and a joint learning strategy. In order to obtain the correct color sampling for training, we also contribute a new dataset namely PixelShift200, which consists of high-quality full color sampled real-world images using the advanced pixel shift technique. Experiments demonstrate that our TENet is superior to existing solutions in both quantitative and qualitative perspective. Our experiments also show the necessity of the proposed PixelShift200 dataset.
Image demosaicing is an ill-posed problem of interpolating full-resolution color images from the color mosaic images (e.g. Bayer mosaic images), and is usually preformed in the beginning of image processing pipeline. Existing approaches can be mainly classified into two categories: model-based and learning-based methods. Model-based approaches @cite_41 @cite_13 @cite_47 @cite_18 @cite_46 @cite_19 focus on the construction of mathematical models and image priors in the spatial-spectral domain facilitating the recovery of missing data. Learning-based approaches @cite_19 @cite_32 build the process mapping by learning from abundant training data. Recently, deep learning has also used successfully for image demosaicing and achieved competitive performance @cite_4 @cite_39 @cite_23 @cite_53 . Micha "el @cite_23 train a deep convolutional neural network (CNN) on millions of carefully selected image patches and achieve the state-of-the-art performance of demosaicking.
@cite_18 @cite_4 @cite_41 @cite_46 @cite_53 @cite_32 @cite_39 @cite_19 @cite_23 @cite_47 @cite_13
2437754452 2556872594 2140341478 2069051299
Demosaicing is an important first step for color image acquisition. For practical reasons, demosaicing algorithms have to be both efficient and yield high quality results in the presence of noise. The demosaicing problem poses several challenges, e.g. zippering and false color artifacts as well as edge blur. In this work, we introduce a novel learning based method that can overcome these challenges. We formulate demosaicing as an image restoration problem and propose to learn efficient regularization inspired by a variational energy minimization framework that can be trained for different sensor layouts. Our algorithm performs joint demosaicing and denoising in close relation to the real physical mosaicing process on a camera sensor. This is achieved by learning a sequence of energy minimization problems composed of a set of RGB filters and corresponding activation functions. We evaluate our algorithm on the Microsoft Demosaicing data set in terms of peak signal to noise ratio (PSNR) and structured similarity index (SSIM). Our algorithm is highly efficient both in image quality and run time. We achieve an improvement of up to 2.6 dB over recent state-of-the-art algorithms. Demosaicking and denoising are the key first stages of the digital imaging pipeline but they are also a severely ill-posed problem that infers three color values per pixel from a single noisy measurement. Earlier methods rely on hand-crafted filters or priors and still exhibit disturbing visual artifacts in hard cases such as moire or thin edges. We introduce a new data-driven approach for these challenges: we train a deep neural network on a large corpus of images instead of using hand-tuned filters. While deep learning has shown great success, its naive application using existing training datasets does not give satisfactory results for our problem because these datasets lack hard cases. To create a better training set, we present metrics to identify difficult patches and techniques for mining community photographs for such patches. Our experiments show that this network and training procedure outperform state-of-the-art both on noisy and noise-free data. Furthermore, our algorithm is an order of magnitude faster than the previous best performing techniques. Demosaicing is a process of obtaining a full-color image by interpolating the missing colors of an image captured from a single sensor color filter array. This paper provides an effective and low-complexity iterative demosaicing algorithm applying a weighted-edge interpolation to handle green pixels followed by a series of color-difference interpolation to update red, blue, and green pixels. Based on our experiments of images, we enable the algorithm a well-designed stopping condition and pre-determine the proper weights of interpolation. Experimental results show that the proposed method performs much better than three state-of-the-art demosaicing techniques in terms of both computational cost and image quality. In comparison to the algorithm of successive approximation, the algorithm proposed here reduces mean squared error up to 14.5 while requiring computational cost only 22 on average. That is, it takes less time but performs better. We introduce a machine learning approach to demosaicing, the reconstruction of color images from incomplete color filter array samples. There are two challenges to overcome by a demosaicing method: 1) it needs to model and respect the statistics of natural images in order to reconstruct natural looking images and 2) it should be able to perform well in the presence of noise. To facilitate an objective assessment of current methods, we introduce a public ground truth data set of natural images suitable for research in image demosaicing and denoising. We then use this large data set to develop a machine learning approach to demosaicing. Our proposed method addresses both demosaicing challenges by learning a statistical model of images and noise from hundreds of natural images. The resulting model performs simultaneous demosaicing and denoising. We show that the machine learning approach has a number of benefits: 1) the model is trained to directly optimize a user-specified performance measure such as peak signal-to-noise ratio (PSNR) or structural similarity; 2) we can handle novel color filter array layouts by retraining the model on such layouts; and 3) it outperforms the previous state-of-the-art, in some setups by 0.7-dB PSNR, faithfully reconstructing edges, textures, and smooth areas. Our results demonstrate that in demosaicing and related imaging applications, discriminatively trained machine learning models have the potential for peak performance at comparatively low engineering effort.