The majority of cost-effective digital video cameras use a single sensor with a color filler array (CFA) to capture visual scenes in color. At each pixel, only one of the three primary colors is sampled and the missing color components need to be interpolated. The color interpolation process is usually referred to as color demosaicking (CDM). For video sequences, the temporal correlation in adjacent frames can also be exploited to improve the performance of CDM. The majority of CDM algorithms assume that the CFA data is noise-free. This assumption, however, is unrealistic and the presence of noise in the CFA data will cause serious CDM artifacts that are difficult to be removed by a subsequent denoising process. Many available denoising algorithms are designed for monochromatic/full-color images/videos and they are not directly applicable to the CFA data. Therefore, it is highly desirable to provide new algorithms for full-color video reconstruction from noisy CFA sequences.
A commonly used strategy is to demosaick the CFA video first and then later denoise the demosaicked video. However, the noise-caused color artifacts generated in the CDM process can be difficult to remove in the following denoising process. Some joint demosaicking and denoising approaches have been proposed. In a joint demosaicking-denoising algorithm, both demosaicking and denoising are treated as an estimation problem and solved by the total least square technique. One computationally more efficient joint demosaicking-denoising scheme first performs demosaicking-denoising on the green channel, and then uses the restored green channel to estimate the noise statistics to restore the red and blue channels. Inspired by the directional linear minimum mean square-error estimation based CDM scheme, a nonlinear and spatially adaptive filler has been proposed by using local polynomial approximation for CDM. This scheme is adapted to noisy CFA inputs for joint demosaicking-denoising.
In addition to the two abovementioned strategies, another approach to reproduce full-color images from noisy CFA data is to perform denoising before demosaicking. However, due to the mosaic structure of CFA, existing monochromatic image/video denoising methods cannot be directly applied to the CFA data. The CFA image can be divided into several monochromatic sub-images using the approaches known from the CFA image compression literature. However, such approaches do not exploit the inter-channel correlation which is useful to reduce noise. A wavelet based CFA image denoising method has been proposed. A principal component analysis (PCA) is proposed based CFA image denoising scheme. A CFA block, which contains color samples from different channels, is used to analyze the local CFA image structure. By transforming the noisy signal into the PCA domain, the signal energy can be better clustered and the noise can be more effectively removed. The PCA-based CFA image denoising algorithm effectively exploits the spatial and spectral correlation in each frame. This method was later extended to gray level images.