Variational Bayesian Blind Image Deconvolution: A review <s> C. Algorithm <s> Camera shake during exposure leads to objectionable image blur and ruins many photographs. Conventional blind deconvolution methods typically assume frequency-domain constraints on images, or overly simplified parametric forms for the motion path during camera shake. Real camera motions can follow convoluted paths, and a spatial domain prior can better maintain visually salient image characteristics. We introduce a method to remove the effects of camera shake from seriously blurred images. The method assumes a uniform camera blur over the image and negligible in-plane camera rotation. In order to estimate the blur from the camera shake, the user must specify an image region without saturation effects. We show results for a variety of digital photographs taken from personal photo collections. <s> BIB001 </s> Variational Bayesian Blind Image Deconvolution: A review <s> C. Algorithm <s> A conventional camera captures blurred versions of scene information away from the plane of focus. Camera systems have been proposed that allow for recording all-focus images, or for extracting depth, but to record both simultaneously has required more extensive hardware and reduced spatial resolution. We propose a simple modification to a conventional camera that allows for the simultaneous recovery of both (a) high resolution image information and (b) depth information adequate for semi-automatic extraction of a layered depth representation of the image. ::: Our modification is to insert a patterned occluder within the aperture of the camera lens, creating a coded aperture. We introduce a criterion for depth discriminability which we use to design the preferred aperture pattern. Using a statistical model of images, we can recover both depth information and an all-focus image from single photographs taken with the modified camera. A layered depth map is then extracted, requiring user-drawn strokes to clarify layer assignments in some cases. The resulting sharp image and layered depth map can be combined for various photographic applications, including automatic scene segmentation, post-exposure refocusing, or re-rendering of the scene from an alternate viewpoint. <s> BIB002 </s> Variational Bayesian Blind Image Deconvolution: A review <s> C. Algorithm <s> This paper presents a fast deblurring method that produces a deblurring result from a single image of moderate size in a few seconds. We accelerate both latent image estimation and kernel estimation in an iterative deblurring process by introducing a novel prediction step and working with image derivatives rather than pixel values. In the prediction step, we use simple image processing techniques to predict strong edges from an estimated latent image, which will be solely used for kernel estimation. With this approach, a computationally efficient Gaussian prior becomes sufficient for deconvolution to estimate the latent image, as small deconvolution artifacts can be suppressed in the prediction. For kernel estimation, we formulate the optimization function using image derivatives, and accelerate the numerical process by reducing the number of Fourier transforms needed for a conjugate gradient method. We also show that the formulation results in a smaller condition number of the numerical system than the use of pixel values, which gives faster convergence. Experimental results demonstrate that our method runs an order of magnitude faster than previous work, while the deblurring quality is comparable. GPU implementation facilitates further speed-up, making our method fast enough for practical use. <s> BIB003 </s> Variational Bayesian Blind Image Deconvolution: A review <s> C. Algorithm <s> Learning good image priors is of utmost importance for the study of vision, computer vision and image processing applications. Learning priors and optimizing over whole images can lead to tremendous computational challenges. In contrast, when we work with small image patches, it is possible to learn priors and perform patch restoration very efficiently. This raises three questions - do priors that give high likelihood to the data also lead to good performance in restoration? Can we use such patch based priors to restore a full image? Can we learn better patch priors? In this work we answer these questions. We compare the likelihood of several patch models and show that priors that give high likelihood to data perform better in patch restoration. Motivated by this result, we propose a generic framework which allows for whole image restoration using any patch based prior for which a MAP (or approximate MAP) estimate can be calculated. We show how to derive an appropriate cost function, how to optimize it and how to use it to restore whole images. Finally, we present a generic, surprisingly simple Gaussian Mixture prior, learned from a set of natural images. When used with the proposed framework, this Gaussian Mixture Model outperforms all other generic prior methods for image denoising, deblurring and inpainting. <s> BIB004 </s> Variational Bayesian Blind Image Deconvolution: A review <s> C. Algorithm <s> Blind image deconvolution is an ill-posed problem that requires regularization to solve. However, many common forms of image prior used in this setting have a major drawback in that the minimum of the resulting cost function does not correspond to the true sharp solution. Accordingly, a range of additional methods are needed to yield good results (Bayesian methods, adaptive cost functions, alpha-matte extraction and edge localization). In this paper we introduce a new type of image regularization which gives lowest cost for the true sharp image. This allows a very simple cost formulation to be used for the blind deconvolution model, obviating the need for additional methods. Due to its simplicity the algorithm is fast and very robust. We demonstrate our method on real images with both spatially invariant and spatially varying blur. <s> BIB005 </s> Variational Bayesian Blind Image Deconvolution: A review <s> C. Algorithm <s> Blind image deconvolution (BID) is a severely ill-posed problem which requires prior information on the latent image to estimate the blur kernel. In this paper, a new observation that blurring always pushes the gradient of a local image region toward its mean value is introduced. And we formulate a novel function to measure the distance between the local gradient and its mean value. A novel regularizer associated with local gradient means is proposed. As it requires to segment the whole image into small regions, we propose an approximate method without any segmentation. Thanks to its simplicity the algorithm is fast and robust. Numerous experimental results on synthetic and real data demonstrate that our method is capable of removing various uniform blurs such as motion blur, atmospheric blur and out-of-focus blur. <s> BIB006 </s> Variational Bayesian Blind Image Deconvolution: A review <s> C. Algorithm <s> In image deconvolution, various boundary conditions (BC) based deconvolution methods have been proposed to reduce boundary artifacts. However, most of them are not considering the accuracy of BC due to computation limitation. In this paper, we propose a BC based deconvolution framework, which considers the convolution matrix as a product of partial convolution matrix and boundary condition matrix. By computing the adjoint matrix of boundary condition matrix, we can solve this large linear system with conjugate gradient algorithm. With this framework, we can easily derive two efficient non-blind image deconvolution algorithms, which treat the borders of image as repeated instances of the edge pixel values and unknown variables, respectively. Experiments on synthetic data and real data are both presented to show the performance of various BCs. Our conclusion is that undetermined BC usually has the best performance, and repeated BC outperforms undetermined BC if the latent image has high local similarity around the boundary. <s> BIB007
The VB based blind deconvolution algorithm is presented in Alg. 1. However, as pointed out by Fergus et al. BIB001 , directly applying it to estimate the blur may end up in the local minima, especially when the kernel support is large. To handle the large blur support problem, they suggest using a multiscale approach, namely building an image pyramid and then applying the BID method at each scale, which has proved to be very effective in BID problems. The rationale is that at the coarsest level, the blur is reduced significantly, so that it is easy to estimate a kernel from the downsampled image. At the next finer level, this kernel estimate is upsampled and can be used as a good initial guess for the single scale BID. Repeating this process until the finest level, we can obtain a better kernel estimate. After kernel estimation, we reconstruct the final sharp image using a non-blind deconvolution method (e.g., BIB007 , BIB002 , BIB004 ). The computation in Alg. 1 is dominated by the solution of Eqs.  and  . Since the most time consuming part when solving these two equations is the 2-D convolution, the computational complexity while not converge do x is inverted exactly. The number of iterations required for convergence depends on the image priors. For example, the use of log prior leads to faster convergence than the use of 0.8 prior, because the log prior is more edge preserving and sparsity promoting than the 0.8 prior. It is also shown in BIB006 that, for the same optimization method and parameter settings, the use of the normalized 1 prior BIB006 results in fewer iterations for convergence than the 1 / 2 prior BIB005 in the kernel estimation step. Due to the use of the covariance matrix, the VB BID method is slower than the MAP method BIB005 [37] BIB006 , and much slower than the edge prediction based methods BIB003  .