Variational Bayesian Blind Image Deconvolution: A review <s> B. Image prior models <s> In this paper, we present a blind deconvolution algorithm based on the total variational (TV) minimization method proposed. The motivation for regularizing with the TV norm is that it is extremely effective for recovering edges of images as well as some blurring functions, e.g., motion blur and out-of-focus blur. An alternating minimization (AM)implicit iterative scheme is devised to recover the image and simultaneously identify the point spread function (psf). Numerical results indicate that the iterative scheme is quite robust, converges very fast (especially for discontinuous blur), and both the image and the psf can be recovered under the presence of high noise level. Finally, we remark that psf's without sharp edges, e.g., Gaussian blur, can also be identified through the TV approach. <s> BIB001 </s> Variational Bayesian Blind Image Deconvolution: A review <s> B. Image prior models <s> In this chapter, ensemble learning is applied to the problem of blind source separation and deconvolution of images. It is assumed that the observed images were constructed by mixing a set of images (consisting of independent, identically distributed pixels), convolving the mixtures with unknown blurring filters and then adding Gaussian noise. <s> BIB002 </s> Variational Bayesian Blind Image Deconvolution: A review <s> B. Image prior models <s> In this paper, the blind image deconvolution (BID) problem is addressed using the Bayesian framework. In order to solve for the proposed Bayesian model, we present a new methodology based on a variational approximation, which has been recently introduced for several machine learning problems, and can be viewed as a generalization of the expectation maximization (EM) algorithm. This methodology reaps all the benefits of a "full Bayesian model" while bypassing some of its difficulties. We present three algorithms that solve the proposed Bayesian problem in closed form and can be implemented in the discrete Fourier domain. This makes them very cost effective even for very large images. We demonstrate with numerical experiments that these algorithms yield promising improvements as compared to previous BID algorithms. Furthermore, the proposed methodology is quite general with potential application to other Bayesian models for this and other imaging problems. <s> BIB003 </s> Variational Bayesian Blind Image Deconvolution: A review <s> B. Image prior models <s> Following the hierarchical Bayesian framework for blind deconvolution problems, in this paper, we propose the use of simultaneous autoregressions as prior distributions for both the image and blur, and gamma distributions for the unknown parameters (hyperparameters) of the priors and the image formation noise. We show how the gamma distributions on the unknown hyperparameters can be used to prevent the proposed blind deconvolution method from converging to undesirable image and blur estimates and also how these distributions can be inferred in realistic situations. We apply variational methods to approximate the posterior probability of the unknown image, blur, and hyperparameters and propose two different approximations of the posterior distribution. One of these approximations coincides with a classical blind deconvolution method. The proposed algorithms are tested experimentally and compared with existing blind deconvolution methods <s> BIB004 </s> Variational Bayesian Blind Image Deconvolution: A review <s> B. Image prior models <s> Camera shake during exposure leads to objectionable image blur and ruins many photographs. Conventional blind deconvolution methods typically assume frequency-domain constraints on images, or overly simplified parametric forms for the motion path during camera shake. Real camera motions can follow convoluted paths, and a spatial domain prior can better maintain visually salient image characteristics. We introduce a method to remove the effects of camera shake from seriously blurred images. The method assumes a uniform camera blur over the image and negligible in-plane camera rotation. In order to estimate the blur from the camera shake, the user must specify an image region without saturation effects. We show results for a variety of digital photographs taken from personal photo collections. <s> BIB005 </s> Variational Bayesian Blind Image Deconvolution: A review <s> B. Image prior models <s> A conventional camera captures blurred versions of scene information away from the plane of focus. Camera systems have been proposed that allow for recording all-focus images, or for extracting depth, but to record both simultaneously has required more extensive hardware and reduced spatial resolution. We propose a simple modification to a conventional camera that allows for the simultaneous recovery of both (a) high resolution image information and (b) depth information adequate for semi-automatic extraction of a layered depth representation of the image. ::: Our modification is to insert a patterned occluder within the aperture of the camera lens, creating a coded aperture. We introduce a criterion for depth discriminability which we use to design the preferred aperture pattern. Using a statistical model of images, we can recover both depth information and an all-focus image from single photographs taken with the modified camera. A layered depth map is then extracted, requiring user-drawn strokes to clarify layer assignments in some cases. The resulting sharp image and layered depth map can be combined for various photographic applications, including automatic scene segmentation, post-exposure refocusing, or re-rendering of the scene from an alternate viewpoint. <s> BIB006 </s> Variational Bayesian Blind Image Deconvolution: A review <s> B. Image prior models <s> Image priors based on products have been recognized to offer many advantages because they allow simultaneous enforcement of multiple constraints. However, they are inconvenient for Bayesian inference because it is hard to find their normalization constant in closed form. In this paper, a new Bayesian algorithm is proposed for the image restoration problem that bypasses this difficulty. An image prior is defined by imposing Student-t densities on the outputs of local convolutional filters. A variational methodology, with a constrained expectation step, is used to infer the restored image. Numerical experiments are shown that compare this methodology to previous ones and demonstrate its advantages. <s> BIB007 </s> Variational Bayesian Blind Image Deconvolution: A review <s> B. Image prior models <s> In this paper, we present novel algorithms for total variation (TV) based blind deconvolution and parameter estimation utilizing a variational framework. Using a hierarchical Bayesian model, the unknown image, blur, and hyperparameters for the image, blur, and noise priors are estimated simultaneously. A variational inference approach is utilized so that approximations of the posterior distributions of the unknowns are obtained, thus providing a measure of the uncertainty of the estimates. Experimental results demonstrate that the proposed approaches provide higher restoration performance than non-TV-based methods without any assumptions about the unknown hyperparameters. <s> BIB008 </s> Variational Bayesian Blind Image Deconvolution: A review <s> B. Image prior models <s> In this paper, a new image prior is introduced and used in image restoration. This prior is based on products of spatially weighted total variations (TV). These spatial weights provide this prior with the flexibility to better capture local image features than previous TV based priors. Bayesian inference is used for image restoration with this prior via the variational approximation. The proposed restoration algorithm is fully automatic in the sense that all necessary parameters are estimated from the data and is faster than previous similar algorithms. Numerical experiments are shown which demonstrate that image restoration based on this prior compares favorably with previous state-of-the-art restoration algorithms. <s> BIB009 </s> Variational Bayesian Blind Image Deconvolution: A review <s> B. Image prior models <s> In this paper we propose a novel blind image deconvolution method developed within the Bayesian framework. A variant of the non-convex l p -norm prior with 0 < p < 1 is used as the image prior and a total variation (TV) based prior is utilized as the blur prior. The proposed method is derived by utilizing bounds for both the image and blur priors using the majorization-minimization principle. Maximum a posteriori Bayesian inference is performed and as a result, the unknown image, blur and model parameters are simultaneously estimated. We also show that as a special case, the developed method provides very competitive non-blind image restoration results when the blurring function is assumed to be known. Experimental results are presented to demonstrate the advantage of the proposed method compared to existing ones. <s> BIB010 </s> Variational Bayesian Blind Image Deconvolution: A review <s> B. Image prior models <s> We present a general method for blind image deconvolution using Bayesian inference with super-Gaussian sparse image priors. We consider a large family of priors suitable for modeling natural images, and develop the general procedure for estimating the unknown image and the blur. Our formulation includes a number of existing modeling and inference methods as special cases while providing additional flexibility in image modeling and algorithm design. We also present an analysis of the proposed inference compared to other methods and discuss its advantages. Theoretical and experimental results demonstrate that the proposed formulation is very effective, efficient, and flexible. <s> BIB011 </s> Variational Bayesian Blind Image Deconvolution: A review <s> B. Image prior models <s> Typical blur from camera shake often deviates from the standard uniform convolutional assumption, in part because of problematic rotations which create greater blurring away from some unknown center point. Consequently, successful blind deconvolution for removing shake artifacts requires the estimation of a spatially-varying or non-uniform blur operator. Using ideas from Bayesian inference and convex analysis, this paper derives a simple non-uniform blind deblurring algorithm with a spatially-adaptive image penalty. Through an implicit normalization process, this penalty automatically adjust its shape based on the estimated degree of local blur and image structure such that regions with large blur or few prominent edges are discounted. Remaining regions with modest blur and revealing edges therefore dominate on average without explicitly incorporating structure-selection heuristics. The algorithm can be implemented using an optimization strategy that is virtually tuning-parameter free and simpler than existing methods, and likely can be applied in other settings such as dictionary learning. Detailed theoretical analysis and empirical comparisons on real images serve as validation. <s> BIB012 </s> Variational Bayesian Blind Image Deconvolution: A review <s> B. Image prior models <s> Blind deconvolution is the problem of recovering a sharp image and a blur kernel from a noisy blurry image. Recently, there has been a significant effort on understanding the basic mechanisms to solve blind deconvolution. While this effort resulted in the deployment of effective algorithms, the theoretical findings generated contrasting views on why these approaches worked. On the one hand, one could observe experimentally that alternating energy minimization algorithms converge to the desired solution. On the other hand, it has been shown that such alternating minimization algorithms should fail to converge and one should instead use a so-called Variational Bayes approach. To clarify this conundrum, recent work showed that a good image and blur prior is instead what makes a blind deconvolution algorithm work. Unfortunately, this analysis did not apply to algorithms based on total variation regularization. In this manuscript, we provide both analysis and experiments to get a clearer picture of blind deconvolution. Our analysis reveals the very reason why an algorithm based on total variation works. We also introduce an implementation of this algorithm and show that, in spite of its extreme simplicity, it is very robust and achieves a performance comparable to the state of the art. <s> BIB013 </s> Variational Bayesian Blind Image Deconvolution: A review <s> B. Image prior models <s> Iteratively reweighted least squares (IRLS) is one of the most effective methods to minimize the l p regularized linear inverse problem. Unfortunately, the regularizer is nonsmooth and nonconvex when 0 < p < 1. In spite of its properties and mainly due to its high computation cost, IRLS is not widely used in image deconvolution and reconstruction. In this paper, we first derive the IRLS method from the perspective of majorization minimization and then propose an Alternating Direction Method of Multipliers (ADMM) to solve the reweighted linear equations. Interestingly, the resulting algorithm has a shrinkage operator that pushes each component to zero in a multiplicative fashion. Experimental results on both image deconvolution and reconstruction demonstrate that the proposed method outperforms state-of-the-art algorithms in terms of speed and recovery quality. <s> BIB014
An unquestionable landmark on the recent history of BID is the paper by Miskin and MacKay BIB002 . In that work the authors propose the use of a mixture of Laplacians to restore cartoon images and utilize, for the first time in the BID literature, VB inference (to be described later) to restore the observed image. Later, Likas and Galatsanos BIB003 proposed a Gaussian prior to impose smoothness on the image and blur, see also BIB004 , and Fergus et al. BIB005 proposed a mixture-of-Gaussians (MOG) to impose sparsity.  to impose piecewisesmoothness. The TV prior model has been frequently used in BID, see for instance BIB001 - BIB013 , see also BIB007 and BIB009 . Fergus et al. BIB005 represents the first publication on the use of filtered versions of the original  image to estimate image and blur. The use in BIB008 of majorization methods with variational inference and diagonal covariance approximation led to a new way to approach BID in image processing (not widely acknowledged in the computer vision community). As we will see in the following, the TV prior used in BIB008 is a particular case of the use of Super Gaussian Distributions in BID. Since the influential work of Fergus et al. BIB005 , sparse prior models have attracted the attention of BID researchers and are, in our opinion, rightly considered to be the state of the art representation in filtered domains. It is a well known fact that when high-pass filters are applied to natural images, the resulting coefficients are sparse; i.e., most of the coefficients are zero or very small while only a small number of them are large (e.g., at the edges). This is a very important characteristic that should be taken into account when restoring natural images. The p prior has been used in a large number of works such BIB008 , BIB010 , BIB006 - BIB014 . They use a prior distribution based on the minimization of quasi-norms · p p with 0 < p ≤ 1. Levin et al. BIB006 suggest the use of p in the range [0.6, 0.8] for natural images. The Super-Gaussianity property presented by Palmer in  , was used in Babacan et al. BIB011 as the building block to propose a general representation for sparse priors. As we will see, almost all previous and very recently proposed prior models can be represented using SG. This representation is used in the same work BIB011 to introduce two new image priors log and exp. Recent models like the one proposed by Zhang and Wifp BIB012 , or the Student-t prior recently proposed by Mohammad-Djafari  are particular cases of SG distributions.