Variational Bayesian Blind Image Deconvolution: A review <s> C. Blur models <s> Restoring a clear image from a single motion-blurred image due to camera shake has long been a challenging problem in digital imaging. Existing blind deblurring techniques either only remove simple motion blurring, or need user interactions to work on more complex cases. In this paper, we present an approach to remove motion blurring from a single image by formulating the blind blurring as a new joint optimization problem, which simultaneously maximizes the sparsity of the blur kernel and the sparsity of the clear image under certain suitable redundant tight frame systems (curvelet system for kernels and framelet system for images). Without requiring any prior information of the blur kernel as the input, our proposed approach is able to recover high-quality images from given blurred images. Furthermore, the new sparsity constraints under tight frame systems enable the application of a fast algorithm called linearized Bregman iteration to efficiently solve the proposed minimization problem. The experiments on both simulated images and real images showed that our algorithm can effectively removing complex motion blurring from nature images. <s> BIB001 </s> Variational Bayesian Blind Image Deconvolution: A review <s> C. Blur models <s> Blind deconvolution is the recovery of a sharp version of a blurred image when the blur kernel is unknown. Recent algorithms have afforded dramatic progress, yet many aspects of the problem remain challenging and hard to understand. The goal of this paper is to analyze and evaluate recent blind deconvolution algorithms both theoretically and experimentally. We explain the previously reported failure of the naive MAP approach by demonstrating that it mostly favors no-blur explanations. We show that, using reasonable image priors, a naive simulations MAP estimation of both latent image and blur kernel is guaranteed to fail even with infinitely large images sampled from the prior. On the other hand, we show that since the kernel size is often smaller than the image size, a MAP estimation of the kernel alone is well constrained and is guaranteed to succeed to recover the true blur. The plethora of recent deconvolution techniques makes an experimental evaluation on ground-truth data important. As a first step toward this experimental evaluation, we have collected blur data with ground truth and compared recent algorithms under equal settings. Additionally, our data demonstrate that the shift-invariant blur assumption made by most algorithms is often violated. <s> BIB002 </s> Variational Bayesian Blind Image Deconvolution: A review <s> C. Blur models <s> We present a general method for blind image deconvolution using Bayesian inference with super-Gaussian sparse image priors. We consider a large family of priors suitable for modeling natural images, and develop the general procedure for estimating the unknown image and the blur. Our formulation includes a number of existing modeling and inference methods as special cases while providing additional flexibility in image modeling and algorithm design. We also present an analysis of the proposed inference compared to other methods and discuss its advantages. Theoretical and experimental results demonstrate that the proposed formulation is very effective, efficient, and flexible. <s> BIB003 </s> Variational Bayesian Blind Image Deconvolution: A review <s> C. Blur models <s> This paper presents a new Metropolis-adjusted Langevin algorithm (MALA) that uses convex analysis to simulate efficiently from high-dimensional densities that are log-concave, a class of probability distributions that is widely used in modern high-dimensional statistics and data analysis. The method is based on a new first-order approximation for Langevin diffusions that exploits log-concavity to construct Markov chains with favourable convergence properties. This approximation is closely related to Moreau-Yoshida regularisations for convex functions and uses proximity mappings instead of gradient mappings to approximate the continuous-time process. The proposed method complements existing MALA methods in two ways. First, the method is shown to have very robust stability properties and to converge geometrically for many target densities for which other MALA are not geometric, or only if the step size is sufficiently small. Second, the method can be applied to high-dimensional target densities that are not continuously differentiable, a class of distributions that is increasingly used in image processing and machine learning and that is beyond the scope of existing MALA and HMC algorithms. To use this method it is necessary to compute or to approximate efficiently the proximity mappings of the logarithm of the target density. For several popular models, including many Bayesian models used in modern signal and image processing and machine learning, this can be achieved with convex optimisation algorithms and with approximations based on proximal splitting techniques, which can be implemented in parallel. The proposed method is demonstrated on two challenging high-dimensional and non-differentiable models related to image resolution enhancement and low-rank matrix estimation that are not well addressed by existing MCMC methodology. <s> BIB004 </s> Variational Bayesian Blind Image Deconvolution: A review <s> C. Blur models <s> Blur kernel estimation is a crucial step in the deblurring process for images. Estimation of the kernel, especially in the presence of noise, is easily perturbed, and the quality of the resulting deblurred images is hence degraded. Since every motion blur in a single exposure image can be represented by 2D parametric curves, we adopt a piecewise-linear model to approximate the curves for the reliable blur kernel estimation. The model is found to be an effective tradeoff between flexibility and robustness as it takes advantage of two extremes: (1) the generic model, represented by a discrete 2D function, which has a high degree of freedom (DOF) for the maximum flexibility but suffers from noise and (2) the linear model, which enhances robustness and simplicity but has limited expressiveness due to its low DOF. We evaluate several deblurring methods based on not only the generic model, but also the piecewise-linear model as an alternative. After analyzing the experiment results using real-world images with significant levels of noise and a benchmark data set, we conclude that the proposed model is not only robust with respect to noise, but also flexible in dealing with various types of blur. <s> BIB005 </s> Variational Bayesian Blind Image Deconvolution: A review <s> C. Blur models <s> Blind deconvolution is to recover a sharp version of a given blurry image or signal when the blur kernel is unknown. Because this problem is ill-conditioned in nature, effectual criteria pertaining to both the sharp image and blur kernel are required to constrain the space of candidate solutions. While the problem has been extensively studied for long, it is still unclear how to regularize the blur kernel in an elegant, effective fashion. In this paper, we show that the blurry image itself actually encodes rich information about the blur kernel, and such information can indeed be found by exploring and utilizing a well-known phenomenon, that is, sharp images are often high pass, whereas blurry images are usually low pass. More precisely, we shall show that the blur kernel can be retrieved through analyzing and comparing how the spectrum of an image as a convolution operator changes before and after blurring. Subsequently, we establish a convex kernel regularizer, which depends only on the given blurry image. Interestingly, the minimizer of this regularizer guarantees to give a good estimate to the desired blur kernel if the original image is sharp enough. By combining this powerful regularizer with the prevalent nonblind devonvolution techniques, we show how we could significantly improve the deblurring results through simulations on synthetic images and experiments on realistic images. <s> BIB006 </s> Variational Bayesian Blind Image Deconvolution: A review <s> C. Blur models <s> Blind deconvolution is the problem of recovering a sharp image and a blur kernel from a noisy blurry image. Recently, there has been a significant effort on understanding the basic mechanisms to solve blind deconvolution. While this effort resulted in the deployment of effective algorithms, the theoretical findings generated contrasting views on why these approaches worked. On the one hand, one could observe experimentally that alternating energy minimization algorithms converge to the desired solution. On the other hand, it has been shown that such alternating minimization algorithms should fail to converge and one should instead use a so-called Variational Bayes approach. To clarify this conundrum, recent work showed that a good image and blur prior is instead what makes a blind deconvolution algorithm work. Unfortunately, this analysis did not apply to algorithms based on total variation regularization. In this manuscript, we provide both analysis and experiments to get a clearer picture of blind deconvolution. Our analysis reveals the very reason why an algorithm based on total variation works. We also introduce an implementation of this algorithm and show that, in spite of its extreme simplicity, it is very robust and achieves a performance comparable to the state of the art. <s> BIB007
Although the above described prior models were proposed for the image, all of them can also be used for the blur as well. The BID literature also contains specific blur models which we now describe. Molina et al.  propose a Dirichlet prior for kernel modeling. Since the curvelet representation can take into account both the continuity and sparsity of the motion blur kernel, Cai et al. BIB001 suggest the use of this representation for this type of blur. Oh and Kim BIB005 propose a piecewise-linear model for motion blur in order to reduce the dimensionality of the solution space and make the kernel estimation process more robust. Based on the assumption that the power spectrum of natural images drops quadratically as the frequency increases Goldstein and Fattal  introduce a power spectrum prior on the blur kernel. Recently, a novel convex blur regularizer based on the spectral properties of the convolution operators can be found in BIB006 . Since the spectral properties used are based on a linear and shift-invariant model without considering the noise, these methods do not work well for spatially varying blurs and noisy observations. Let us consider the observation model in Eq. (1) and assume that the original image x is known. In this case we have N observations and aim at estimating K coefficients, where K is the size of the blur. Since the image size is usually much larger than the blur size, N observations should be sufficient to obtain a good blur estimate, even more so if L filtered observations are used. Based on the fact that usually K << N , many authors BIB007 , BIB003 , BIB002 , BIB004 have recently advocated the use of flat priors on the blur, enforcing only its nonnegativity and normalization constraints.