Variational Bayesian Blind Image Deconvolution: A review <s> A. Variational Inference in the image space for Super Gaussian priors <s> Classical equilibrium statistical mechanics magnetic systems the Ising model the low-temperature and high-temperature expansions the Landau-Ginsberg model near the transition the renormalization group perturbative evaluation of the critical exponents near four dimensions on spontaneous symmetry breaking other models the transfer matrix path integrals for quantum mechanics semiclassical methods relativistic quantum field theory particle-field duality time dependent correlations the approach to equilibrium the stochastic approach to equilibrium the stochastic approach computer simulation. <s> BIB001 </s> Variational Bayesian Blind Image Deconvolution: A review <s> A. Variational Inference in the image space for Super Gaussian priors <s> In this paper, we present a blind deconvolution algorithm based on the total variational (TV) minimization method proposed. The motivation for regularizing with the TV norm is that it is extremely effective for recovering edges of images as well as some blurring functions, e.g., motion blur and out-of-focus blur. An alternating minimization (AM)implicit iterative scheme is devised to recover the image and simultaneously identify the point spread function (psf). Numerical results indicate that the iterative scheme is quite robust, converges very fast (especially for discontinuous blur), and both the image and the psf can be recovered under the presence of high noise level. Finally, we remark that psf's without sharp edges, e.g., Gaussian blur, can also be identified through the TV approach. <s> BIB002 </s> Variational Bayesian Blind Image Deconvolution: A review <s> A. Variational Inference in the image space for Super Gaussian priors <s> Convex optimization problems arise frequently in many different fields. A comprehensive introduction to the subject, this book shows in detail how such problems can be solved numerically with great efficiency. The focus is on recognizing convex optimization problems and then finding the most appropriate technique for solving them. The text contains many worked examples and homework exercises and will appeal to students, researchers and practitioners in fields such as engineering, computer science, mathematics, statistics, finance, and economics. <s> BIB003 </s> Variational Bayesian Blind Image Deconvolution: A review <s> A. Variational Inference in the image space for Super Gaussian priors <s> We present a general method for blind image deconvolution using Bayesian inference with super-Gaussian sparse image priors. We consider a large family of priors suitable for modeling natural images, and develop the general procedure for estimating the unknown image and the blur. Our formulation includes a number of existing modeling and inference methods as special cases while providing additional flexibility in image modeling and algorithm design. We also present an analysis of the proposed inference compared to other methods and discuss its advantages. Theoretical and experimental results demonstrate that the proposed formulation is very effective, efficient, and flexible. <s> BIB004 </s> Variational Bayesian Blind Image Deconvolution: A review <s> A. Variational Inference in the image space for Super Gaussian priors <s> In this paper we study the problem of blind deconvolution. Our analysis is based on the algorithm of Chan and Wong [2] which popularized the use of sparse gradient priors via total variation. We use this algorithm because many methods in the literature are essentially adaptations of this framework. Such algorithm is an iterative alternating energy minimization where at each step either the sharp image or the blur function are reconstructed. Recent work of Levin et al. [14] showed that any algorithm that tries to minimize that same energy would fail, as the desired solution has a higher energy than the no-blur solution, where the sharp image is the blurry input and the blur is a Dirac delta. However, experimentally one can observe that Chan and Wong's algorithm converges to the desired solution even when initialized with the no-blur one. We provide both analysis and experiments to resolve this paradoxical conundrum. We find that both claims are right. The key to understanding how this is possible lies in the details of Chan and Wong's implementation and in how seemingly harmless choices result in dramatic effects. Our analysis reveals that the delayed scaling (normalization) in the iterative step of the blur kernel is fundamental to the convergence of the algorithm. This then results in a procedure that eludes the no-blur solution, despite it being a global minimum of the original energy. We introduce an adaptation of this algorithm and show that, in spite of its extreme simplicity, it is very robust and achieves a performance comparable to the state of the art. <s> BIB005
Since SG distributions are flexible enough to represent most of the image models used in the BID literature, we restrict, without loss of generality, the VB description to this representation. Furthermore we will formulate the inference in the image space; a detailed account of the use of SMG for the filter representation can be found in BIB004 . As it has already been explained above, the posterior distribution cannot be calculated analytically using the Bayes' rule in Eq. BIB002 . To approximate p(z, Ω|y), VB minimizes the following Kullback-Leibler divergence KL(q(Θ) p(Θ|y)) = q(Θ) log q(Θ) p(Θ|y) dΘ = q(Θ) p(Θ, y) dΘ + const. The Kullback-Leibler divergence is always non negative and is zero if and only if q(Θ) = p(Θ|y). Since the minimizer q(Θ) = p(Θ|y) cannot be calculated, some assumptions on q(Θ) have to be made. One possible assumption is that q(Θ) has a specific parametric form, e.g., a Gaussian distribution. Another widely used assumption is that q(Θ) factorizes into disjoint groups, i.e., This factorized form of variational inference is called mean field theory in physics BIB001 . Using Eq. (13), the KL divergence can be minimized with respect to each factor while holding the other factors fixed. The optimal solution for each factor is then  log q(θ) = E [ln p(Θ, y)] q(Θ) + const, whereΘ = Θ \ θ is the set of unknowns excluding θ and E [ln p(Θ, y)] q(Θ) denotes the expectation taken with respect to all the approximating factorsΘ. This system of equations is solved by an alternating minimization procedure, where each distribution q(θ) is iteratively updated using the latest distributions of all the other factors. Since the KL divergence (12) is convex with respect to q(θ) BIB003 , the convergence of this alternating minimization procedure is guaranteed. The penalty function ρ(·) defined in (5) can be represented as (see  ) where ρ * (ξ/2) is the concave conjugate function Furthermore, the infimum in (15) is achieved at ξ = ρ (s)/s, as shown in BIB004 . Directly applying VB inference using p(x, h, y) is unfeasible, since the expectation of the logarithm of the joint distribution with respect to q(x) is intractable. Since ρ(s) is the penalty associated to a SG distribution we can write where Z is a constant. This Gaussian like lower bound will allow the expectation of the joint distribution to be calculated analytically. We have where ξ = {ξ γ (i), γ = 1, . . . , L, i = 1, . . . , N } with all component positive. We then have where q(ξ) is a degenerate distribution on ξ. We then minimize the above integral on q(ξ), q(x), and q(h) assuming that q(h) is degenerate. According to BIB005 , we obtainĥ q(x) ∝ M (y, x,ĥ,ξ),