We propose a novel multi-stream architecture and training methodology that exploits semantic labels for facial image deblurring. The proposed Uncertainty Guided Multi-Stream Semantic Network (UMSN) processes regions belonging to each semantic class independently and learns to combine their outputs into the final deblurred result. Pixel-wise semantic labels are obtained using a segmentation network. A predicted confidence measure is used during training to guide the network towards challenging regions of the human face such as the eyes and nose. The entire network is trained in an end-to-end fashion. Comprehensive experiments on three different face datasets demonstrate that the proposed method achieves significant improvements over the recent state-of-the-art face deblurring methods. Code is available at: this https URL
Classical image deblurring methods estimate the blur kernel given a blurry image and then apply deconvolution to get the deblurred image. To calculate the blur kernel some techniques assume prior information about the image, and formulate maximum a-posertior (MAP) to obtain the deblurred image. Different priors such sparsity, @math gradient prior, patch prior, manifold Prior, and low-rank have been proposed to obtain regularized reconstructions @cite_45 @cite_11 @cite_35 @cite_42 @cite_27 @cite_16 @cite_9 @cite_1 @cite_31 . @cite_3 estimate the Fourier coefficients of the blur kernel to deblur the image. @cite_20 learn the latent features of the blurry images using the latent features of the clean images to estimate the deblurred image. These CNN-based methods do not perform well compared to the state-of-the-art MAP-based methods for large motion kernels. Recent non-blind image deblurring methods like @cite_15 @cite_43 @cite_27 @cite_30 @cite_29 assume and use some knowledge about the blur kernel. Given the latent blurry input, @cite_29 estimate multiple latent images corresponding to different prior strengths to estimate the final deblurred image. CNN-based techniques like @cite_32 and @cite_25 estimate the blur kernel and address dynamic deblurring.
@cite_30 @cite_35 @cite_15 @cite_9 @cite_42 @cite_1 @cite_29 @cite_3 @cite_32 @cite_43 @cite_27 @cite_45 @cite_31 @cite_16 @cite_25 @cite_20 @cite_11
2798840374 2032506791 2141115311 2300657047 2110671603 2047123483 1976730913 2776004874 2792274428 2396267364 2154194773 2075861921 1988446025 1792921166 233979554 2025900737 2792166771
Blind motion deblurring methods are primarily responsible for recovering an accurate estimate of the blur kernel. Non-blind deblurring (NBD) methods, on the other hand, attempt to faithfully restore the original image, given the blur estimate. However, NBD is quite susceptible to errors in blur kernel. In this work, we present a convolutional neural network-based approach to handle kernel uncertainty in non-blind motion deblurring. We provide multiple latent image estimates corresponding to different prior strengths obtained from a given blurry observation in order to exploit the complementarity of these inputs for improved learning. To generalize the performance to tackle arbitrary kernel noise, we train our network with a large number of real and synthetic noisy blur kernels. Our network mitigates the effects of kernel noise so as to yield detail-preserving and artifact-free restoration. Our quantitative and qualitative evaluations on benchmark datasets demonstrate that the proposed method delivers state-of-the-art results. To further underscore the benefits that can be achieved from our network, we propose two adaptations of our method to improve kernel estimates, and image deblurring quality, respectively. Blind image deconvolution, i.e., estimating a blur kernel k and a latent image x from an input blurred image y, is a severely ill-posed problem. In this paper we introduce a new patch-based strategy for kernel estimation in blind deconvolution. Our approach estimates a “trusted” subset of x by imposing a patch prior specifically tailored towards modeling the appearance of image edge and corner primitives. To choose proper patch priors we examine both statistical priors learned from a natural image dataset and a simple patch prior from synthetic structures. Based on the patch priors, we iteratively recover the partial latent image x and the blur kernel k. A comprehensive evaluation shows that our approach achieves state-of-the-art results for uniformly blurred images. We present a new algorithm for removing motion blur from a single image. Our method computes a deblurred image using a unified probabilistic model of both blur kernel estimation and unblurred image restoration. We present an analysis of the causes of common artifacts found in current deblurring methods, and then introduce several novel terms within this probabilistic model that are inspired by our analysis. These terms include a model of the spatial randomness of noise in the blurred image, as well a new local smoothness prior that reduces ringing artifacts by constraining contrast in the unblurred image wherever the blurred image exhibits low contrast. Finally, we describe an effficient optimization scheme that alternates between blur kernel estimation and unblurred image restoration until convergence. As a result of these steps, we are able to produce high quality deblurred results in low computation time. We are even able to produce results of comparable quality to techniques that require additional input images beyond a single blurry photograph, and to methods that require additional hardware. We present a new method for blind motion deblurring that uses a neural network trained to compute estimates of sharp image patches from observations that are blurred by an unknown motion kernel. Instead of regressing directly to patch intensities, this network learns to predict the complex Fourier coefficients of a deconvolution filter to be applied to the input patch for restoration. For inference, we apply the network independently to all overlapping patches in the observed image, and average its outputs to form an initial estimate of the sharp image. We then explicitly estimate a single global blur kernel by relating this estimate to the observed image, and finally perform non-blind deconvolution with this kernel. Our method exhibits accuracy and robustness close to state-of-the-art iterative methods, while being much faster when parallelized on GPU hardware. This paper presents a robust algorithm to deblur two consecutively captured blurred photos from camera shaking. Previous dual motion deblurring algorithms succeeded in small and simple motion blur and are very sensitive to noise. We develop a robust feedback algorithm to perform iteratively kernel estimation and image deblurring. In kernel estimation, the stability and capability of the algorithm is greatly improved by incorporating a robust cost function and a set of kernel priors. The robust cost function serves to reject outliers and noise, while kernel priors, including sparseness and continuity, remove ambiguity and maintain kernel shape. In deblurring, we propose a novel and robust approach which takes two blurred images as input to infer the clear image. The deblurred image is then used as feedback to refine kernel estimation. Our method can successfully estimate large and complex motion blurs which cannot be handled by previous dual or single image motion deblurring algorithms. The results are shown to be significantly better than those of previous approaches. We propose a simple yet effective 0-regularized prior based on intensity and gradient for text image deblurring. The proposed image prior is motivated by observing distinct properties of text images. Based on this prior, we develop an efficient optimization method to generate reliable intermediate results for kernel estimation. The proposed method does not require any complex filtering strategies to select salient edges which are critical to the state-of-the-art deblurring algorithms. We discuss the relationship with other deblurring algorithms based on edge selection and provide insight on how to select salient edges in a more principled way. In the final latent image restoration step, we develop a simple method to remove artifacts and render better deblurred images. Experimental results demonstrate that the proposed algorithm performs favorably against the state-of-the-art text image deblurring methods. In addition, we show that the proposed method can be effectively applied to deblur low-illumination images. This paper presents a fast deblurring method that produces a deblurring result from a single image of moderate size in a few seconds. We accelerate both latent image estimation and kernel estimation in an iterative deblurring process by introducing a novel prediction step and working with image derivatives rather than pixel values. In the prediction step, we use simple image processing techniques to predict strong edges from an estimated latent image, which will be solely used for kernel estimation. With this approach, a computationally efficient Gaussian prior becomes sufficient for deconvolution to estimate the latent image, as small deconvolution artifacts can be suppressed in the prediction. For kernel estimation, we formulate the optimization function using image derivatives, and accelerate the numerical process by reducing the number of Fourier transforms needed for a conjugate gradient method. We also show that the formulation results in a smaller condition number of the numerical system than the use of pixel values, which gives faster convergence. Experimental results demonstrate that our method runs an order of magnitude faster than previous work, while the deblurring quality is comparable. GPU implementation facilitates further speed-up, making our method fast enough for practical use. We present an approach for blind image deblurring, which handles non-uniform blurs. Our algorithm has two main components: (i) A new method for recovering the unknown blur-field directly from the blurry image, and (ii) A method for deblurring the image given the recovered non-uniform blur-field. Our blur-field estimation is based on analyzing the spectral content of blurry image patches by Re-blurring them. Being unrestricted by any training data, it can handle a large variety of blur sizes, yielding superior blur-field estimation results compared to training-based deep-learning methods. Our non-uniform deblurring algorithm is based on the internal image-specific patch-recurrence prior. It attempts to recover a sharp image which, on one hand – results in the blurry image under our estimated blur-field, and on the other hand – maximizes the internal recurrence of patches within and across scales of the recovered sharp image. The combination of these two components gives rise to a blind-deblurring algorithm, which exceeds the performance of state-of-the-art CNN-based blind-deblurring by a significant margin, without the need for any training data. We present an effective blind image deblurring method based on a data-driven discriminative prior.Our work is motivated by the fact that a good image prior should favor clear images over blurred images.In this work, we formulate the image prior as a binary classifier which can be achieved by a deep convolutional neural network (CNN).The learned prior is able to distinguish whether an input image is clear or not.Embedded into the maximum a posterior (MAP) framework, it helps blind deblurring in various scenarios, including natural, face, text, and low-illumination images.However, it is difficult to optimize the deblurring method with the learned image prior as it involves a non-linear CNN.Therefore, we develop an efficient numerical approach based on the half-quadratic splitting method and gradient decent algorithm to solve the proposed model.Furthermore, the proposed model can be easily extended to non-uniform deblurring.Both qualitative and quantitative experimental results show that our method performs favorably against state-of-the-art algorithms as well as domain-specific image deblurring approaches. Low-rank matrix approximation has been successfully applied to numerous vision problems in recent years. In this paper, we propose a novel low-rank prior for blind image deblurring. Our key observation is that directly applying a simple low-rank model to a blurry input image significantly reduces the blur even without using any kernel information, while preserving important edge information. The same model can be used to reduce blur in the gradient map of a blurry input. Based on these properties, we introduce an enhanced prior for image deblurring by combining the low rank prior of similar patches from both the blurry image and its gradient map. We employ a weighted nuclear norm minimization method to further enhance the effectiveness of low-rank prior for image deblurring, by retaining the dominant edges and eliminating fine texture and slight edges in intermediate images, allowing for better kernel estimation. In addition, we evaluate the proposed enhanced low-rank prior for both the uniform and the non-uniform deblurring. Quantitative and qualitative experimental evaluations demonstrate that the proposed algorithm performs favorably against the state-of-the-art deblurring methods. This paper presents a new method for deblurring photos using a sharp reference example that contains some shared content with the blurry photo. Most previous deblurring methods that exploit information from other photos require an accurately registered photo of the same static scene. In contrast, our method aims to exploit reference images where the shared content may have undergone substantial photometric and non-rigid geometric transformations, as these are the kind of reference images most likely to be found in personal photo albums. Our approach builds upon a recent method for example-based deblurring using non-rigid dense correspondence (NRDC) [ 2011] and extends it in two ways. First, we suggest exploiting information from the reference image not only for blur kernel estimation, but also as a powerful local prior for the non-blind deconvolution step. Second, we introduce a simple yet robust technique for spatially varying blur estimation, rather than assuming spatially uniform blur. Unlike the above previous method, which has proven successful only with simple deblurring scenarios, we demonstrate that our method succeeds on a variety of real-world examples. We provide quantitative and qualitative evaluation of our method and show that it outperforms the state-of-the-art. Blind deconvolution is to recover a sharp version of a given blurry image or signal when the blur kernel is unknown. Because this problem is ill-conditioned in nature, effectual criteria pertaining to both the sharp image and blur kernel are required to constrain the space of candidate solutions. While the problem has been extensively studied for long, it is still unclear how to regularize the blur kernel in an elegant, effective fashion. In this paper, we show that the blurry image itself actually encodes rich information about the blur kernel, and such information can indeed be found by exploring and utilizing a well-known phenomenon, that is, sharp images are often high pass, whereas blurry images are usually low pass. More precisely, we shall show that the blur kernel can be retrieved through analyzing and comparing how the spectrum of an image as a convolution operator changes before and after blurring. Subsequently, we establish a convex kernel regularizer, which depends only on the given blurry image. Interestingly, the minimizer of this regularizer guarantees to give a good estimate to the desired blur kernel if the original image is sharp enough. By combining this powerful regularizer with the prevalent nonblind devonvolution techniques, we show how we could significantly improve the deblurring results through simulations on synthetic images and experiments on realistic images. We present a deblurring algorithm that uses a hardware attachment coupled with a natural image prior to deblur images from consumer cameras. Our approach uses a combination of inexpensive gyroscopes and accelerometers in an energy optimization framework to estimate a blur function from the camera's acceleration and angular velocity during an exposure. We solve for the camera motion at a high sampling rate during an exposure and infer the latent image using a joint optimization. Our method is completely automatic, handles per-pixel, spatially-varying blur, and out-performs the current leading image-based methods. Our experiments show that it handles large kernels -- up to at least 100 pixels, with a typical size of 30 pixels. We also present a method to perform "ground-truth" measurements of camera motion blur. We use this method to validate our hardware and deconvolution approach. To the best of our knowledge, this is the first work that uses 6 DOF inertial sensors for dense, per-pixel spatially-varying image deblurring and the first work to gather dense ground-truth measurements for camera-shake blur. We describe a new method for recovering the blur kernel in motion-blurred images based on statistical irregularities their power spectrum exhibits. This is achieved by a power-law that refines the one traditionally used for describing natural images. The new model better accounts for biases arising from the presence of large and strong edges in the image. We use this model together with an accurate spectral whitening formula to estimate the power spectrum of the blur. The blur kernel is then recovered using a phase retrieval algorithm with improved convergence and disambiguation capabilities. Unlike many existing methods, the new approach does not perform a maximum a posteriori estimation, which involves repeated reconstructions of the latent image, and hence offers attractive running times. We compare the new method with state-of-the-art methods and report various advantages, both in terms of efficiency and accuracy. Recurrence of small image patches across different scales of a natural image has been previously used for solving ill-posed problems (e.g. super- resolution from a single image). In this paper we show how this multi-scale property can also be used for “blind-deblurring”, namely, removal of an unknown blur from a blurry image. While patches repeat ‘as is’ across scales in a sharp natural image, this cross-scale recurrence significantly diminishes in blurry images. We exploit these deviations from ideal patch recurrence as a cue for recovering the underlying (unknown) blur kernel. More specifically, we look for the blur kernel k, such that if its effect is “undone” (if the blurry image is deconvolved with k), the patch similarity across scales of the image will be maximized. We report extensive experimental evaluations, which indicate that our approach compares favorably to state-of-the-art blind deblurring methods, and in particular, is more robust than them. Most existing nonblind image deblurring methods assume that the blur kernel is free of error. However, it is often unavoidable in practice that the input blur kernel is erroneous to some extent. Sometimes, the error could be severe, e.g., for images degraded by nonuniform motion blurring. When an inaccurate blur kernel is used as the input, significant distortions will appear in the image recovered by existing methods. In this paper, we present a novel convex minimization model that explicitly takes account of error in the blur kernel. The resulting minimization problem can be efficiently solved by the so-called accelerated proximal gradient method. In addition, a new boundary extension scheme is incorporated in the proposed model to further improve the results. The experiments on both synthesized and real images showed the efficiency and robustness of our algorithm to both the image noise and the model error in the blur kernel. Non-blind image deconvolution is an ill-posed problem. The presence of noise and band-limited blur kernels makes the solution of this problem non-unique. Existing deconvolution techniques produce a residual between the sharp image and the estimation that is highly correlated with the sharp image, the kernel, and the noise. In most cases, different restoration models must be constructed for different blur kernels and different levels of noise, resulting in low computational efficiency or highly redundant model parameters. Here we aim to develop a single model that handles different types of kernels and different levels of noise: general non-blind deconvolution. Specifically, we propose a very deep convolutional neural network that predicts the residual between a pre-deconvolved image and the sharp image rather than the sharp image. The residual learning strategy makes it easier to train a single model for different kernels and different levels of noise, encouraging high effectiveness and efficiency. Quantitative evaluations demonstrate the practical applicability of the proposed model for different blur kernels. The model also shows the state-of-the-art performance on synthesized blurry images.