Numerous recent approaches attempt to remove image blur due to camera shake, either with one or multiple input images, by explicitly solving an inverse and inherently ill-posed deconvolution problem. If the photographer takes a burst of images, a modality available in virtually all modern digital cameras, we show that it is possible to combine them to get a clean sharp version. This is done without explicitly solving any blur estimation and subsequent inverse problem. The proposed algorithm is strikingly simple: it performs a weighted average in the Fourier domain, with weights depending on the Fourier spectrum magnitude. The method can be seen as a generalization of the align and average procedure, with a weighted average, motivated by hand-shake physiology and theoretically supported, taking place in the Fourier domain. The method’s rationale is that camera shake has a random nature, and therefore, each image in the burst is generally blurred differently. Experiments with real camera data, and extensive comparisons, show that the proposed Fourier burst accumulation algorithm achieves state-of-the-art results an order of magnitude faster, with simplicity for on-board implementation on camera phones. Finally, we also present experiments in real high dynamic range (HDR) scenes, showing how the method can be straightforwardly extended to HDR photography.
A popular technique in astronomical photography, known as or , is to take a series of thousands of short-exposure images and then select and fuse only the sharper ones @cite_35 . Fried @cite_40 showed that the probability of getting a sharp lucky short-exposure image through turbulence follows a negative exponential. Thus, when the captured series or video is sufficiently long, there will exist such a frame with high probability.
@cite_35 @cite_40
2044439645 2011035601 2050334711 2579111433
In short-exposure imaging through turbulence, there is some probability that the image will be nearly diffraction limited because the instantaneous wave-front distortion over the aperture was negligible. A number of years ago in a rather brief paper, Hufnagel (1966) argued heuristically that the probability of getting a good image would decrease exponentially with aperture area. This paper undertakes a rigorous quantitative analysis of the probability. We find that the probability of obtaining a good short-exposure image is Prob ≈ 5.6 exp[−0.1557 (D r0)2] (for D r0 ≥ 3.5), where D is the aperture diameter and r0 is the coherence length of the distorted wave front, as defined by Fried (1967). A good image is taken to be one for which the squared wave-front distortion over the aperture is 1 rad2 or less. The analysis is based on the decomposition of the distorted wave front over the aperture, in an orthonormal series with randomly independent coefficients. The orthonormal functions used are the eigenfunctions of a Karhunen-Loeve integral equation. The integral equation is solved using a separation of variables into radial and azimuthal dependence. The azimuthal dependence was solved analytically and the radial, numerically. The first 569 radial eigenfunctions and eigenvalues were obtained. The probability of obtaining a good short-exposure image corresponds to a hyperspace integral in which the spatial dimensions are the independent random coefficients in the orthonormal series expansion. It is equal to the probability that a randomly chosen point in the hyperspace will lie within a hypersphere of unit radius, the points in the hyperspace being randomly chosen in accordance with the product of independent Gaussian probability distribution—one distribution for each dimension. The variance of these distrbutions is directly proportional to the eigenvalues of the Karhunen-Loeve equation. This hyperspace integral (involving up to several hundred dimensions) has been evaluated using Monte Carlo techniques. We propose a new algorithm dramatically enhancing the efficiency of the lucky imaging technique for AO-corrected images in the visible range. It is achieved by a selection based on the relative strength of signal for each spatial frequency in the Fourier domain, making a more efficient use of information contained in each frame. Realistic simulations show that our algorithm allows us to reach the diffraction limit in the visible range on an AO-equipped 8 m telescope and enhances the Strehl ratio of an AO long exposure by a factor of up to 4. It outperforms the lucky imaging technique at an equivalent selection ratio. The fraction of selected data in simulation is also boosted from two to eight times for a given Strehl-ratio performance. An automated video enhancement technique capable of image fusion from a stream of randomly-distorted images of a still scene is presented in this paper. The technique is based on the "lucky-region" fusion (LRF) approach and aims to improve locally the image quality according to the following steps: (1) for each image of the video stream an image quality map (IQM) which characterizes locally the image quality is computed, (2) each IQM is compared to that of the current fused image leading to the selection of best quality regions (the "lucky-regions"), and (3) the selected regions are merged into the fused video stream. While the LRF approach succeeds in producing images with significantly improved image quality compared to the source images, its performance depends on the imaging conditions and requires adjustment of its fusion parameter - the fusion kernel size - in order to adapt to an evolving environment (e.g. a turbulent atmosphere). Parameter selection was so far performed manually using a trial-and-error approach which causes the technique to be impractical for a real world implementation. The automated LRF technique presented is relaxed from this requirement and selects automatically the fusion parameter based on the analysis of the source images making it more suitable for practical systems. The improved LRF technique is applied to imaging through atmospheric turbulence for various imaging conditions and scenes of interest. In each case automatically-fused video streams demonstrate increases in image quality comparable to that obtained with manual selection of the fusion parameter. We propose a deep learning approach to remove motion blur from a single image captured in the wild, i.e., in an uncontrolled setting. Thus, we consider motion blur degradations that are due to both camera and object motion, and by occlusion and coming into view of objects. In this scenario, a model-based approach would require a very large set of parameters, whose fitting is a challenge on its own. Hence, we take a data-driven approach and design both a novel convolutional neural network architecture and a dataset for blurry images with ground truth. The network produces directly the sharp image as output and is built into three pyramid stages, which allow to remove blur gradually from a small amount, at the lowest scale, to the full amount, at the scale of the input image. To obtain corresponding blurry and sharp image pairs, we use videos from a high frame-rate video camera. For each small video clip we select the central frame as the sharp image and use the frame average as the corresponding blurred image. Finally, to ensure that the averaging process is a sufficient approximation to real blurry images we estimate optical flow and select frames with pixel displacements smaller than a pixel. We demonstrate state of the art performance on datasets with both synthetic and real images.