Super-resolution: a comprehensive survey <s>  <s> We propose a super-resolution technique specifically aimed at enhancing low-resolution text images from handheld devices. The Teager filter, a quadratic unsharp masking filter, is used to highlight high frequencies which are then combined with the warped and interpolated image sequence following motion estimation using Taylor series decomposition. Comparative performance evaluation is presented in the form of OCR results of the super-resolution <s> BIB001 </s> Super-resolution: a comprehensive survey <s>  <s> Many interesting and globally ordered patterns of behavior, such as solidification, arise in statistical physics and are generally referred to as collective phenomena. The obvious analogies to parallel computation can be extended quite far, so that simple computations may be endowed with the most desirable properties of collective phenomena: robustness against circuit defects, extreme parallelism, asynchronous operation and efficient implementation in silicon. To obtain these advantages for more complicated and useful computations, the relatively simple pattern recognition task of fingerprint identification has been selected. Simulations show that an intuitively understandable neural network can generate fingerprint-like patterns within a framework which should allow control of wire length and scale invariance. The purpose of generating such patterns is to create a network whose stable states are noiseless fingerprint patterns, so that noisy fingerprint patterns used as input to the network will evoke the corresponding noiseless patterns as output. There is a developing theory for predicting the behavior of such networks and thereby reducing the amount of simulation that must be done to design them. <s> BIB002 </s> Super-resolution: a comprehensive survey <s>  <s> A signal processing method is developed for solving the problem of fitting multiple lines in a two-dimensional image. It formulates the multi-line-fitting problem in a special parameter estimation framework such that a signal structure similar to the sensor array processing signal representation is obtained. Then the recently developed algorithms in that formalism can be exploited to produce super-resolution estimates for line parameters. The number of lines may also be estimated in this framework. The signal representation used can be generalized to handle problems of line fitting and of straight edge detection. Details of the proposed algorithm and several experimental results are presented. The method exhibits considerable computational speed superiority over existing single- and multiple-line-fitting algorithms such as the Hough transform method. Potential applications include road tracking in robotic vision, mask wafer alignment in semiconductor manufacturing, aerial image analysis, text alignment in document analysis, and particle tracking in bubble chambers. <s> BIB003 </s> Super-resolution: a comprehensive survey <s>  <s> Faces often appear very small in surveillance imagery because of the wide fields of view that are typically used and the relatively large distance between the cameras and the scene. For tasks such as face recognition, resolution enhancement techniques are therefore generally needed. Although numerous resolution enhancement algorithms have been proposed in the literature, most of them are limited by the fact that they make weak, if any, assumptions about the scene. We propose an algorithm to learn a prior on the spatial distribution of the image gradient for frontal images of faces. We proceed to show how such a prior can be incorporated into a resolution enhancement algorithm to yield 4- to 8-fold improvements in resolution (i.e., 16 to 64 times as many pixels). The additional pixels are, in effect, hallucinated. <s> BIB004 </s> Super-resolution: a comprehensive survey <s>  <s> The objective of this work is the super-resolution enhancement of image sequences. We consider in particular images of scenes for which the point-to-point image transformation is a plane projective transformation. We first describe the imaging model, and a maximum likelihood (ML) estimator of the super-resolution image. We demonstrate the extreme noise sensitivity of the unconstrained ML estimator. We show that the Irani and Peleg (1991, 1993) super-resolution algorithm does not suffer from this sensitivity, and explain that this stability is due to the error back-projection method which effectively constrains the solution. We then propose two estimators suitable for the enhancement of text images: a maximum a posteriori (MAP) estimator based on a Huber prior and an estimator regularized using the total variation norm. We demonstrate the improved noise robustness of these approaches over the Irani and Peleg estimator. We also show the effects of a poorly estimated point spread function (PSF) on the super-resolution result and explain conditions necessary for this parameter to be included in the optimization. Results are evaluated on both real and synthetic sequences of text images. In the case of the real images, the projective transformations relating the images are estimated automatically from the image data, so that the entire algorithm is automatic. <s> BIB005 </s> Super-resolution: a comprehensive survey <s>  <s> This paper introduces a new algorithm for enhancing image resolution from an image sequence. The approach we propose herein uses the integrating resampler for warping. The method is a direct computation, which is fundamentally different from the iterative back-projection approaches proposed in previous work. This paper shows that image-warping techniques may have a strong impact on the quality of image resolution enhancement. By coupling the degradation model of the imaging system directly into the integrating resampler, we can better approximate the warping characteristics of real sensors, which also significantly improve the quality of super-resolution images. Examples of super-resolutions are given for gray-scale images. Evaluations are made visually by comparing the resulting images and those using bi-linear resampling and back-projection and quantitatively using OCR as a fundamental measure. The paper shows that even when the images are qualitatively similar, quantitative differences appear in machine processing. <s> BIB006 </s> Super-resolution: a comprehensive survey <s>  <s> Super-resolution is usually posed as a reconstruction problem. The low resolution input images are assumed to be noisy, downsampled versions of an unknown super-resolution image that is to be estimated. A common way of inverting the down-sampling process is to write down the reconstruction constraints and then solve them, often adding a smoothness prior to regularize the solution. In this paper, we present two results which both show that there is more to super-resolution than image reconstruction. We first analyze the reconstruction constraints and show that they provide less and less useful information as the magnification factor increases. Afterwards, we describe a “hallucination” algorithm, incorporating the recognition of local features in the low resolution images, which outperforms existing reconstruction-based algorithms. <s> BIB007 </s> Super-resolution: a comprehensive survey <s>  <s> Preface. Contributing Authors. 1. Introduction S. Chaudhuri. 2. Image Zooming: Use of Wavelets N. Kaulgud, U.D. Desai. 3. Generalized Interpolation for Super-Resolution D. Rajan, S. Chaudhuri. 4. High Resolution Image from Low Resolution Images B.C. Tom, et al. 5. Super-Resolution Imaging Using Blur as a Cue D. Rajan, S. Chaudhuri. 6. Super-Resolution via Image Warping T.E. Boult, et al. 7. Resolution Enhancement using Multiple Apertures T. Komatsu, et al. 8. Super-Resolution from Mutual Motion A. Zomet, S. Peleg. 9. Super-Resolution from Compressed Video C.A. Segall, et al. 10. Super-Resolution: Limits and Beyond S. Baker, T. Kanade. Index. <s> BIB008 </s> Super-resolution: a comprehensive survey <s>  <s> Super-resolution aims to produce a high-resolution image from a set of one or more low-resolution images by recovering or inventing plausible high-frequency image content. Typical approaches try to reconstruct a high-resolution image using the sub-pixel displacements of several low-resolution images, usually regularized by a generic smoothness prior over the high-resolution image space. Other methods use training data to learn low-to-high-resolution matches, and have been highly successful even in the single-input-image case. Here we present a domain-specific image prior in the form of a p.d.f. based upon sampled images, and show that for certain types of super-resolution problems, this sample-based prior gives a significant improvement over other common multiple-image super-resolution techniques. <s> BIB009 </s> Super-resolution: a comprehensive survey <s>  <s> The attainment of super resolution (SR) from a sequence of degraded undersampled images could be viewed as reconstruction of the high-resolution (HR) image from a finite set of its projections on a sampling lattice. This can then be formulated as an optimization problem whose solution is obtained by minimizing a cost function. The approaches adopted and their analysis to solve the formulated optimization problem are crucial, The image acquisition scheme is important in the modeling of the degradation process. The need for model accuracy is undeniable in the attainment of SR along with the design of the algorithm whose robust implementation will produce the desired quality in the presence of model parameter uncertainty. To keep the presentation focused and of reasonable size, data acquisition with multisensors instead of, say a video camera is considered. <s> BIB010 </s> Super-resolution: a comprehensive survey <s>  <s> It is very important for the performance evaluation of iris recognition algorithms to construct very large iris databases. However, limited by the real conditions, there are no very large common iris databases now. In this paper, an iris image synthesis method based on principal component analysis (PCA) and super-resolution is proposed. The iris recognition algorithm based on PCA is first introduced and then, iris image synthesis method is presented. The synthesis method first constructs coarse iris images with the given coefficients. Then, synthesized iris images are enhanced using super-resolution. Through controlling the coefficients, we can create many iris images with specified classes. Extensive experiments show that the synthesized iris images have satisfactory cluster and the synthesized iris databases can be very large. <s> BIB011 </s> Super-resolution: a comprehensive survey <s>  <s> Super-resolution has been applied mainly to grayscale images, but producing a high-resolution color image using a single CCD sensor has not been investigated in detail until recently. This work aims at producing a high-resolution color image directly from raw "color mosaic" images obtained by a single CCD equipped with a color filter array. This method is based on a generalized formulation of super-resolution that simultaneously performs both resolution enhancement and demosaicing. The other key factor of our topic is a precise sub-pixel registration of multiple raw images. We examined direct registration of raw images based on an imaging model, which shows precise estimation of motion among severely aliased raw images. The proposed method is verified through experiments using synthetic and real images. <s> BIB012 </s> Super-resolution: a comprehensive survey <s>  <s> 1 Introduction.- 1.1 Background.- 1.2 Modelling assumptions.- 1.3 Applications.- 1.4 Principal contributions.- 2 Literature Survey.- 2.1 Image registration.- 2.1.1 Registration by a geometric transformation.- 2.1.2 Ensuring global consistency.- 2.1.3 Other parametric surfaces.- 2.2 Image mosaicing.- 2.3 Super-resolution.- 2.3.1 Simple super-resolution schemes.- 2.3.2 Methods using a generative model.- 2.3.3 Super-resolution using statistical prior image models.- 3 Registration: Geometric and Photometric.- 3.1 Introduction.- 3.2 Imaging geometry.- 3.3 Estimating homographies.- 3.3.1 Linear estimators.- 3.3.2 Non-linear refinement.- 3.3.3 The maximum likelihood estimator of H.- 3.4 A practical two-view method.- 3.5 Assessing the accuracy of registration.- 3.5.1 Assessment criteria.- 3.5.2 Obtaining a ground-truth homography.- 3.6 Feature-based vs. direct methods.- 3.7 Photometric registration.- 3.7.1 Sources of photometric difference.- 3.7.2 The photometric model.- 3.7.3 Estimating the parameters.- 3.7.4 Results.- 3.8 Application: Recovering latent marks in forensic images.- 3.8.1 Motivation.- 3.8.2 Method.- 3.8.3 Further examples.- 3.9 Summary.- 4 Image Mosaicing.- 4.1 Introduction.- 4.2 Basic method.- 4.2.1 Outline.- 4.2.2 Practical considerations.- 4.3 Rendering from the mosaic.- 4.3.1 The reprojection manifold.- 4.3.2 The blending function.- 4.3.3 Eliminating seams by photometric registration.- 4.3.4 Eliminating seams due to vignetting.- 4.3.5 A fast alternative to median filtering.- 4.4 Simultaneous registration of multiple views.- 4.4.1 Motivation.- 4.4.2 Extending the two-view framework to N-views.- 4.4.3 A novel algorithm for feature-matching over N-views.- 4.4.4 Results.- 4.5 Automating the choice of reprojection frame.- 4.5.1 Motivation.- 4.5.2 Synthetic camera rotations.- 4.6 Applications of image mosaicing.- 4.7 Mosaicing non-planar surfaces.- 4.8 Mosaicing "user's guide".- 4.9 Summary.- 4.9.1 Further examples.- 5 Super-resolution: Maximum Likelihood and Related Approaches.- 5.1 Introduction.- 5.2 What do we mean by "resolution"?.- 5.3 Single-image methods.- 5.4 The multi-view imaging model.- 5.4.1 A note on the assumptions made in the model.- 5.4.2 Discretization of the imaging model.- 5.4.3 Related approaches.- 5.4.4 Computing the elements in Mn.- 5.4.5 Boundary conditions.- 5.5 Justification for the Gaussian PSF.- 5.6 Synthetic test images.- 5.7 The average image.- 5.7.1 Noise robustness.- 5.8 Rudin's forward-projection method.- 5.9 The maximum-likelihood estimator.- 5.10 Predicting the behaviour of the ML estimator.- 5.11 Sensitivity of the ML estimator to noise sources.- 5.11.1 Observation noise.- 5.11.2 Poorly estimated PSF.- 5.11.3 Inaccurate registration parameters.- 5.12 Irani and Peleg's method.- 5.12.1 Least-squares minimization by steepest descent.- 5.12.2 Irani and Peleg's algorithm.- 5.12.3 Relationship to the ML estimator.- 5.12.4 Convergence properties.- 5.13 Gallery of results.- 5.14 Summary.- 6 Super-resolution Using Bayesian Priors.- 6.1 Introduction.- 6.2 The Bayesian framework.- 6.2.1 Markov random fields.- 6.2.2 Gibbs priors.- 6.2.3 Some common cases.- 6.3 The optimal Wiener filter as a MAP estimator.- 6.4 Generic image priors.- 6.5 Practical optimization.- 6.6 Sensitivity of the MAP estimators to noise sources.- 6.6.1 Exercising the prior models.- 6.6.2 Robustness to image noise.- 6.7 Hyper-parameter estimation by cross-validation.- 6.8 Gallery of results.- 6.9 Super-resolution "user's guide".- 6.10 Summary.- 7 Super-resolution Using Sub-space Models.- 7.1 Introduction.- 7.2 Bound constraints.- 7.3 Learning a face model using PCA.- 7.4 Super-resolution using the PCA model.- 7.4.1 An ML estimator (FS-ML).- 7.4.2 MAP estimators.- 7.5 The behaviour of the face model estimators.- 7.6 Examples using real images.- 7.7 Summary.- 8 Conclusions and Extensions.- 8.1 Summary.- 8.2 Extensions.- 8.2.1 Application to digital video.- 8.2.2 Model-based super-resolution.- 8.3 Final observations.- A Large-scale Linear and Non-linear Optimization.- References. <s> BIB013 </s> Super-resolution: a comprehensive survey <s>  <s> Super-Resolution reconstruction produces one or a set of high-resolution images from a sequence of low-resolution frames. This article reviews a variety of Super-Resolution methods proposed in the last 20 years, and provides some insight into, and a summary of, our recent contributions to the general Super-Resolution problem. In the process, a detailed study of several very important aspects of Super-Resolution, often ignored in the literature, is presented. Spe- cifically, we discuss robustness, treatment of color, and dynamic operation modes. Novel methods for addressing these issues are accompanied by experimental results on simulated and real data. Finally, some future challenges in Super-Resolution are outlined and discussed. © 2004 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 14, 47-57, 2004; Published online in Wiley InterScience (www.interscience.wiley. com). DOI 10.1002/ima.20007 <s> BIB014 </s> Super-resolution: a comprehensive survey <s>  <s> Images magnified by standard methods display a degradation of detail that is particularly noticeable in the blurry edges of text. Current super-resolution algorithms address the lack of sharpness by filling in the image with probable details. These algorithms break the outlines of text. Our novel algorithm for super-resolution of text magnifies images in real-time by interpolation with a variable linear filter. The coefficients of the filter are determined nonlinearly from the neighborhood to which it is applied. We train the mapping that defines the coefficients to specifically enhance edges of text, producing a conservative algorithm that infers the detail of magnified text. Possible applications include resizing web page layouts or other interfaces, and enhancing low resolution camera captures of text. In general, learning spatially-variable filters is applicable to other image filtering tasks. <s> BIB015 </s> Super-resolution: a comprehensive survey <s>  <s> To increase the range of sizes of video scene text recognizable by optical character recognition (OCR), we developed a Bayesian super-resolution algorithm that uses a text-specific bimodal prior. We evaluated the effectiveness of the bimodal prior, compared with and in conjunction with a piecewise smoothness prior, visually and by measuring the accuracy of the OCR results on the variously super-resolved images. The bimodal prior improved the readability of 4- to 7-pixel-high scene text significantly better than bicubic interpolation, and increased the accuracy of OCR results better than the piecewise smoothness prior. <s> BIB016 </s> Super-resolution: a comprehensive survey <s>  <s> This paper presents a prior model for text image super-resolution in the Bayesian framework. In contrast to generic image super-resolution task, super-resolution of text images can be benefited from strong prior knowledge of the image class: firstly, low-resolution images are assumed to be generated from a high-resolution image by a sort of degradation which can be grasped through example pairs of the original and the corresponding degradation; secondly, text images are composed of two homogeneous regions, text and background regions. These properties were represented in a Markov random field (MRF) framework. Experiments showed that our model is more appropriate to text image super-resolution than the other prior models. <s> BIB017 </s> Super-resolution: a comprehensive survey <s>  <s> The super-resolution problem is considered using a mean-squared error minimizing solution of a generalized undersampling model in this non-iterative frequency domain approach. While previous frequency domain approaches have been based on a bandlimited image model, this approach uses a non-bandlimited stationary spectral model. This allows improved reconstruction of certain image features. The model and algorithm are presented along with an example <s> BIB018 </s> Super-resolution: a comprehensive survey <s>  <s> In this paper, we consider the super-resolution image reconstruction problem. We propose a Markov chain Monte Carlo (MCMC) approach to find the maximum a posterior probability (MAP) estimation of the unknown high-resolution image. Firstly, Gaussian Markov random field (GMRF) is exploited for modeling the prior probability distribution of the unknown high-resolution image. Then, a MCMC technique (in particular, the Gibbs sampler) is introduced to generate samples from the posterior probability distribution to compute the MAP estimation of the unknown high-resolution image, which is obtained as the mean of the samples. Moreover, we derive a bound on the convergence time of the proposed MCMC approach. Finally, the experimental results are presented to verify the superior performance of the proposed approach and the validity of the proposed bound. <s> BIB019 </s> Super-resolution: a comprehensive survey <s>  <s> Super-resolution imaging is to overcome the inherent limitations of image acquisition to create high-resolution images from their low-resolution counterparts. In this paper, a novel state-space approach is proposed to incorporate the temporal correlations among the low-resolution observations into the framework of the Kalman filtering. The proposed approach exploits both the temporal correlations information among the high-resolution images and the temporal correlations information among the low-resolution images to improve the quality of the reconstructed high-resolution sequence. Experimental results show that the proposed framework is superior to bi-linear interpolation, bi-cubic spline interpolation and the conventional Kalman filter approach, due to the consideration of the temporal correlations among the low-resolution images. <s> BIB020 </s> Super-resolution: a comprehensive survey <s>  <s> Super-resolution image reconstruction produces a high-resolution image from a set of shifted, blurred, and decimated versions thereof. Previously published papers have not addressed the computational complexity of this ill-conditioned large scale problem adequately. In this paper, the computational complexity of MAP-based multiframe super-resolution algorithms is studied, and a new fast algorithm, as well as methods for parallel image reconstruction is also presented. The proposed fast algorithm splits the multiple input low-resolution images into several subsets according to their translation relations, and then applies normal MAP algorithm to each subset, the reconstructed images are processed subsequently at a successive level until the desired resolution is achieved. Experiment results are also provided to demonstrate the efficiency of the proposed techniques. <s> BIB021 </s> Super-resolution: a comprehensive survey <s>  <s> Super-resolution is the problem of generating one or a set of high-resolution images from one or a sequence of low-resolution frames. Most methods have been proposed for super-resolution based on multiple low resolution images of the same scene, which is called multiple-frame super-resolution. Only a few approaches produce a high-resolution image from a single low-resolution image, with the help of one or a set of training images from scenes of the same or different types. It is referred to as single-frame super-resolution. This article reviews a variety of single-frame super-resolution methods proposed in the recent years. In the paper, a new manifold learning method: locally linear embedding (LLE) and its relation with single-frame super-resolution is introduced. Detailed study of a critical issue: "neighborhood issue" is presented with related experimental results and analysis and possible future research is given. <s> BIB022 </s> Super-resolution: a comprehensive survey <s>  <s> Abstract : Super-resolution algorithms produce a single high-resolution image from a set of several, low-resolution images of the desired scene. The low-resolution frames are shifted differently with respect to the high resolution frame with subpixel increments. This paper presents first a theoretical overview of super-resolution algorithms. The most important methods, namely, the iterative back-projection, projection onto convex sets, and maximum a posteriori estimation are then compared within the same framework of implementation. <s> BIB023 </s> Super-resolution: a comprehensive survey <s>  <s> In this paper, we propose a learning-based method for video super-resolution. There are two main contributions of the proposed method. First, information from cameras with different spatial-temporal resolutions is combined in our framework. This is achieved by constructing training dictionary using the high resolution images captured by still camera and the low resolution video is enhanced via searching in this customized database. Second, we enforce the spatio-temporal constraints using the conditional random field (CRF) and the problem of video super-resolution is posed as finding the high resolution video that maximizes the conditional probability. We apply the algorithm to video sequences taken from different scenes using cameras with different qualities and promising results are presented. <s> BIB024 </s> Super-resolution: a comprehensive survey <s>  <s> Regularization plays a vital role in inverse problems, and especially in ill-posed ones. Along with classical regularization techniques based on smoothness, entropy, and sparsity, an emerging powerful regularization is one that leans on image examples. In this paper, we propose an efficient scheme for using image examples as driving a powerful regularization, applied to the image scale-up (super-resolution) problem. In this work, we target specifically scanned documents containing written text, graphics, and equations. Our algorithm starts by assigning per each location in the degraded image several candidate high-quality patches. Those are found as the nearest-neighbors (NN) in an image-database that contains pairs of corresponding low- and high-quality image patches. The found examples are used for the definition of an image prior expression, merged into a global MAP penalty function. We use this penalty function both for rejecting some of the irrelevant outlier examples, and then for reconstructing the desired image. We demonstrate our algorithm on several scanned documents with promising results. <s> BIB025 </s> Super-resolution: a comprehensive survey <s>  <s> We propose a learning-based, single-image super-resolution reconstruction technique using the contourlet transform, which is capable of capturing the smoothness along contoursmaking use of directional decompositions. The contourlet coefficients at finer scales of the unknown high-resolution image are learned locally from a set of high-resolution training images, the inverse contourlet transform of which recovers the super-resolved image. In effect, we learn the high-resolution representation of an oriented edge primitive from the training data. Our experiments show that the proposed approach outperforms standard interpolation techniques as well as a standard (Cartesian) wavelet-based learning both visually and in terms of the PSNR values, especially for images with arbitrarily oriented edges. <s> BIB026 </s> Super-resolution: a comprehensive survey <s>  <s> A typical image formation model for super-resolution (SR) introduces blurring, aliasing, and added noise. The enhancement itself may also introduce ringing. In this paper, we use subjective tests to assess the visual quality of SR-enhanced images. We then examine how well some existing objective quality metrics can characterize the observed subjective quality. Even full-reference metrics like MSE and SSIM do not always capture visual quality of SR images with and without residual aliasing. <s> BIB027 </s> Super-resolution: a comprehensive survey <s>  <s> This thesis concerns the use of spatial and tonal adaptivity in improving the resolution of aliased image sequences under scene or camera motion. Each of the five content chapters focuses on a different subtopic of super-resolution: image registration (chapter 2), image fusion (chapter 3 and 4), super-resolution restoration (chapter 5), and super-resolution synthesis (chapter 6). Chapter 2 derives the Cramer-Rao lower bound of image registration and shows that iterative gradient-based estimators achieve this performance limit. Chapter 3 presents an algorithm for image fusion of irregularly sampled and uncertain data using robust normalized convolution. The size and shape of the fusion kernel is adapted to local curvilinear structures in the image. Each data sample is assigned an intensity-related certainty value to limit the influence of outliers. Chapter 4 presents two fast implementations of the signal-adaptive bilateral filter. The xy-separable implementation filters the image along sampling axes, while the uv-separable implementation filters the image along gauge coordinates. Chapter 5 presents a robust image restoration method using Gaussian error norm rather than quadratic error norm. The robust solution resembles the maximum-likelihood solution under Gaussian noise, and it is not susceptible to outliers. A series of objective quality measures confirms the superiority of this solution to current super-resolution algorithms in the literature. Chapter 6 proposes a super-resolution synthesis algorithm in the DCT domain. The algorithm requires a high-resolution image of similar content to be used as texture source for the low-resolution input image. <s> BIB028 </s> Super-resolution: a comprehensive survey <s>  <s> Aliasing in images is often considered as a nuisance. Artificial low frequency patterns and jagged edges appear when an image is sampled at a too low frequency. However, aliasing also conveys useful information about the high frequency content of the image, which is exploited in super-resolution applications. We use a set of input images of the same scene to extract such high frequency information and create a higher resolution aliasing-free image. Typically, there is a small shift or more complex motion between the different images, such that they contain slightly different information about the scene. Super-resolution image reconstruction can be formulated as a multichannel sampling problem with unknown offsets. This results in a set of equations that are linear in the unknown signal coefficients but nonlinear in the offsets. This thesis concentrates on the computation of these offsets, as they are an essential prerequisite for an accurate high resolution reconstruction. If a part of the image spectra is free of aliasing, the planar shift and rotation parameters can be computed using only this low frequency information. In such a case, the images can be registered pairwise to a reference image. Such a method is not applicable if the images are undersampled by a factor of two or larger. A higher number of images needs to be registered jointly. Two subspace methods are discussed for such highly aliased images. The first approach is based on a Fourier description of the aliased signals as a sum of overlapping parts of the spectrum. It uses a rank condition to find the correct offsets. The second one uses a more general expansion in an arbitrary Hilbert space to compute the signal offsets. The sampled signal is represented as a linear combination of sampled basis functions. The offsets are computed by projecting the signal onto varying subspaces. Under certain conditions, in particular for bandlimited signals, the nonlinear super-resolution equations can be written as a set of polynomial equations. Using Buchberger's algorithm, the solution can then be computed as a Grobner basis for the corresponding polynomial ideal. After a description of a standard algorithm, adaptations are made for the use with noisy measurements. The techniques presented in this thesis are tested in simulations and practical experiments. The experiments are performed on sets of real images taken with a digital camera. The results show the validity of the algorithms: registration parameters are computed with subpixel precision, and aliasing is accurately removed from the resulting high resolution image. This thesis is produced according to the concepts of reproducible research. All the results and examples used in this thesis are reproducible using the code and data available online. <s> BIB029 </s> Super-resolution: a comprehensive survey <s>  <s> Abstract The shortcomings in commonly used kernel-based super-resolution drive the study of improved super-resolution algorithms of higher quality. In the past years a wide range of very different approaches has been taken to improve super-resolution. This paper compares approaches to high quality super-resolution by looking at theoretical backgrounds and practical results. Strengths and weaknesses are listed with the intent to spot chances for combination or improvement of techniques, thereby forming a base for future improved super-resolution algorithms. <s> BIB030 </s> Super-resolution: a comprehensive survey <s>  <s> In this paper, we propose a maximum a posteriori ramework for the super-resolution problem, i.e., reconstructing high-resolution images from shifted, rotated, low-resolution degraded observations. The main contributions of this work are two; first, the use of a new locally adaptive edge preserving prior for the super-resolution problem. Second an efficient two-step reconstruction methodology that includes first an initial registration using only the low-resolution degraded observations. This is followed by a fast iterative algorithm implemented in the discrete Fourier transform domain in which the restoration, interpolation and the registration subtasks of this problem are preformed simultaneously. We present examples with both synthetic and real data that demonstrate the advantages of the proposed framework. <s> BIB031 </s> Super-resolution: a comprehensive survey <s>  <s> This paper examines a simultaneous estimation of both high resolution (high accuracy) 3D information and high-resolution image from multiple low-resolution stereo-pair images. The proposed method is based on the MAP (maximum a posteriori) framework and these two relevant problems (estimation of both high-resolution 3 D and intensity images) are incorporated into a unified cost function. Then the solution is obtained iteratively by a fixed point algorithm, which yeilds 3 D information while the high-resolution image is fixed, and vice versa. Each solution can be obtained by solving a linear equation. It can be expected that the proposed method improves accuracy of both 3 D information and high-resolution image simultaneously. Some experimental results are shown to validate the proposed method. <s> BIB032 </s> Super-resolution: a comprehensive survey <s>  <s> We present a novel method of Bayesian image super-resolution in which marginalization is carried out over latent parameters such as geometric and photometric registration and the image point-spread function. Related Bayesian super-resolution approaches marginalize over the high-resolution image, necessitating the use of an unfavourable image prior, whereas our method allows for more realistic image prior distributions, and reduces the dimension of the integral considerably, removing the main computational bottleneck of algorithms such as Tipping and Bishop's Bayesian image super-resolution. We show results on real and synthetic datasets to illustrate the efficacy of our method. <s> BIB033 </s> Super-resolution: a comprehensive survey <s>  <s> Reconstruction-based super-resolution algorithms use either sub-pixel shifts or relative blur among low-resolution observations as a cue to obtain a high-resolution image. In this paper, we propose a super-resolution algorithm that exploits the information available in the low-resolution observations due to both sub-pixel shifts and relative blur to yield a better quality image. Performance analysis is carried out based on the Cramer-Rao lower bound. Several experimental results on synthetic and real images are given for validation. <s> BIB034 </s> Super-resolution: a comprehensive survey <s>  <s> This paper addresses the problem of super-resolving a single image and recovering the characteristics of the sensor using a learning-based approach. In particular, the point spread function (PSF) of the camera is sought by minimizing the mean Euclidean distance function between patches from the input frame and from degraded versions of high-resolution training images. Once an estimate of the PSF is obtained, a supervised learning algorithm can then be used as is. Results are compared with another method for blind super-resolution by using a series of quality measures. <s> BIB035 </s> Super-resolution: a comprehensive survey <s>  <s> In this paper, we make contact with the field of nonparametric statistics and present a development and generalization of tools and results for use in image processing and reconstruction. In particular, we adapt and expand kernel regression ideas for use in image denoising, upscaling, interpolation, fusion, and more. Furthermore, we establish key relationships with some popular existing methods and show how several of these algorithms, including the recently popularized bilateral filter, are special cases of the proposed framework. The resulting algorithms and analyses are amply illustrated with practical examples. <s> BIB036 </s> Super-resolution: a comprehensive survey <s>  <s> A robust whole-image super-resolution is highly demanded. Whole-image super-resolution requires target-region extraction and motion estimation because an image sequence usually includes multiple targets and a background. A single- motion model is insufficient to represent whole-image motion. In addition, high-accuracy motion estimation is necessary for super-resolution. Simultaneous target-region extraction and high-accuracy motion estimation are challenging problems. We propose a robust and accurate algorithm that can provide extracted single-motion regions and their motion parameters. The proposed algorithm consists of three phases: feature-based motion estimation, single-motion region extraction, and region-based motion estimation. Once we obtain motion parameters of the whole image, we can reconstruct a high-resolution whole image by applying a robust super- resolution. Results of experiments demonstrate that the proposed approach can robustly reconstruct complex multiple- target scenes. <s> BIB037 </s> Super-resolution: a comprehensive survey <s>  <s> Super-resolution reconstruction proposes a fusion of several low-quality images into one higher quality result with better optical resolution. Classic super-resolution techniques strongly rely on the availability of accurate motion estimation for this fusion task. When the motion is estimated inaccurately, as often happens for nonglobal motion fields, annoying artifacts appear in the super-resolved outcome. Encouraged by recent developments on the video denoising problem, where state-of-the-art algorithms are formed with no explicit motion estimation, we seek a super-resolution algorithm of similar nature that will allow processing sequences with general motion patterns. In this paper, we base our solution on the Nonlocal-Means (NLM) algorithm. We show how this denoising method is generalized to become a relatively simple super-resolution algorithm with no explicit motion estimation. Results on several test movies show that the proposed method is very successful in providing super-resolution on general sequences. <s> BIB038 </s> Super-resolution: a comprehensive survey <s>  <s> During this decade, the enlargement in the extensive use of digital imaging technologies in consumer (e.g., digital video) and other markets (e.g. security and military) has brought with it a simultaneous demand for higher-resolution (HR) images. The demand for such images can be partially met by algorithmic advances in SuperResolution Reconstruction (SRR) technology in addition to hardware development. Not only do such HR images give the viewer a more pleasing picture but also offer additional details that are significant for subsequent analysis in many applications. SRR algorithm is considered to be one of the most promising techniques that can help overcome the limitations due to optics and sensor resolution. In general, the problem of super-resolution can be expressed as that of combining a set of aliased, noisy, lowresolution, blurry images to produce a higher resolution image or image sequence. The idea is to increase the information content in the final image by exploiting the additional spatio-temporal information that is available in each of the LR images. Consequently, the major advantage of the SRR algorithm is that it may cost less and the existing LR imaging systems can be still utilized. The SRR algorithm is proved to be useful in several practical cases where multiple frames of the same scene can be obtained, including medical imaging, satellite imaging, and video applications. <s> BIB039 </s> Super-resolution: a comprehensive survey <s>  <s> In many applications, such as observation systems, it is important to detect and identify a possible threat as early as possible. Especially the approaching ones are of interest. Such moving threats/objects are often at a very large distance and are observed as very small and unrecognizable objects at the image plane of a camera. In this research multi-frame super-resolution (SR) reconstruction methods are developed that are capable of improving the resolution, signal-to-noise ratio and contrast of moving objects in under-sampled image sequences. These improvements will help to increase the detection and recognition rate of moving objects of various sizes. The most difficult case is improving the small moving objects which consist of solely “mixed” boundary pixels on a cluttered background. We developed a SR reconstruction method that performs well on both simulated and real-world data and it is shown that, for reconstructing small moving objects, our method outperforms state-of-the-art existing SR reconstruction methods. <s> BIB040 </s> Super-resolution: a comprehensive survey <s>  <s> Image Super-Resolution: Historical Overview and Future Challenges, J. Yang and T. Huang Introduction to Super-Resolution Notations Techniques for Super-Resolution Challenge issues for Super-Resolution Super-Resolution Using Adaptive Wiener Filters, R.C. Hardie Introduction Observation Model AWF SR Algorithms Experimental Results Conclusions Acknowledgments Locally Adaptive Kernel Regression for Space-Time Super-Resolution, H. Takeda and P. Milanfar Introduction Adaptive Kernel Regression Examples Conclusion AppendiX Super-Resolution With Probabilistic Motion Estimation, M. Protter and M. Elad Introduction Classic Super-Resolution: Background The Proposed Algorithm Experimental Validation Summary Spatially Adaptive Filtering as Regularization in Inverse Imaging, A. Danielyan, A. Foi, V. Katkovnik, and K. Egiazarian Introduction Iterative filtering as regularization Compressed sensing Super-resolution Conclusions Registration for Super-Resolution, P. Vandewalle, L. Sbaiz, and M. Vetterli Camera Model What Is Resolution? Super-Resolution as a Multichannel Sampling Problem Registration of Totally Aliased Signals Registration of Partially Aliased Signals Conclusions Towards Super-Resolution in the Presence of Spatially Varying Blur, M. Sorel, F. Sroubek and J. Flusser Introduction Defocus and Optical Aberrations Camera Motion Blur Scene Motion Algorithms Conclusion Acknowledgments Toward Robust Reconstruction-Based Super-Resolution, M. Tanaka and M. Okutomi Introduction Overviews Robust SR Reconstruction with Pixel Selection Robust Super-Resolution Using MPEG Motion Vectors Robust Registration for Super-Resolution Conclusions Multi-Frame Super-Resolution from a Bayesian Perspective, L. Pickup, S. Roberts, A. Zisserman and D. Capel The Generative Model Where Super-Resolution Algorithms Go Wrong Simultaneous Super-Resolution Bayesian Marginalization Concluding Remarks Variational Bayesian Super Resolution Reconstruction, S. Derin Babacan, R. Molina, and A.K. Katsaggelos Introduction Problem Formulation Bayesian Framework for Super Resolution Bayesian Inference Variational Bayesian Inference Using TV Image Priors Experiments Estimation of Motion and Blur Conclusions Acknowledgements Pattern Recognition Techniques for Image Super-Resolution, K. Ni and T.Q. Nguyen Introduction Nearest Neighbor Super-Resolution Markov Random Fields and Approximations Kernel Machines for Image Super-Resolution Multiple Learners and Multiple Regressions Design Considerations and Examples Remarks Glossary Super-Resolution Reconstruction of Multi-Channel Images, O.G. Sezer and Y. Altunbasak Introduction Notation Image Acquisition Model Subspace Representation Reconstruction Algorithm Experiments & Discussions Conclusion New Applications of Super-Resolution in Medical Imaging, M.D.Robinson, S.J. Chiu, C.A. Toth, J.A. Izatt, J.Y. Lo, and S. Farsiu Introduction The Super-Resolution Framework New Medical Imaging Applications Conclusion Acknowledgment Practicing Super-Resolution: What Have We Learned? N. Bozinovic Abstract Introduction MotionDSP: History and Concepts Markets and Applications Technology Results Lessons Learned Conclusions <s> BIB041 </s> Super-resolution: a comprehensive survey <s>  <s> Ultrasound image resolution enhancement is an ongoing challenge to date. Though many works have been performed using device-based approach, there exists few works dealing with post-processing methods. This paper investigates a technique based on the Alternating Direction Method of Multipliers for the resolution enhancement in ultrasound imaging, which includes the deblurring and denoising tasks. We here point out the characteristics of the proposed technique and thereby underline the features that must be dealt with for successful ultrasound image restoration. Synthetic and in vivo ultrasound images are processed in order to asses its performances. <s> BIB042 </s> Super-resolution: a comprehensive survey <s>  <s> The low resolution of images has been one of the major limitations in recognising humans from a distance using their biometric traits, such as face and iris. Superresolution has been employed to improve the resolution and the recognition performance simultaneously, however the majority of techniques employed operate in the pixel domain, such that the biometric feature vectors are extracted from a super-resolved input image. Feature-domain superresolution has been proposed for face and iris, and is shown to further improve recognition performance by capitalising on direct super-resolving the features which are used for recognition. However, current feature-domain superresolution approaches are limited to simple linear features such as Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA), which are not the most discriminant features for biometrics. Gabor-based features have been shown to be one of the most discriminant features for biometrics including face and iris. This paper proposes a framework to conduct super-resolution in the non-linear Gabor feature domain to further improve the recognition performance of biometric systems. Experiments have confirmed the validity of the proposed approach, demonstrating superior performance to existing linear approaches for both face and iris biometrics. <s> BIB043 </s> Super-resolution: a comprehensive survey <s>  <s> This paper discusses resolution enhancement of a set of images with varying exposure durations, having a high combined dynamic range. So far, little has been said in relation to the Human Visual System when it comes to Super-Resolution and High Dynamic Range fusion, unlike the case for traditional Super-Resolution where errors are measured with respect to human perception in the pixel domain. We propose a Super-Resolution method in the L*a*b* domain to bridge that gap and present some image reconstruction results. <s> BIB044
resolution (LR) observations . It has been used for many different applications (Table 1) , such as, satellite and aerial imaging, medical image processing, ultrasound imaging BIB042 , line-fitting BIB003 , automated mosaicking, infrared imaging, facial image improvement, text images improvement, compressed images and video enhancement, sign and number plate reading, iris recognition BIB011 BIB043 , fingerprint image enhancement, digital holography  , and highdynamic range imaging BIB044 . SR is an algorithm that aims to provide details finer than the sampling grid of a given imaging device by increasing the number of pixels per unit area in an image BIB038 . Before getting into the details of SR algorithms, we need to know about the possible hardware-based approaches to the problem of increasing the number of pixels per unit area. Such approaches include (1) decreasing the pixel size and (2) increasing the sensor size  BIB039 . The former solution is a useful solution, but decreasing the pixel size beyond a specific threshold (which has already been reached by the current technologies) decreases the amount of light which reaches the associated cell of the pixel on the sensor. This results in an increase in the shot noise. Furthermore, pixels of smaller sizes (relative to the aperture's size) are more sensitive to diffraction effects compared to pixels of larger sizes. The latter solution increases the capacitance of the system, which slows down the charge transfer rate. Furthermore, the mentioned hardware-based approaches are usually expensive for large-scale imaging devices. Therefore, algorithmicbased approaches (i.e., SR algorithms) are usually preferred to the hardware-based solutions. SR should not be confused with similar techniques, such as interpolation, restoration, or image rendering. In interpolation (applied usually to a single image), the high-frequency details are not restored, unlike SR BIB012 . In image restoration, obtained by deblurring, sharpening, and similar techniques, BIB004 BIB005 BIB006 BIB007 BIB009  BIB015 BIB016 BIB001 BIB017 BIB018 BIB019 BIB020 BIB021 BIB024  BIB031 BIB025   BIB032 (Chinese text),  BIB033  BIB034   BIB037  388 BIB026  BIB027 BIB035 the sizes of the input and the output images are the same, but the quality of the output gets improved. In SR, besides improving the quality of the output, its size (the number of pixels per unit area) is also increased BIB022 . In image rendering, addressed by computer graphics, a model of an HR scene together with imaging parameters is given. These are then used to predict the HR observation of the camera, while in SR it is the other way around. Over the past two decades, many research papers, books BIB008  BIB041 and PhD theses BIB002  BIB013   BIB036 BIB028 BIB029   BIB040  have been written on SR algorithms. Several survey papers [47, 48, BIB010  BIB014 BIB023 BIB030   BIB039 have also been published on the topic. Some of these surveys provide good overviews of SR algorithms, but only for a limited number of methods. For example, [47, 48] provide the details of most frequency domain methods and some of the probability-based methods BIB010  417 ] take a closer look at the reconstruction-based SR methods and some of the learning-based methods BIB014 BIB023 BIB030 have provided a comparative analysis of reconstruction-based SR algorithms but only for a very few methods; and finally  provides details of some of the single-image based SR algorithms. None of these surveys provide a comprehensive overview of all the different solutions of the SR problem. Furthermore, none of them include the latest advances in the field, especially for the learning-based methods and regularized-based methods. In addition to providing a comprehensive overview of most of the published SR works (until 2012) , this survey covers most of the weaknesses of the previously published surveys. The present paper describes the basics of almost all the different types of super-resolution algorithms that have been published up to 2012. Then, for each of these basic methods, the evolving paths of the methods have been discussed by providing the modifications that have been applied to the basics by different researchers. Comparative discussions are also provided when available in the surveyed papers. The first parts (the basics of the methods) can be studied by beginners in the field so as to have a better understanding of the available methods, while the last parts (the evolving paths of the methods and the comparative results) can be used by experts in the field to find out about the current status of their desired methods. The rest of this paper is organized as follows: the next section provides a taxonomy covering all the different types of SR algorithms. Section 3 reviews the imaging models that have been used in most SR algorithms. Section 4 explains the frequency domain SR methods. Section 5 describes the spatial domain SR algorithms. Some other issues related to Fig. 1 The proposed taxonomy for the surveyed SR algorithms and their dedicated sections in this paper SR algorithms, like handling color, the assessment of SR algorithms, improvement factors, common databases, and 3D SR, are discussed in Sect. 6. Finally, the paper comes to a conclusion in Sect. 7.