Super-resolution: a comprehensive survey <s> Biomodality priori (BP) <s> In this paper we consider the problem of reconstructing a high resolution image from a set of undersampled and degraded frames, all of them obtained from high resolution images with unknown shifting displacements between them. We derive an iterative method to estimate the unknown shifts and the high resolution image given the low resolution observations. Finally, the proposed method is tested on real images. <s> BIB001 </s> Super-resolution: a comprehensive survey <s> Biomodality priori (BP) <s> To increase the range of sizes of video scene text recognizable by optical character recognition (OCR), we developed a Bayesian super-resolution algorithm that uses a text-specific bimodal prior. We evaluated the effectiveness of the bimodal prior, compared with and in conjunction with a piecewise smoothness prior, visually and by measuring the accuracy of the OCR results on the variously super-resolved images. The bimodal prior improved the readability of 4- to 7-pixel-high scene text significantly better than bicubic interpolation, and increased the accuracy of OCR results better than the piecewise smoothness prior. <s> BIB002 </s> Super-resolution: a comprehensive survey <s> Biomodality priori (BP) <s> We propose a new solution to the problem of obtaining a single high-resolution image from multiple blurred, noisy, and undersampled images. Our estimator, derived using the Bayesian stochastic framework, is novel in that it employs a new hierarchical non-stationary image prior. This prior adapts the restoration of the super-resolved image to the local spatial statistics of the image. Numerical experiments demonstrate the effectiveness of the proposed approach. <s> BIB003 </s> Super-resolution: a comprehensive survey <s> Biomodality priori (BP) <s> This article proposes a new framework to regularize linear inverse problems using the total variation on non-local graphs. This non-local graph allows to adapt the penalization to the geometry of the underlying function to recover. A fast algorithm computes iteratively both the solution of the regularization process and the non-local graph adapted to this solution. We show numerical applications of this method to the resolution of image processing inverse problems such as inpainting, super-resolution and compressive sampling. <s> BIB004 </s> Super-resolution: a comprehensive survey <s> Biomodality priori (BP) <s> Effective image prior is necessary for image super resolution, due to its severely under-determined nature. Although the edge smoothness prior can be effective, it is generally difficult to have analytical forms to evaluate the edge smoothness, especially for soft edges that exhibit gradual intensity transitions. This paper finds the connection between the soft edge smoothness and a soft cut metric on an image grid by generalizing the Geocuts method (Y. Boykov and V. Kolmogorov, 2003), and proves that the soft edge smoothness measure approximates the average length of all level lines in an intensity image. This new finding not only leads to an analytical characterization of the soft edge smoothness prior, but also gives an intuitive geometric explanation. Regularizing the super resolution problem by this new form of prior can simultaneously minimize the length of all level lines, and thus resulting in visually appealing results. In addition, this paper presents a novel combination of this soft edge smoothness prior and the alpha matting technique for color image super resolution, by normalizing edge segments with their alpha channel description, to achieve a unified treatment of edges with different contrast and scale. <s> BIB005 </s> Super-resolution: a comprehensive survey <s> Biomodality priori (BP) <s> In many situations, interdependency between motion estimation and other estimation tasks is observable. This is for instance true in the area of super-resolution (SR). In order to successfully reconstruct a SR image, accurate motion vector fields are needed. On the other hand, one can only get accurate (subpixel) motion vectors, if there exist highly accurate higher-resolution reference images. Neglecting this interdependency may lead to poor estimation results for motion estimation as well as for the SR image. To address this problem, a new motion estimation scheme is presented that jointly optimises the motion vector field and a SR reference image. For this purpose and in order to attenuate aliasing and noise, which deteriorate the motion estimation, an observation model for the image acquisition process is applied and a maximum a posteriori (MAP) optimisation is performed, using Markov random field image models for regularisation. Results show that the new motion estimator provides more accurate motion vector fields than classical block motion estimation techniques. The joint optimisation scheme yields an estimate of the motion vector field as well as a SR image. <s> BIB006 </s> Super-resolution: a comprehensive survey <s> Biomodality priori (BP) <s> Super-resolution image reconstruction is a powerful methodology for resolution enhancement from a set of blurred and noisy low-resolution images. Following a Bayesian framework, we propose a procedure for super-resolution image reconstruction based on Markov random fields (MRF), where a Potts-Strauss model is assumed for the a priori probability density function of the actual image. The first step is given by aligning all the low-resolution observations over a high-resolution grid and then improving the resolution through the iterated conditional modes (ICM) algorithm. The method was analyzed considering a number of simulated low-resolution and globally translated observations and the results demonstrate the effectiveness of the algorithm in reconstructing the desirable high-resolution image. <s> BIB007 </s> Super-resolution: a comprehensive survey <s> Biomodality priori (BP) <s> The edge and corner structures are two categories of perceptually important image characteristics, and hence, edge-and-corner preserving regularization is required for many problems in image processing. In this paper, the first novelty is to propose a new edge-and-corner preserving approach for image interpolation, based on the coupling of robust orientation diffusion, edge shock filtering, and a type of newly designed corner shock filtering. The proposed interpolation scheme is not only able to remove the staircase artifacts along the edge structures, but also able to restrain the rounding artifacts around the corner structures. The second novelty in this paper is to analyze the filtering behavior of two standard structure tensor based variational PDE (partial differential equation) approaches, following which an edge-and-corner preserving common PDE framework is proposed for different applications in image processing. Numerous experimental results confirm the effectiveness of the proposed interpolation approach, and demonstrate its superiority to other interpolation algorithms. The common PDE framework is applied to several other image processing problems, including image denoising, deringing, deblocking, inpainting, and super-resolution reconstruction. <s> BIB008 </s> Super-resolution: a comprehensive survey <s> Biomodality priori (BP) <s> In this paper, we propose a new super resolution method Maximum a Posteriori based on a universal hidden Markov tree model (MAP-uHMT) for remote sensing images. The hidden Markov tree theory in the wavelet domain is used to set up a prior model for reconstructing super resolution images from a sequence of warped, blurred, sub-sampled and contaminated low resolution images. Both the simulation results with a Landsat7 panchromatic image and actual results with four Landsat7 panchromatic images which were captured on different dates show that our method achieves better super resolution images both visually and quantitatively than other methods, based on PSNR in the simulation and derived PSF with actual data. <s> BIB009 </s> Super-resolution: a comprehensive survey <s> Biomodality priori (BP) <s> Designing effective image priors is of great interest to image super-resolution (SR), which is a severely under-determined problem. An edge smoothness prior is favored since it is able to suppress the jagged edge artifact effectively. However, for soft image edges with gradual intensity transitions, it is generally difficult to obtain analytical forms for evaluating their smoothness. This paper characterizes soft edge smoothness based on a novel SoftCuts metric by generalizing the Geocuts method . The proposed soft edge smoothness measure can approximate the average length of all level lines in an intensity image. Thus, the total length of all level lines can be minimized effectively by integrating this new form of prior. In addition, this paper presents a novel combination of this soft edge smoothness prior and the alpha matting technique for color image SR, by adaptively normalizing image edges according to their alpha-channel description. This leads to the adaptive SoftCuts algorithm, which represents a unified treatment of edges with different contrasts and scales. Experimental results are presented which demonstrate the effectiveness of the proposed method. <s> BIB010 </s> Super-resolution: a comprehensive survey <s> Biomodality priori (BP) <s> This paper proposes a framework for single-image super-resolution. The underlying idea is to learn a map from input low-resolution images to target high-resolution images based on example pairs of input and output images. Kernel ridge regression (KRR) is adopted for this purpose. To reduce the time complexity of training and testing for KRR, a sparse solution is found by combining the ideas of kernel matching pursuit and gradient descent. As a regularized solution, KRR leads to a better generalization than simply storing the examples as has been done in existing example-based algorithms and results in much less noisy images. However, this may introduce blurring and ringing artifacts around major edges as sharp changes are penalized severely. A prior model of a generic image class which takes into account the discontinuity property of images is adopted to resolve this problem. Comparison with existing algorithms shows the effectiveness of the proposed method. <s> BIB011 </s> Super-resolution: a comprehensive survey <s> Biomodality priori (BP) <s> We present a new method of magnification for textured images featuring scale invariance properties. This work is originally motivated by an application to astronomical images. One goal is to propose a method to quantitatively predict statistical and visual properties of images taken by a forthcoming higher resolution telescope from older images at lower resolution. This is done by performing a virtual super resolution using a family of scale invariant stochastic processes, namely compound Poisson cascades, and fractional integration. The procedure preserves the visual aspect as well as the statistical properties of the initial image. An augmentation of information is performed by locally adding random small scale details below the initial pixel size. This extrapolation procedure yields a potentially infinite number of magnified versions of an image. It allows for large magnification factors (virtually infinite) and is physically conservative: zooming out to the initial resolution yields the initial image back. The (virtually) super resolved images can be used to predict the quality of future observations as well as to develop and test compression or denoising techniques. <s> BIB012 </s> Super-resolution: a comprehensive survey <s> Biomodality priori (BP) <s> Super-resolution (SR) is the term used to define the process of estimating a high-resolution (HR) image or a set of HR images from a set of low-resolution (LR) observations. In this paper we propose a class of SR algorithms based on the maximum a posteriori (MAP) framework. These algorithms utilize a new multichannel image prior model, along with the state-of-the-art single channel image prior and observation models. A hierarchical (two-level) Gaussian nonstationary version of the multichannel prior is also defined and utilized within the same framework. Numerical experiments comparing the proposed algorithms among themselves and with other algorithms in the literature, demonstrate the advantages of the adopted multichannel approach. <s> BIB013 </s> Super-resolution: a comprehensive survey <s> Biomodality priori (BP) <s> As a powerful statistical image modeling technique, sparse representation has been successfully used in various image restoration applications. The success of sparse representation owes to the development of the l1-norm optimization techniques and the fact that natural images are intrinsically sparse in some domains. The image restoration quality largely depends on whether the employed sparse domain can represent well the underlying image. Considering that the contents can vary significantly across different images or different patches in a single image, we propose to learn various sets of bases from a precollected dataset of example image patches, and then, for a given patch to be processed, one set of bases are adaptively selected to characterize the local sparse domain. We further introduce two adaptive regularization terms into the sparse representation framework. First, a set of autoregressive (AR) models are learned from the dataset of example image patches. The best fitted AR models to a given patch are adaptively selected to regularize the image local structures. Second, the image nonlocal self-similarity is introduced as another regularization term. In addition, the sparsity regularization parameter is adaptively estimated for better image restoration performance. Extensive experiments on image deblurring and super-resolution validate that by using adaptive sparse domain selection and adaptive regularization, the proposed method achieves much better results than many state-of-the-art algorithms in terms of both PSNR and visual perception. <s> BIB014 </s> Super-resolution: a comprehensive survey <s> Biomodality priori (BP) <s> We propose total subset variation (TSV), a convexity preserving generalization of the total variation (TV) prior, for higher order clique MRF. A proposed differentiable approximation of the TSV prior makes it amenable for use in large images (e.g. 1080p). A convex relaxation of sub-exponential distribution is proposed as a criterion to determine the parameters of the optimization problem resulting from the TSV prior. For the super-resolution application, experiments show reconstruction error improvement with respect to the TV and other methods. <s> BIB015 </s> Super-resolution: a comprehensive survey <s> Biomodality priori (BP) <s> We introduce a class of inverse problem estimators computed by mixing adaptively a family of linear estimators corresponding to different priors. Sparse mixing weights are calculated over blocks of coefficients in a frame providing a sparse signal representation. They minimize an l1 norm taking into account the signal regularity in each block. Adaptive directional image interpolations are computed over a wavelet frame with an O(N logN) algorithm, providing state-of-the-art numerical results. <s> BIB016 </s> Super-resolution: a comprehensive survey <s> Biomodality priori (BP) <s> In this paper, we propose a novel single image super-resolution method based on MAP statistical reconstruction. Our approach takes the wavelet domain Implicit Markov Random Field (IMRF) model as the prior constraint. Specifically, we apply the IMRF model to show the probability distribution of natural images in wavelet domain, and utilize the MAP theory of Bayesian estimation to construct the objective function with this model. Furthermore, we employ steepest descent method to optimize this objective function. In the experiments, we use both Peak Signal to Noise Ratio (PSNR) and visual effect to evaluate our method. The experimental results demonstrate that our method obtains the superior performance in comparison with traditional single image super-resolution approaches. <s> BIB017 </s> Super-resolution: a comprehensive survey <s> Biomodality priori (BP) <s> We propose a new single image super resolution (SR) algorithm via Bayesian modeling with a natural image prior modeled by a high-order Markov random field (MRF). SR is one of the long-standing and active topics in image processing community. It is of great use in many practical applications, such as astronomical observation, medical imaging, and the adaptation of low-resolution contents onto high-resolution displays. One category of the conventional approaches for image SR is formulating the problem with Bayesian modeling techniques and then obtaining its maximum-a-posteriori solution, which actually boils down to a regularized regression task. Although straightforward, this approach cannot exploit the full potential offered by the probabilistic modeling, as only the posterior mode is sought. On the other hand, current Bayesian SR approaches using the posterior mean estimation typically use very simple prior models for natural images to ensure the computational tractability. In this paper, we present a Bayesian image SR approach with a flexible high-order MRF model as the prior for natural images. The minimum mean square error (MMSE) criteria are used for estimating the HR image. A Markov chain Monte Carlo-based sampling algorithm is presented for obtaining the MMSE solution. The proposed method cannot only enjoy the benefits offered by the flexible prior, but also has the advantage of making use of the probabilistic modeling to perform a posterior mean estimation, thus is less sensitive to the local minima problem as the MAP solution. Experimental results indicate that the proposed method can generate competitive or better results than state-of-the-art SR algorithms. <s> BIB018 </s> Super-resolution: a comprehensive survey <s> Biomodality priori (BP) <s> Multiscale morphological operators are studied extensively in the literature for image processing and feature extraction purposes. In this paper, we model a nonlinear regularization method based on multiscale morphology for edge-preserving super resolution (SR) image reconstruction. We formulate SR image reconstruction as a deblurring problem and then solve the inverse problem using Bregman iterations. The proposed algorithm can suppress inherent noise generated during low-resolution image formation as well as during SR image estimation efficiently. Experimental results show the effectiveness of the proposed regularization and reconstruction method for SR image. <s> BIB019
which is modeled as an exponentiated fourth-order polynomial. The maxima of this polynomial are located at centers of distributions of the foreground and background pixels. It can be expressed as BIB002 : where C is a normalization constant, μ 0 and μ 1 are the centers of the peaks of the polynomial which in BIB002 are estimated by expectation maximization (EM), and is the width of the foreground and background distributions. It is shown in BIB002 that this a priori works quite well for two class problems like text improvement in which the text is considered as the foreground and the rest of the image as the background. Other regularization terms Other regularization terms used in the multiple image SR literature are: -Natural image prior   BIB011 BIB012 BIB018 . -Stationary simultaneous auto regression (SAR) BIB001 which applies uniform smoothness to all the locations in the image. -Non-stationary SAR BIB003 in which the variance of the SAR prediction can be different from one location in the image to another. -Soft edge smoothness a priori, which estimates the average length of all level lines in an intensity image BIB005 BIB010 . -Double-exponential Markov random field, which is simply the absolute value of each pixel value BIB006 . -Potts-Strauss MRF BIB007 . -Non-local graph-based regularization BIB004 . -Corner and edge preservation regularization term BIB008 . -Multichannel smoothness a priori which considers the smoothness between frames (temporal residual) and within frames (spatial residual) of a video sequence BIB013 . -Non-local self-similarity BIB014 . -Total subset variation, which is a convex generalization of the TV regularization term BIB015 . -Mumford-Shah regularization term  . -Morphological-based regularization BIB019 . -Wavelet based BIB009 BIB016 BIB017 . Another form of regularization term is used with hallucination or learning-based SR algorithms, which are discussed in the next section.