Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> Abstract Given a low resolution camera, we would like to get an image with improved resolution using camera motion. From several low resolution images, with subpixel relative displacements, we get an improved resolution image. We find a high resolution image such that when simulating the imaging process we get low resolution images closest to the observed images. The method is tested on computer simulations, and simulated annealing is also used for the optimization process. <s> BIB001 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> While Bayesian methods can significantly improve the quality of tomographic reconstructions, they require the solution of large iterative optimization problems. Recent results indicate that the convergence of these optimization problems can be improved by using sequential pixel updates, or Gauss-Seidel iterations. However, Gauss-Seidel iterations may be perceived as less useful when parallel computing architectures are use. We show that for degrees of parallelism of typical practical interest, the Gauss-Seidel iterations updates may be computed in parallel with little loss in convergence speed. In this case, the theoretical speed up of parallel implementations is nearly linear with the number of processors. <s> BIB002 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> Super-resolution reconstruction deals with the recovery of a single high-resolution image from a set of warped, blurred and decimated versions of it. Several algorithms were already proposed for the solution of this general problem. We concentrate on a special case where the warps are pure translations, the blur is space invariant and the same for all the images, and the noise is white. We exploit previous results to develop a new highly efficient super-resolution reconstruction algorithm for this case, which separates the treatment to de-blurring and measurements fusion. The fusion part is shown to be a non-iterative and very simple algorithm, preserving the optimality of the entire reconstruction process, in the maximum-likelihood sense. Simulations demonstrate the capabilities of the proposed algorithm. <s> BIB003 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> Superresolution reconstruction produces a high-resolution image from a set of low-resolution images. Previous iterative methods for superresolution had not adequately addressed the computational and numerical issues for this ill-conditioned and typically underdetermined large scale problem. We propose efficient block circulant preconditioners for solving the Tikhonov-regularized superresolution problem by the conjugate gradient method. We also extend to underdetermined systems the derivation of the generalized cross-validation method for automatic calculation of regularization parameters. The effectiveness of our preconditioners and regularization techniques is demonstrated with superresolution results for a simulated sequence and a forward looking infrared (FLIR) camera image sequence. <s> BIB004 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> Normal video sequences contain substantial overlap between successive frames, and each region in the scene appears in multiple frames. The super-resolution process creates high-resolution pictures of regions that are sampled in multiple frames, having a higher spatial resolution than the original video frames. <s> BIB005 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> This paper presents a new technique for generating a high resolution image from a blurred image sequence; this is also referred to as super-resolution restoration of images. The image sequence consists of decimated, blurred and noisy versions of the high resolution image. The high resolution image is modeled as a Markov random field (MRF) and a maximum a posteriori (MAP) estimation technique is used. A simple gradient descent method is used to optimize the functional. Further, line fields are introduced in the cost function and optimization using Graduated Non-Convexity (GNC) is shown to yield improved results. Lastly, we present results of optimization using Simulated Annealing (SA). <s> BIB006 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> In this chapter, we present a parametric method for generating a super-resolution image from a sequence consisting of blurred and noisy observations. The high resolution image is modeled as a Markov random field (MRF) and a maximum a posteriori (MAP) estimation technique is used for super-resolution restoration. Unlike other super-resolution imaging methods, the proposed technique does not require sub-pixel registration of given observations. A simple gradient descent method is used to optimize the cost. The discontinuities in the intensity process can be preserved by introducing suitable line processes. Superiority of this technique to standard methods of image interpolation is illustrated. The motivation for using blur as a cue is also explained. <s> BIB007 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> In this chapter the problem of reconstructing a high resolution image from multiple aliased and shifted by sub-pixel shifts low resolution images is considered. The low resolution images are possibly degraded by unknown blurs and their sub-pixel shifts are not known. This problem is described in the frequency and spatial domains. Algorithms for providing solutions to it are reviewed. In addition, two approaches are presented in detail for solving this low-to-high resolution problem. In the first of these two approaches registration and restoration is performed simultaneously using the expectation-maximization (EM) algorithm. The high resolution image is then reconstructed using regularized interpolation which is performed as a separate step. For this reason this approach is abbreviated as RR-I which corresponds to registration/restorationinterpolation. In the second of these approaches registration, restoration and interpolation are perfomed simultaneously using the EM algorithm. Therefore this approach is abbreviated as RRI which corresponds to registration/restoration/interpolation. Numerical experiments are presented that demonstrate the effectiveness of the two approaches. <s> BIB008 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> In the last two decades, many papers have been published, proposing a variety methods of multiframe resolution enhancement. These methods are usually very sensitive to their assumed model of data and noise, which limits their utility. This paper reviews some of these methods and addresses their shortcomings. We propose a different implementation using L/sub 1/ norm minimization and robust regularization to deal with different data and noise models. This computationally inexpensive method is robust to errors in motion and blur estimation, and results in sharp edges. Simulation results confirm the effectiveness of our method and demonstrate its superiority to other robust super-resolution methods. <s> BIB009 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> Super-resolution aims to produce a high-resolution image from a set of one or more low-resolution images by recovering or inventing plausible high-frequency image content. Typical approaches try to reconstruct a high-resolution image using the sub-pixel displacements of several low-resolution images, usually regularized by a generic smoothness prior over the high-resolution image space. Other methods use training data to learn low-to-high-resolution matches, and have been highly successful even in the single-input-image case. Here we present a domain-specific image prior in the form of a p.d.f. based upon sampled images, and show that for certain types of super-resolution problems, this sample-based prior gives a significant improvement over other common multiple-image super-resolution techniques. <s> BIB010 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> Super-Resolution reconstruction produces one or a set of high-resolution images from a sequence of low-resolution frames. This article reviews a variety of Super-Resolution methods proposed in the last 20 years, and provides some insight into, and a summary of, our recent contributions to the general Super-Resolution problem. In the process, a detailed study of several very important aspects of Super-Resolution, often ignored in the literature, is presented. Spe- cifically, we discuss robustness, treatment of color, and dynamic operation modes. Novel methods for addressing these issues are accompanied by experimental results on simulated and real data. Finally, some future challenges in Super-Resolution are outlined and discussed. © 2004 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 14, 47-57, 2004; Published online in Wiley InterScience (www.interscience.wiley. com). DOI 10.1002/ima.20007 <s> BIB011 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> Super-resolution reconstruction algorithm produces a high-resolution image from a low-resolution image sequence. The accuracy and the stability of the motion estimation (ME) are essential for the whole restoration. In this paper, a new super-resolution reconstruction algorithm is developed using a robust ME method, which fuses multiple estimated motion vectors within the sequence. The new algorithm has two major improvements compared with the previous research. First, instead of only two frames, the whole sequence is used to obtain a more accurate and stable estimation of the motion vector of each frame; second, the reliability of the ME is quantitatively measured and introduced into the cost function of the reconstruction algorithm. The algorithm is applied to both synthetic and real sequences, and the results are presented in the paper. <s> BIB012 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> Super-resolution reconstruction is the process of reconstructing a high-resolution image from multiple low-resolution images. Most super-resolution reconstruction methods neglect camera response function, exposure time, white balancing, and external illumination changes. In this paper, we show how to extend traditional super-resolution reconstruction methods to handle these factors. We provide formulations for several different super-resolution reconstruction approaches. Experimental results are also included. <s> BIB013 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> To increase the range of sizes of video scene text recognizable by optical character recognition (OCR), we developed a Bayesian super-resolution algorithm that uses a text-specific bimodal prior. We evaluated the effectiveness of the bimodal prior, compared with and in conjunction with a piecewise smoothness prior, visually and by measuring the accuracy of the OCR results on the variously super-resolved images. The bimodal prior improved the readability of 4- to 7-pixel-high scene text significantly better than bicubic interpolation, and increased the accuracy of OCR results better than the piecewise smoothness prior. <s> BIB014 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> In this paper, we study the problem of reconstructing a high-resolution image from several blurred low-resolution image frames. The image frames consist of decimated, blurred and noisy versions of the high-resolution image. The high-resolution image is modeled as a Markov random field (MRF), and a maximum a posteriori (MAP) estimation technique is used for the restoration. We show that with the periodic boundary condition, the high-resolution image can be restored efficiently by using fast Fourier transforms. We also apply the preconditioned conjugate gradient method to restore the high-resolution image. Computer simulations are given to illustrate the effectiveness of the proposed method. <s> BIB015 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> This paper presents a prior model for text image super-resolution in the Bayesian framework. In contrast to generic image super-resolution task, super-resolution of text images can be benefited from strong prior knowledge of the image class: firstly, low-resolution images are assumed to be generated from a high-resolution image by a sort of degradation which can be grasped through example pairs of the original and the corresponding degradation; secondly, text images are composed of two homogeneous regions, text and background regions. These properties were represented in a Markov random field (MRF) framework. Experiments showed that our model is more appropriate to text image super-resolution than the other prior models. <s> BIB016 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> An efficient scheme is presented to the color super-resolution problem for recovery of a color high-resolution image with knowledge of multiple Bayer filtered low-resolution images. To recover a visually natural high-resolution image, we restrict fairly smooth initial candidates to all images satisfying all bounds imposed on the several nonsmooth convex color total variations as well as a non-smooth convex inter cross correlation measure among color channels. In the proposed scheme, the data-fidelity is optimized in a systematic way, over all initial candidates, with the hybrid steepest descent method for quasi-nonexpansive mappings [Yamada & Ogura 2004], by minimizing successively an weighted average of pure mean square errors between the low-resolution transforms of the high-resolution estimate and the multiple low-resolution images. Numerical examples show that the proposed scheme recovers visually natural high resolution images by resolving the tradeoff between noise suppression and edge preservation of the recovered image while keeping fair inter channel cross correlation among color channels. <s> BIB017 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> In this paper, we propose a robust approach to super-resolution static sprite generation from multiple low-resolution images. Considering both short-term and long-term motion influences, a hybrid global motion estimation technique is first presented for sprite generation. An iterative super-resolution reconstruction algorithm is then proposed for the super-resolution sprite construction. This algorithm is robust to outliers and results in a sprite image with high visual quality. In addition, the generated super-resolution sprite can provide better reconstructed images than a conventional low-resolution sprite. Experimental results demonstrate the effectiveness of the proposed methods. <s> BIB018 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> Super-resolution image reconstruction produces a high-resolution image from a set of shifted, blurred, and decimated versions thereof. Previously published papers have not addressed the computational complexity of this ill-conditioned large scale problem adequately. In this paper, the computational complexity of MAP-based multiframe super-resolution algorithms is studied, and a new fast algorithm, as well as methods for parallel image reconstruction is also presented. The proposed fast algorithm splits the multiple input low-resolution images into several subsets according to their translation relations, and then applies normal MAP algorithm to each subset, the reconstructed images are processed subsequently at a successive level until the desired resolution is achieved. Experiment results are also provided to demonstrate the efficiency of the proposed techniques. <s> BIB019 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> In this work, we propose a new super-resolution algorithm to simultaneously estimate all frames of a video sequence. The new algorithm is based on the Bayesian maximum a posteriori estimation. In contrast to other multi-frame super-resolution algorithms, the proposed algorithm does not include the motion in the observation model. Instead, transformations caused by the motion are used as a prior information in order to achieve smoothness in the motion trajectory. The proposed algorithm provides lower computational complexity than the traditional super-resolution algorithms when several frames need to be restored. The new algorithm is also robust to motion errors and outliers. We provide results to illustrate the superiority of the new method. <s> BIB020 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> Superresolution is the process of producing a high resolution image from a collection of low resolution images. This process has potential application in a wide spectrum of fields in which navigation, surveillance, and observation are important, yet in which target images have limited resolution. There have been numerous methods proposed and developed to implement superresolution, each with its own advantages and limitations. However, there is no standard method or software for superresolution. In this paper a genetic algorithm solution for determining the registration and point spread function (PSF) parameters for superresolution is proposed and implemented, and a superresolved image is generated using genetic algorithm optimization of an existing superresolution method. <s> BIB021 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> In this paper, we consider the super-resolution image reconstruction problem. We propose a Markov chain Monte Carlo (MCMC) approach to find the maximum a posterior probability (MAP) estimation of the unknown high-resolution image. Firstly, Gaussian Markov random field (GMRF) is exploited for modeling the prior probability distribution of the unknown high-resolution image. Then, a MCMC technique (in particular, the Gibbs sampler) is introduced to generate samples from the posterior probability distribution to compute the MAP estimation of the unknown high-resolution image, which is obtained as the mean of the samples. Moreover, we derive a bound on the convergence time of the proposed MCMC approach. Finally, the experimental results are presented to verify the superior performance of the proposed approach and the validity of the proposed bound. <s> BIB022 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> Super-resolution is the process of combining multiple low-resolution images to produce a higher resolution image. Because the super-resolution problem is an ill-posed one, many regularized algorithms have been proposed. Those algorithms usually use the same regularization term for a whole image. However, since an image generally contains various regions of different characteristics, simple regularization is not good enough for the prospective result. In this paper, we propose a region-based super-resolution algorithm to apply a suitable regularization term for each region. In the algorithm, the image is divided into homogeneous and inhomogeneous regions. According to the type of region, we apply different filters for regularization. The regularization parameters are also adaptively determined during the iteration. Simulation results show that the proposed algorithm is superior to the conventional algorithm in terms of objective quality as well as subjective one <s> BIB023 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> Theoretical and practical limitations usually constrain the achievable resolution of any imaging device. Super-Resolution (SR) methods are developed through the years to go beyond this limit by acquiring and fusing several low-resolution (LR) images of the same scene, producing a high-resolution (HR) image. The early works on SR, ::: although occasionally mathematically optimal for particular models of data and noise, produced poor results when applied to real images. In this paper, we discuss two of the main issues related to designing a practical SR system, namely reconstruction accuracy and computational efficiency. Reconstruction accuracy refers to the problem of designing a robust SR method applicable to images from different imaging systems. We study a general framework for optimal reconstruction of images from grayscale, color, or color filtered (CFA) cameras. The performance of our proposed method is boosted by using powerful priors and is robust to both measurement (e.g. CCD read out noise) and system noise (e.g. motion estimation error). Noting that the motion estimation is often considered a bottleneck in terms of SR performance, we introduce the concept of "constrained motions" for enhancing the quality of super-resolved images. We show that using such constraints will enhance the quality of the motion estimation and therefore results in more accurate reconstruction of the HR images. We also justify some practical assumptions that greatly reduce the computational complexity and memory requirements of the proposed methods. We use efficient approximation of the Kalman Filter (KF) and adopt a dynamic point of view to the SR problem. Novel methods for addressing these issues are accompanied by experimental results on real data. <s> BIB024 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> In the last two decades, two related categories of problems have been studied independently in image restoration literature: super-resolution and demosaicing. A closer look at these problems reveals the relation between them, and, as conventional color digital cameras suffer from both low-spatial resolution and color-filtering, it is reasonable to address them in a unified context. In this paper, we propose a fast and robust hybrid method of super-resolution and demosaicing, based on a maximum a posteron estimation technique by minimizing a multiterm cost function. The L1 norm is used for measuring the difference between the projected estimate of the high-resolution image and each low-resolution image, removing outliers in the data and errors due to possibly inaccurate motion estimation. Bilateral regularization is used for spatially regularizing the luminance component, resulting in sharp edges and forcing interpolation along the edges and not across them. Simultaneously, Tikhonov regularization is used to smooth the chrominance components. Finally, an additional regularization term is used to force similar edge location and orientation in different color channels. We show that the minimization of the total cost function is relatively easy and fast. Experimental results on synthetic and real data sets confirm the effectiveness of our method. <s> BIB025 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> We study the problem of reconstruction of a high-resolution image from several blurred low-resolution image frames. The image frames consist of blurred, decimated, and noisy versions of a high-resolution image. The high-resolution image is modeled as a Markov random field (MRF), and a maximum a posteriori (MAP) estimation technique is used for the restoration. We show that with the periodic boundary condition, a high-resolution image can be restored efficiently by using fast Fourier transforms. We also apply the preconditioned conjugate gradient method to restore high-resolution images in the aperiodic boundary condition. Computer simulations are given to illustrate the effectiveness of the proposed approach. <s> BIB026 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> Super-resolution reconstruction is the process of reconstructing a high-resolution image from multiple low-resolution images. Most super-resolution reconstruction methods assume that exposure time is fixed for all observations, which is not necessarily true. In reality, cameras have limited dynamic range and nonlinear response to the quantity of light received, and exposure time might be adjusted automatically or manually to capture the desired portion of the scene's dynamic range. In this letter, we propose a Bayesian super-resolution algorithm based on an imaging model that includes camera response function, exposure time, sensor noise, and quantization error in addition to spatial blurring and sampling. <s> BIB027 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> Super-resolution reconstruction produces one or a set of high-resolution (HR) images from a sequence of low-resolution (LR) images. Due to translational registration, super-resolution reconstruction can apply only on the sequences that have simple translation motion. This paper proposed a novel super-resolution reconstruction that that can apply on real sequences or complex motion sequences. The proposed super-resolution reconstruction uses a high accuracy registration algorithm, the fast affine block-based registration [16], in the maximum likelihood framework. Moreover, the regularization is used to compensate the missing measurement information. The experimental results show that the proposed reconstruction can apply on real sequence such as Foreman and Suzie. <s> BIB028 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> Super-Resolution Reconstruction (SRR) aims to produces one or a set of high-resolution (HR) images from a sequence of low-resolution (LR) images. Due to translational registration, super-resolution reconstruction can apply only on the sequences that have simple translation motion. This paper proposed a novel super-resolution reconstruction that that can apply on real sequences or complex motion sequences. The proposed SRR uses a high accuracy registration algorithm, the fast affine block-based registration [16], in the maximum likelihood framework. Moreover, the BTV (Bi-Total Variance) regularization [12] is used to compensate the missing measurement information. The experimental results show that the proposed reconstruction can apply on real sequence such as Suzie. <s> BIB029 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> This paper develops a multi-frame image super-resolution approach from a Bayesian view-point by marginalizing over the unknown registration parameters relating the set of input low-resolution views. In Tipping and Bishop's Bayesian image super-resolution approach [16], the marginalization was over the super-resolution image, necessitating the use of an unfavorable image prior. By integrating over the registration parameters rather than the high-resolution image, our method allows for more realistic prior distributions, and also reduces the dimension of the integral considerably, removing the main computational bottleneck of the other algorithm. In addition to the motion model used by Tipping and Bishop, illumination components are introduced into the generative model, allowing us to handle changes in lighting as well as motion. We show results on real and synthetic datasets to illustrate the efficacy of this approach. <s> BIB030 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> In this work, we propose a new robust and edge-preserving super-resolution algorithm to simultaneously estimate all frames of a sequence. The new algorithm is based on the regularized super-resolution approach. In contrast to other multi-frame super-resolution algorithms, the proposed algorithm does not include the motion in the observation model. Instead, transformations caused by the motion are used in the prior model to produce a sequence with improved quality and smoothness in the motion trajectory. We use a Huber norm in the prior term to achieve an algorithm robust to outliers in the motion model while avoiding blurring of edges. The proposed method is significantly more robust than other simultaneous super-resolution methods. We provide results to illustrate the performance of the algorithm. <s> BIB031 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> In this paper, we study face hallucination, or synthesizing a high-resolution face image from an input low-resolution image, with the help of a large collection of other high-resolution face images. Our theoretical contribution is a two-step statistical modeling approach that integrates both a global parametric model and a local nonparametric model. At the first step, we derive a global linear model to learn the relationship between the high-resolution face images and their smoothed and down-sampled lower resolution ones. At the second step, we model the residue between an original high-resolution image and the reconstructed high-resolution image after applying the learned linear model by a patch-based non-parametric Markov network to capture the high-frequency content. By integrating both global and local models, we can generate photorealistic face images. A practical contribution is a robust warping algorithm to align the low-resolution face images to obtain good hallucination results. The effectiveness of our approach is demonstrated by extensive experiments generating high-quality hallucinated face images from low-resolution input with no manual alignment. <s> BIB032 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> In this paper, we propose a learning-based method for video super-resolution. There are two main contributions of the proposed method. First, information from cameras with different spatial-temporal resolutions is combined in our framework. This is achieved by constructing training dictionary using the high resolution images captured by still camera and the low resolution video is enhanced via searching in this customized database. Second, we enforce the spatio-temporal constraints using the conditional random field (CRF) and the problem of video super-resolution is posed as finding the high resolution video that maximizes the conditional probability. We apply the algorithm to video sequences taken from different scenes using cameras with different qualities and promising results are presented. <s> BIB033 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> This paper presents a new method for super-resolution (SR) reconstruction of a high-resolution (HR) image from several low-resolution (LR) images. The HR image is assumed to be composed of homogeneous regions. Thus, the a priori distribution of the pixels is modeled by a finite mixture model (FMM) and a Potts Markov model (PMM) for the labels. The whole a priori model is then a hierarchical Markov model. The LR images are assumed to be obtained from the HR image by lowpass filtering, arbitrarily translation, decimation, and finally corruption by a randomnoise. The problem is then put in a Bayesian detection and estimation framework, and appropriate algorithms are developed based on Markov chain Monte Carlo (MCMC) Gibbs sampling. At the end, we have not only an estimate of the HR image but also an estimate of the classification labels which leads to a segmentation result. <s> BIB034 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> This article proposes a new framework to regularize linear inverse problems using the total variation on non-local graphs. This non-local graph allows to adapt the penalization to the geometry of the underlying function to recover. A fast algorithm computes iteratively both the solution of the regularization process and the non-local graph adapted to this solution. We show numerical applications of this method to the resolution of image processing inverse problems such as inpainting, super-resolution and compressive sampling. <s> BIB035 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> Effective image prior is necessary for image super resolution, due to its severely under-determined nature. Although the edge smoothness prior can be effective, it is generally difficult to have analytical forms to evaluate the edge smoothness, especially for soft edges that exhibit gradual intensity transitions. This paper finds the connection between the soft edge smoothness and a soft cut metric on an image grid by generalizing the Geocuts method (Y. Boykov and V. Kolmogorov, 2003), and proves that the soft edge smoothness measure approximates the average length of all level lines in an intensity image. This new finding not only leads to an analytical characterization of the soft edge smoothness prior, but also gives an intuitive geometric explanation. Regularizing the super resolution problem by this new form of prior can simultaneously minimize the length of all level lines, and thus resulting in visually appealing results. In addition, this paper presents a novel combination of this soft edge smoothness prior and the alpha matting technique for color image super resolution, by normalizing edge segments with their alpha channel description, to achieve a unified treatment of edges with different contrast and scale. <s> BIB036 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> Face recognition at a distance is a challenging and important law-enforcement surveillance problem, with low image resolution and blur contributing to the difficulties. We present a method for combining a sequence of video frames of a subject in order to create a super-resolved image of the face with increased resolution and reduced blur. An Active Appearance Model (AAM) of face shape and appearance is fit to the face in each video frame. The AAM fit provides the registration used by a robust image super-resolution algorithm that iteratively solves for a higher resolution face image from a set of video frames. This process is tested with real-world outdoor video using a PTZ camera and a commercial face recognition engine. Both improved visual perception and automatic face recognition performance are observed in these experiments. <s> BIB037 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> This work presents a new class of algorithms for super-resolution (SR) of image sequences. This class of algorithms estimates simultaneously all frames of a sequence by employing an iterative minimization of a regularized cost function. Similarly to other SR techniques, the proposed approach exploits the correlation among the frames of the sequence. This correlated information helps to improve the resolution of the captured images. By employing the motion information only in the prior term of the cost function, the proposed method achieves a better fidelity and more robust performance, when compared to other methods. This original approach provides a fidelity equivalent to the obtained by the simultaneous methods, while achieving a lower computational complexity. The isolation of the motion equations in the prior term allows an increased control over the motion errors, and as a consequence, provides an improved robustness. The performance of the proposed method is discussed and compared with other methods in the literature. In the comparative experiments, it is considered both the minimization over the Euclidean norm, which is employed to achieve low computational complexity, and the minimization over the Huber norm or norm, which is used to produce images with sharp edges and higher robustness to large motion errors. <s> BIB038 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> We present a novel method of Bayesian image super-resolution in which marginalization is carried out over latent parameters such as geometric and photometric registration and the image point-spread function. Related Bayesian super-resolution approaches marginalize over the high-resolution image, necessitating the use of an unfavourable image prior, whereas our method allows for more realistic image prior distributions, and reduces the dimension of the integral considerably, removing the main computational bottleneck of algorithms such as Tipping and Bishop's Bayesian image super-resolution. We show results on real and synthetic datasets to illustrate the efficacy of our method. <s> BIB039 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> Super resolution image reconstruction allows the recovery of a high-resolution (HR) image from several low-resolution images that are noisy, blurred, and down sampled. In this paper, we present a joint formulation for a complex super-resolution problem in which the scenes contain multiple independently moving objects. This formulation is built upon the maximum a posteriori (MAP) framework, which judiciously combines motion estimation, segmentation, and super resolution together. A cyclic coordinate descent optimization procedure is used to solve the MAP formulation, in which the motion fields, segmentation fields, and HR images are found in an alternate manner given the two others, respectively. Specifically, the gradient-based methods are employed to solve the HR image and motion fields, and an iterated conditional mode optimization method to obtain the segmentation fields. The proposed algorithm has been tested using a synthetic image sequence, the "Mobile and Calendar" sequence, and the original "Motorcycle and Car" sequence. The experiment results and error analyses verify the efficacy of this algorithm <s> BIB040 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> We investigate new ways of improving the performance of Bayesian-based super-resolution image reconstruction by using a discontinuity adaptive image prior distribution based on robust statistics and a fast and efficient way of initializing the optimization process. The latter is an adapted normalized convolution (NC) technique that incorporates the uncertainty induced by registration errors. We present both qualitative and quantitative results on real video sequences and demonstrate the advantages of the proposed method compared to conventional methodologies. <s> BIB041 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> We address the problem of super resolved generation of novel views of a 3D scene with the reference images obtained from cameras in general positions; a problem which has not been tackled before in the context of super resolution and is also of importance to the field of image based rendering. We formulate the problem as one of estimation of the color at each pixel in the high resolution novel view without explicit and accurate depth recovery. We employ a reconstruction based approach using MRF-MAP formalism and solve using graph cut optimization. We also give an effective method to handle occlusion. We present compelling results on real images. <s> BIB042 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> The MRI image reconstruction supported by Toeplitz-based iterative super-resolution algorithm is presented in the paper. It is shown that the approach improves MRI spatial resolution in cases when multi-blade sequences are used. The PROPELLER MRI method collects data from rectangular regions (blades) rotated around the origin of the k-space. Inter-blade patient motion is the premise for the use of super-resolution technique. Images obtained from sets of irregularly located frequency domain samples are combined into the high resolution MRI image. The super-resolution reconstruction replaces usually applied direct averaging of low-resolution images. Furthermore high order affine motion model has been applied. <s> BIB043 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> In this paper, we propose an image super-resolution approach using a novel generic image prior - gradient profile prior, which is a parametric prior describing the shape and the sharpness of the image gradients. Using the gradient profile prior learned from a large number of natural images, we can provide a constraint on image gradients when we estimate a hi-resolution image from a low-resolution image. With this simple but very effective prior, we are able to produce state-of-the-art results. The reconstructed hi-resolution image is sharp while has rare ringing or jaggy artifacts. <s> BIB044 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> This paper explains and implements our efficient multi-frame super-resolution mosaicking algorithm. In this algorithm, feature points between images are matched using SIFT, and then random M-least squares is used to estimate the homography between frames. Next, separate frames are registered and the overlapping region is extracted. A generative model is then adopted and combined with maximum a posteriori estimation to construct the underdetermined sparse linear system. To solve the ill-posed large-scale inverse system, we derive and implement a new hybrid regularization (bilateral total variance Hubert) method. Cross validation is utilized to estimate the derivative of the blur kernel as well as the regularization parameter. Super-resolution is then applied to the individual sub-frames from the overlapping region. Finally, multi-band blending is used to stitch these resolution-enhanced frames to form the final image. The whole process is semi-real time (roughly 30 seconds for 35 frames) and the effectiveness of our algorithm is validated by applying it to real and synthetic UAV video frames. <s> BIB045 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> In this paper we formulate a new time dependent convolutional model for super-resolution based on a constrained variational model that uses the total variation of the signal as a regularizing functional. We propose an iterative refinement procedure based on Bregman iteration to improve spatial resolution. The model uses a dataset of low resolution images and incorporates a downsampling operator to relate the high resolution scale to the low resolution one. We present an algorithm for the model and we perform a series of numerical experiments to show evidence of the good behavior of the numerical scheme and quality of the results. <s> BIB046 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> Human faces and license plates are the most important targets for many digital surveillance systems. The image quality is too poor to recognize the targets due to the uncontrollable effects such as poor light conditions or far distances from the concerned objects. The cameras are zoomed in to make the targets to be discernible. Image interpolation and super-resolution are two popular techniques for removing the blurry effects of the zoomed images. In this paper, the super-resolution approach is applied on the reconstructions of facial and license plate images. Training samples are collected and their pyramidal edge images are built. The intuition idea for enhancing image quality is to preserve the edge data of the high resolutional images. Face and license plate hallucination are constructed. Some experimental results are conducted to show the effectiveness of the proposed approach. Some conclusions and future works are given. <s> BIB047 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> Designing effective image priors is of great interest to image super-resolution (SR), which is a severely under-determined problem. An edge smoothness prior is favored since it is able to suppress the jagged edge artifact effectively. However, for soft image edges with gradual intensity transitions, it is generally difficult to obtain analytical forms for evaluating their smoothness. This paper characterizes soft edge smoothness based on a novel SoftCuts metric by generalizing the Geocuts method . The proposed soft edge smoothness measure can approximate the average length of all level lines in an intensity image. Thus, the total length of all level lines can be minimized effectively by integrating this new form of prior. In addition, this paper presents a novel combination of this soft edge smoothness prior and the alpha matting technique for color image SR, by adaptively normalizing image edges according to their alpha-channel description. This leads to the adaptive SoftCuts algorithm, which represents a unified treatment of edges with different contrasts and scales. Experimental results are presented which demonstrate the effectiveness of the proposed method. <s> BIB048 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> In this paper, an edge-preserving nonlinear iterative regularization-based image resampling method for a single noise-free image is proposed. Several aspects of the resampling algorithm are investigated: choice of discrepancy and regularization norms, improvements of convergence speed using edge-directional steepest-descent method and patch-based details synthesis. A model of a downsampling operator based on a camera observation model is considered. <s> BIB049 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> We proposed a compressed sensing Super Resolution algorithm based on wavelet. The proposed algorithm performs well with a smaller quantity of training image patches and outputs images with satisfactory subjective quality. It is tested on classical images commonly adopted by Super Resolution researchers with both generic and specialized training sets for comparison with other popular commercial software and state-of-the-art methods. Experiments demonstrate that, the proposed algorithm is competitive among contemporary Super Resolution methods. <s> BIB050 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> This paper proposes a framework for single-image super-resolution. The underlying idea is to learn a map from input low-resolution images to target high-resolution images based on example pairs of input and output images. Kernel ridge regression (KRR) is adopted for this purpose. To reduce the time complexity of training and testing for KRR, a sparse solution is found by combining the ideas of kernel matching pursuit and gradient descent. As a regularized solution, KRR leads to a better generalization than simply storing the examples as has been done in existing example-based algorithms and results in much less noisy images. However, this may introduce blurring and ringing artifacts around major edges as sharp changes are penalized severely. A prior model of a generic image class which takes into account the discontinuity property of images is adopted to resolve this problem. Comparison with existing algorithms shows the effectiveness of the proposed method. <s> BIB051 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> We present a variational approach to obtain high-resolution images from multiframe low-resolution video stills. The objective functional for the variational approach consists of a data fidelity term and a regularizer. The fidelity term is formed by adaptively mimicking l1 and l2 norms. The regularization uses the l1 norm of the framelet coefficients of a high-resolution image with a geometric tight framelet system constructed in this paper. The tight framelet system has abilities to detect multi-orientation and multi-order variations of an image. A two-phase iterative method for super-resolution reconstruction is proposed to construct a high-resolution image. The first phase is to get an approximation of the solution (i.e., the ideal image) using the steepest descent method. The second phase is to enhance the sparsity of the approximate solution by using the soft thresholding operator with variable thresholding parameters. Numerical results based on both synthetic data and real videos show that our algorithm is efficient in terms of removing visual artifacts and preserving edges in restored images. <s> BIB052 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> This paper proposes a context-constrained hallucination approach for image super-resolution. Through building a training set of high-resolution/low-resolution image segment pairs, the high-resolution pixel is hallucinated from its texturally similar segments which are retrieved from the training set by texture similarity. Given the discrete hallucinated examples, a continuous energy function is designed to enforce the fidelity of high-resolution image to low-resolution input and the constraints imposed by the hallucinated examples and the edge smoothness prior. The reconstructed high-resolution image is sharp with minimal artifacts both along the edges and in the textural regions. <s> BIB053 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> This paper addresses the localization and super resolution of license plate in a unifying approach. Higher quality license plate can be obtained using super resolution on successive lower resolution plate images. All existing methods assume that plate zones are correctly extracted from every frame. However, the accurate localization needs a sufficient quality of the image, which is not always true in real video. Super-resolution on all pixels is a possible but much time consuming alternative. We propose a framework which interlaces successfully these two modules. First, coarse candidates are found by an weak but fast license plate detection based on edge map sub-sampling. Then, an improved fast MAP-based super-resolution, using local phase accurate registration and edge preserving prior, applied on these regions of interest. Finally, our robust ICHT-based localizer rejects false-alarms and localizes the high resolution license plate more accurately. Experiments which were conducted on synthetic and real data, proved the robustness of our approach with real-time possibility. <s> BIB054 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> The objective of super-resolution (SR) imaging is to reconstruct a single higher-resolution image based on a set of lower-resolution images that were acquired from the same scene to overcome the limitations of image acquisition process for facilitating better visualization and content recognition. In this paper, a stochastic Markov chain Monte Carlo (MCMC) SR image reconstruction approach is proposed. First, a Bayesian inference formulation, which is based on the observed low-resolution images and the prior high-resolution image model, is mathematically derived. Second, to exploit the MCMC sample-generation technique for the stochastic SR image reconstruction, three fundamental issues are observed as follows. First, since the hyperparameter value of the prior image model controls the degree of regularization and intimately affects the quality of the reconstructed high-resolution image, how to determine an optimal hyperparameter value for different low-resolution input images becomes a very challenging task. Rather than exploiting the exhaustive search, an iterative updating approach is developed in this paper by allowing the value of hyperparameter being simultaneously updated in each sample-generation iteration. Second, the samples generated during the so-called burn-in period (measured in terms of the number of samples initially generated) of the MCMC-based sample-generation process are considered unreliable and should be discarded. To determine the length of the burn-in period for each set of low-resolution input images, a time-period bound in closed form is mathematically derived. Third, image artifacts could be incurred in the reconstructed high-resolution image, if the number of samples (counting after the burn-in period) generated by the MCMC-based sample-generation process is insufficient. For that, a variation-sensitive bilateral filter is proposed as a 'complementary' post-processing scheme, to improve the reconstructed high-resolution image quality, when the number of samples is insufficient. Extensive simulation results have clearly shown that the proposed stochastic SR image reconstruction method consistently yields superior performance. <s> BIB055 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> We present a complete super-resolution system using a camera, that is assumed to be on a vibrating platform and continually capturing frames of a static scene, that have to be super-resolved in particular regions of interest. In a practical system the shutter of the camera is not synchronised with the vibrations it is subjected to. So, we propose a novel method for frame selection according to their degree of blurring and we combine a tracker with the sequence of selected frames to identify the subimages containing the region of interest. The extracted subimages are subsequently co-registered using a state of the art sub-pixel registration algorithm. Further selection of the co-registered subimages takes place, according to the confidence in the registration result. Finally, the subimage of interest is super-resolved using a state of the art super-resolution algorithm. The proposed frame selection method is of generic applicability and it is validated with the help of manual frame quality assessment. <s> BIB056 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> We formulate the multiframe blind deconvolution problem in an incremental expectation maximization (EM) framework. Beyond deconvolution, we show how to use the same framework to address: (i) super-resolution despite noise and unknown blurring; (ii) saturation-correction of overexposed pixels that confound image restoration. The abundance of data allows us to address both of these without using explicit image or blur priors. The end result is a simple but effective algorithm with no hyperparameters. We apply this algorithm to real-world images from astronomy and to super resolution tasks: for both, our algorithm yields increased resolution and deconvolved images simultaneously. <s> BIB057 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> Given a set of images captured using different integer zoom settings of a camera, we propose a fast approach to obtain super-resolution (SR) for the least zoomed image at a resolution of the most zoomed image. We first obtain SR approximation to the superresolved image using a learning based approach that uses training database consisting of low-resolution (LR) and their high-resolution (HR) images. We model the LR observations as the aliased and noisy versions of their HR parts and estimate the decimation using the learned SR approximation and the available least zoomed observation. A discontinuity preserving Markov random field (MRF) is used as a prior and its parameters are estimated using the SR approximation. Finally Maximum a posteriori (MAP)-MRF formulation is used and the final cost function is optimized using particle swarm optimization (PSO) technique which is computationally efficient compared to simulated annealing. The proposed method can be used in multiresolution fusion for remotely sensed images where the available HR panchromatic image can be used to obtain HR multispectral images. Another interesting application is in immersive navigation for walk through application. Here one can change zoom setting without compromising on the spatial resolution. <s> BIB058 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> In this paper, a selective perceptual-based (SELP) framework is presented to reduce the complexity of popular super-resolution (SR) algorithms while maintaining the desired quality of the enhanced images/video. A perceptual human visual system model is proposed to compute local contrast sensitivity thresholds. The obtained thresholds are used to select which pixels are super-resolved based on the perceived visibility of local edges. Processing only a set of perceptually significant pixels reduces significantly the computational complexity of SR algorithms without losing the achievable visual quality. The proposed SELP framework is integrated into a maximum-a posteriori-based SR algorithm as well as a fast two-stage fusion-restoration SR estimator. Simulation results show a significant reduction on average in computational complexity with comparable signal-to-noise ratio gains and visual quality. <s> BIB059 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> In this paper, we propose a novel generic image prior-gradient profile prior, which implies the prior knowledge of natural image gradients. In this prior, the image gradients are represented by gradient profiles, which are 1-D profiles of gradient magnitudes perpendicular to image structures. We model the gradient profiles by a parametric gradient profile model. Using this model, the prior knowledge of the gradient profiles are learned from a large collection of natural images, which are called gradient profile prior. Based on this prior, we propose a gradient field transformation to constrain the gradient fields of the high resolution image and the enhanced image when performing single image super-resolution and sharpness enhancement. With this simple but very effective approach, we are able to produce state-of-the-art results. The reconstructed high resolution images or the enhanced images are sharp while have rare ringing or jaggy artifacts. <s> BIB060 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> In this paper, we propose an energy-based algorithm for motion-compensated video super-resolution (VSR) targeted on upscaling of standard definition (SD) video to high-definition (HD) video. Since the motion (flow field) of the image sequence is generally unknown, we introduce a formulation for the joint estimation of a super-resolution (SR) sequence and its flow field. Via the calculus of variations, this leads to a coupled system of partial differential equations for image sequence and motion estimation. We solve a simplified form of this system and, as a by-product, we indeed provide a motion field for super-resolved sequences. To the best of our knowledge, computing super-resolved flows has not been done before. Most advanced SR methods found in literature cannot be applied to general video with arbitrary scene content and/or arbitrary optical flows, as it is possible with our simultaneous VSR method. A series of experiments shows that our method outperforms other VSR methods when dealing with general video input and that it continues to provide good results even for large scaling factors up to 8 × 8. <s> BIB061 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> The neighbor-embedding (NE) algorithm for single-image super-resolution (SR) reconstruction assumes that the feature spaces of low-resolution (LR) and high-resolution (HR) patches are locally isometric. However, this is not true for SR because of one-to-many mappings between LR and HR patches. To overcome or at least to reduce the problem for NE-based SR reconstruction, we apply a joint learning technique to train two projection matrices simultaneously and to map the original LR and HR feature spaces onto a unified feature subspace. Subsequently, the k -nearest neighbor selection of the input LR image patches is conducted in the unified feature subspace to estimate the reconstruction weights. To handle a large number of samples, joint learning locally exploits a coupled constraint by linking the LR-HR counterparts together with the K-nearest grouping patch pairs. In order to refine further the initial SR estimate, we impose a global reconstruction constraint on the SR outcome based on the maximum a posteriori framework. Preliminary experiments suggest that the proposed algorithm outperforms NE-related baselines. <s> BIB062 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> We propose a simple preconditioning method for accelerating the solution of edge-preserving image super-resolution (SR) problems in which a linear shift-invariant point spread function is employed. Our technique involves reordering the high-resolution (HR) pixels in a similar manner to what is done in preconditioning methods for quadratic SR formulations. However, due to the edge preserving requirements, the Hessian matrix of the cost function varies during the minimization process. We develop an efficient update scheme for the preconditioner in order to cope with this situation. Unlike some other acceleration strategies that round the displacement values between the low-resolution (LR) images on the HR grid, the proposed method does not sacrifice the optimality of the observation model. In addition, we describe a technique for preconditioning SR problems involving rational magnification factors. The use of such factors is motivated in part by the fact that, under certain circumstances, optimal SR zooms are nonintegers. We show that, by reordering the pixels of the LR images, the structure of the problem to solve is modified in such a way that preconditioners based on circulant operators can be used. <s> BIB063 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> Reconstruction- and example-based super-resolution (SR) methods are promising for restoring a high-resolution (HR) image from low-resolution (LR) image(s). Under large magnification, reconstruction-based methods usually fail to hallucinate visual details while example-based methods sometimes introduce unexpected details. Given a generic LR image, to reconstruct a photo-realistic SR image and to suppress artifacts in the reconstructed SR image, we introduce a multi-scale dictionary to a novel SR method that simultaneously integrates local and non-local priors. The local prior suppresses artifacts by using steering kernel regression to predict the target pixel from a small local area. The non-local prior enriches visual details by taking a weighted average of a large neighborhood as an estimate of the target pixel. Essentially, these two priors are complementary to each other. Experimental results demonstrate that the proposed method can produce high quality SR recovery both quantitatively and perceptually. <s> BIB064 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> We propose a new single image super resolution (SR) algorithm via Bayesian modeling with a natural image prior modeled by a high-order Markov random field (MRF). SR is one of the long-standing and active topics in image processing community. It is of great use in many practical applications, such as astronomical observation, medical imaging, and the adaptation of low-resolution contents onto high-resolution displays. One category of the conventional approaches for image SR is formulating the problem with Bayesian modeling techniques and then obtaining its maximum-a-posteriori solution, which actually boils down to a regularized regression task. Although straightforward, this approach cannot exploit the full potential offered by the probabilistic modeling, as only the posterior mode is sought. On the other hand, current Bayesian SR approaches using the posterior mean estimation typically use very simple prior models for natural images to ensure the computational tractability. In this paper, we present a Bayesian image SR approach with a flexible high-order MRF model as the prior for natural images. The minimum mean square error (MMSE) criteria are used for estimating the HR image. A Markov chain Monte Carlo-based sampling algorithm is presented for obtaining the MMSE solution. The proposed method cannot only enjoy the benefits offered by the flexible prior, but also has the advantage of making use of the probabilistic modeling to perform a posterior mean estimation, thus is less sensitive to the local minima problem as the MAP solution. Experimental results indicate that the proposed method can generate competitive or better results than state-of-the-art SR algorithms. <s> BIB065 </s> Super-resolution: a comprehensive survey <s> Cost functions and optimization methods <s> Multiscale morphological operators are studied extensively in the literature for image processing and feature extraction purposes. In this paper, we model a nonlinear regularization method based on multiscale morphology for edge-preserving super resolution (SR) image reconstruction. We formulate SR image reconstruction as a deblurring problem and then solve the inverse problem using Bregman iterations. The proposed algorithm can suppress inherent noise generated during low-resolution image formation as well as during SR image estimation efficiently. Experimental results show the effectiveness of the proposed regularization and reconstruction method for SR image. <s> BIB066
The most common cost function, from both algebraic and statistical perspectives, is the LS cost function, minimizing the l 2 norm of the residual vector of the imaging model of Eq. (4) BIB011 BIB013 BIB023 . If the noise in the model is additive and Gaussian with zero mean, the LS cost function estimates the ML of f BIB011 . Farsiu et al. BIB009 BIB024 BIB025 proposed to use l 1 instead of the usual l 2 for both the regularization and error terms, as it is more robust against outliers than l 2 . However, choosing the proper optimization method depends on the nature of the cost function. If it is a convex function, gradient-based methods (like gradient descent, (scaled) conjugate gradient, preconditioned conjugate gradient) might be used for finding the solution (Table 10 ). The only difference between conjugate and gradient descent is in the gradient direction. BIB003 BIB004 BIB007 BIB005 BIB010 BIB012 BIB014 BIB013 BIB015 BIB016 BIB017 BIB018 BIB019 BIB020 BIB026 BIB023 BIB024 BIB027 BIB028 BIB029 BIB030 BIB031  BIB036  BIB037 BIB038 BIB043 BIB044 BIB045 BIB047 BIB048 BIB049 BIB051 BIB052 BIB053 BIB059 BIB060 BIB062 BIB063 BIB064 If it is non-linear, a fixed-point iteration algorithm like Gauss-Seidel iteration BIB002  BIB032  BIB039  BIB040 BIB061 is appropriate. If it is non-convex, the time-consuming simulated annealing can be used BIB001   BIB033 BIB054 , or else graduated non-convexity BIB006 BIB041 BIB055 (with normalized convolution for obtaining an initial good approximation) BIB056 , EM BIB008 BIB014  BIB057 , genetic algorithm BIB021 , Markov chain Monte Carlo using Gibbs sampler BIB022  BIB034 BIB033  BIB065 , energy minimization using graph-cuts  BIB036 BIB042  , Bregman iteration BIB046 BIB066 , proximal iteration BIB035 , (regularized) orthogonal matching pursuit BIB050 BIB051 , and particle swarm optimization BIB058 might be used.