Super-resolution: a comprehensive survey <s> Color images <s> Abstract Image resolution can be improved when the relative displacements in image sequences are known accurately, and some knowledge of the imaging process is available. The proposed approach is similar to back-projection used in tomography. Examples of improved image resolution are given for gray-level and color images, when the unknown image displacements are computed from the image sequence. <s> BIB001 </s> Super-resolution: a comprehensive survey <s> Color images <s> Image sensing is usually done with multiple sensors, like the RGB sensors in color imaging, the IR and EO sensors in surveillance and satellite imaging, etc. The resolution of each sensor can be increased by considering the images of the other sensors, and using the statistical redundancy among the sensors. Particularly, we use the fact that most discontinuities in the image of one sensor correspond to discontinuities in the other sensors. Two applications are presented: Increasing the resolution of a single color image by using the correlation among the three color channels, and enhancing noisy IR images. <s> BIB002 </s> Super-resolution: a comprehensive survey <s> Color images <s> We propose a Bayesian approach to image hallucination. Given a generic low resolution image, we hallucinate a high resolution image using a set of training images. Our work is inspired by recent progress on natural image statistics that the priors of image primitives can be well represented by examples. Specifically, primal sketch priors (e.g., edges, ridges and corners) are constructed and used to enhance the quality of the hallucinated high resolution image. Moreover, a contour smoothness constraint enforces consistency of primitives in the hallucinated image by a Markov-chain based inference algorithm. A reconstruction constraint is also applied to further improve the quality of the hallucinated image. Experiments demonstrate that our approach can hallucinate high quality super-resolution images. <s> BIB003 </s> Super-resolution: a comprehensive survey <s> Color images <s> In this paper, we propose a novel method for solving single-image super-resolution problems. Given a low-resolution image as input, we recover its high-resolution counterpart using a set of training examples. While this formulation resembles other learning-based methods for super-resolution, our method has been inspired by recent manifold teaming methods, particularly locally linear embedding (LLE). Specifically, small image patches in the lowand high-resolution images form manifolds with similar local geometry in two distinct feature spaces. As in LLE, local geometry is characterized by how a feature vector corresponding to a patch can be reconstructed by its neighbors in the feature space. Besides using the training image pairs to estimate the high-resolution embedding, we also enforce local compatibility and smoothness constraints between patches in the target high-resolution image through overlapping. Experiments show that our method is very flexible and gives good empirical results. <s> BIB004 </s> Super-resolution: a comprehensive survey <s> Color images <s> We propose a single-frame, learning-based super-resolution restoration technique by using the wavelet domain to define a constraint on the solution. Wavelet coefficients at finer scales of the unknown high-resolution image are learned from a set of high-resolution training images and the learned image in the wavelet domain is used for further regularization while super-resolving the picture. We use an appropriate smoothness prior with discontinuity preservation in addition to the wavelet-based constraint to estimate the super-resolved image. The smoothness term ensures the spatial correlation among the pixels, whereas the learning term chooses the best edges from the training set. Because this amounts to extrapolating the high-frequency components, the proposed method does not suffer from oversmoothing effects. The results demonstrate the effectiveness of the proposed approach. © 2004 Wiley Periodicals, Inc. Int J Imaging Syst Technol 14, 105–112, 2004; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ima.20013 <s> BIB005 </s> Super-resolution: a comprehensive survey <s> Color images <s> Super-Resolution reconstruction produces one or a set of high-resolution images from a sequence of low-resolution frames. This article reviews a variety of Super-Resolution methods proposed in the last 20 years, and provides some insight into, and a summary of, our recent contributions to the general Super-Resolution problem. In the process, a detailed study of several very important aspects of Super-Resolution, often ignored in the literature, is presented. Spe- cifically, we discuss robustness, treatment of color, and dynamic operation modes. Novel methods for addressing these issues are accompanied by experimental results on simulated and real data. Finally, some future challenges in Super-Resolution are outlined and discussed. © 2004 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 14, 47-57, 2004; Published online in Wiley InterScience (www.interscience.wiley. com). DOI 10.1002/ima.20007 <s> BIB006 </s> Super-resolution: a comprehensive survey <s> Color images <s> Super-resolution has been applied mainly to grayscale images, but producing a high-resolution color image using a single CCD sensor has not been investigated in detail until recently. This work aims at producing a high-resolution color image directly from raw "color mosaic" images obtained by a single CCD equipped with a color filter array. This method is based on a generalized formulation of super-resolution that simultaneously performs both resolution enhancement and demosaicing. The other key factor of our topic is a precise sub-pixel registration of multiple raw images. We examined direct registration of raw images based on an imaging model, which shows precise estimation of motion among severely aliased raw images. The proposed method is verified through experiments using synthetic and real images. <s> BIB007 </s> Super-resolution: a comprehensive survey <s> Color images <s> Images magnified by standard methods display a degradation of detail that is particularly noticeable in the blurry edges of text. Current super-resolution algorithms address the lack of sharpness by filling in the image with probable details. These algorithms break the outlines of text. Our novel algorithm for super-resolution of text magnifies images in real-time by interpolation with a variable linear filter. The coefficients of the filter are determined nonlinearly from the neighborhood to which it is applied. We train the mapping that defines the coefficients to specifically enhance edges of text, producing a conservative algorithm that infers the detail of magnified text. Possible applications include resizing web page layouts or other interfaces, and enhancing low resolution camera captures of text. In general, learning spatially-variable filters is applicable to other image filtering tasks. <s> BIB008 </s> Super-resolution: a comprehensive survey <s> Color images <s> In this paper, we propose a robust approach to super-resolution static sprite generation from multiple low-resolution images. Considering both short-term and long-term motion influences, a hybrid global motion estimation technique is first presented for sprite generation. An iterative super-resolution reconstruction algorithm is then proposed for the super-resolution sprite construction. This algorithm is robust to outliers and results in a sprite image with high visual quality. In addition, the generated super-resolution sprite can provide better reconstructed images than a conventional low-resolution sprite. Experimental results demonstrate the effectiveness of the proposed methods. <s> BIB009 </s> Super-resolution: a comprehensive survey <s> Color images <s> Theoretical and practical limitations usually constrain the achievable resolution of any imaging device. Super-Resolution (SR) methods are developed through the years to go beyond this limit by acquiring and fusing several low-resolution (LR) images of the same scene, producing a high-resolution (HR) image. The early works on SR, ::: although occasionally mathematically optimal for particular models of data and noise, produced poor results when applied to real images. In this paper, we discuss two of the main issues related to designing a practical SR system, namely reconstruction accuracy and computational efficiency. Reconstruction accuracy refers to the problem of designing a robust SR method applicable to images from different imaging systems. We study a general framework for optimal reconstruction of images from grayscale, color, or color filtered (CFA) cameras. The performance of our proposed method is boosted by using powerful priors and is robust to both measurement (e.g. CCD read out noise) and system noise (e.g. motion estimation error). Noting that the motion estimation is often considered a bottleneck in terms of SR performance, we introduce the concept of "constrained motions" for enhancing the quality of super-resolved images. We show that using such constraints will enhance the quality of the motion estimation and therefore results in more accurate reconstruction of the HR images. We also justify some practical assumptions that greatly reduce the computational complexity and memory requirements of the proposed methods. We use efficient approximation of the Kalman Filter (KF) and adopt a dynamic point of view to the SR problem. Novel methods for addressing these issues are accompanied by experimental results on real data. <s> BIB010 </s> Super-resolution: a comprehensive survey <s> Color images <s> In the last two decades, two related categories of problems have been studied independently in image restoration literature: super-resolution and demosaicing. A closer look at these problems reveals the relation between them, and, as conventional color digital cameras suffer from both low-spatial resolution and color-filtering, it is reasonable to address them in a unified context. In this paper, we propose a fast and robust hybrid method of super-resolution and demosaicing, based on a maximum a posteron estimation technique by minimizing a multiterm cost function. The L1 norm is used for measuring the difference between the projected estimate of the high-resolution image and each low-resolution image, removing outliers in the data and errors due to possibly inaccurate motion estimation. Bilateral regularization is used for spatially regularizing the luminance component, resulting in sharp edges and forcing interpolation along the edges and not across them. Simultaneously, Tikhonov regularization is used to smooth the chrominance components. Finally, an additional regularization term is used to force similar edge location and orientation in different color channels. We show that the minimization of the total cost function is relatively easy and fast. Experimental results on synthetic and real data sets confirm the effectiveness of our method. <s> BIB011 </s> Super-resolution: a comprehensive survey <s> Color images <s> In this paper, we propose a novel learning-based method for image hallucination, with image super-resolution being a specific application that we focus on here. Given a low-resolution image, its underlying higher-resolution details are synthesized based on a set of training images. In order to build a compact yet descriptive training set, we investigate the characteristic local structures contained in large volumes of small image patches. Inspired by progress in manifold learning research, we take the assumption that small image patches in the low-resolution and high-resolution images form manifolds with similar local geometry in the corresponding image feature spaces. This assumption leads to a super-resolution approach which reconstructs the feature vector corresponding to an image patch by its neighbors in the feature space. In addition, the residual errors associated with the reconstructed image patches are also estimated to compensate for the information loss in the local averaging process. Experimental results show that our hallucination method can synthesize higher-quality images compared with other methods. <s> BIB012 </s> Super-resolution: a comprehensive survey <s> Color images <s> Face recognition at a distance is a challenging and important law-enforcement surveillance problem, with low image resolution and blur contributing to the difficulties. We present a method for combining a sequence of video frames of a subject in order to create a super-resolved image of the face with increased resolution and reduced blur. An Active Appearance Model (AAM) of face shape and appearance is fit to the face in each video frame. The AAM fit provides the registration used by a robust image super-resolution algorithm that iteratively solves for a higher resolution face image from a set of video frames. This process is tested with real-world outdoor video using a PTZ camera and a commercial face recognition engine. Both improved visual perception and automatic face recognition performance are observed in these experiments. <s> BIB013 </s> Super-resolution: a comprehensive survey <s> Color images <s> In this paper we propose a new method for upsampling images which is capable of generating sharp edges with reduced input-resolution grid-related artifacts. The method is based on a statistical edge dependency relating certain edge features of two different resolutions, which is generically exhibited by real-world images. While other solutions assume some form of smoothness, we rely on this distinctive edge dependency as our prior knowledge in order to increase image resolution. In addition to this relation we require that intensities are conserved; the output image must be identical to the input image when downsampled to the original resolution. Altogether the method consists of solving a constrained optimization problem, attempting to impose the correct edge relation and conserve local intensities with respect to the low-resolution input image. Results demonstrate the visual importance of having such edge features properly matched, and the method's capability to produce images in which sharp edges are successfully reconstructed. <s> BIB014 </s> Super-resolution: a comprehensive survey <s> Color images <s> Effective image prior is necessary for image super resolution, due to its severely under-determined nature. Although the edge smoothness prior can be effective, it is generally difficult to have analytical forms to evaluate the edge smoothness, especially for soft edges that exhibit gradual intensity transitions. This paper finds the connection between the soft edge smoothness and a soft cut metric on an image grid by generalizing the Geocuts method (Y. Boykov and V. Kolmogorov, 2003), and proves that the soft edge smoothness measure approximates the average length of all level lines in an intensity image. This new finding not only leads to an analytical characterization of the soft edge smoothness prior, but also gives an intuitive geometric explanation. Regularizing the super resolution problem by this new form of prior can simultaneously minimize the length of all level lines, and thus resulting in visually appealing results. In addition, this paper presents a novel combination of this soft edge smoothness prior and the alpha matting technique for color image super resolution, by normalizing edge segments with their alpha channel description, to achieve a unified treatment of edges with different contrast and scale. <s> BIB015 </s> Super-resolution: a comprehensive survey <s> Color images <s> In this paper, we propose an image super-resolution approach using a novel generic image prior - gradient profile prior, which is a parametric prior describing the shape and the sharpness of the image gradients. Using the gradient profile prior learned from a large number of natural images, we can provide a constraint on image gradients when we estimate a hi-resolution image from a low-resolution image. With this simple but very effective prior, we are able to produce state-of-the-art results. The reconstructed hi-resolution image is sharp while has rare ringing or jaggy artifacts. <s> BIB016 </s> Super-resolution: a comprehensive survey <s> Color images <s> This paper introduces a new color image superresolution algorithm in an adaptive, robust M-estimation framework. Using a robust error norm in the objective function, and adapting the estimation process to each of the low-resolution frames, the proposed method effectively suppresses the outliers due to violations of the assumed observation model, and results in color superresolution estimates with crisp details and no color artifacts, without the use of regularization. Experiments on both synthetic and real sequences demonstrate the superior performance over using the L2 and L1 error norms in the objective function. <s> BIB017 </s> Super-resolution: a comprehensive survey <s> Color images <s> Assuming that the local geometry of low-resolution image patches is similar to that of the high-resolution counterparts, neighbor embedding based super-resolution methods learn a high-resolution image from one or more low-resolution input images by embedding its patches optimally with training ones. However, their performance suffers from inappropriate choices of features, neighborhood sizes and training patches. To address the issues, we propose an extended Neighbor embedding based super-resolution through edge detection and Feature Selection (henceforth NeedFS). Three major contributions of NeedFS are: (1) A new combination of features are proposed, which preserve edges and smoothen color regions better; (2) the training patches are learned discriminately with different neighborhood sizes based on edge detection; (3) only those edge training patches are bootstrapped to provide extra useful information with least redundancy. Experiments show that NeedFS performs better in both quantitative and qualitative evaluation. NeedFS is also robust even with a very limited training set and thus is promising for real applications. <s> BIB018 </s> Super-resolution: a comprehensive survey <s> Color images <s> Methods for super-resolution can be broadly classified into two families of methods: (i) The classical multi-image super-resolution (combining images obtained at subpixel misalignments), and (ii) Example-Based super-resolution (learning correspondence between low and high resolution image patches from a database). In this paper we propose a unified framework for combining these two families of methods. We further show how this combined approach can be applied to obtain super resolution from as little as a single image (with no database or prior examples). Our approach is based on the observation that patches in a natural image tend to redundantly recur many times inside the image, both within the same scale, as well as across different scales. Recurrence of patches within the same image scale (at subpixel misalignments) gives rise to the classical super-resolution, whereas recurrence of patches across different scales of the same image gives rise to example-based super-resolution. Our approach attempts to recover at each pixel its best possible resolution increase based on its patch redundancy within and across scales. <s> BIB019 </s> Super-resolution: a comprehensive survey <s> Color images <s> Human faces and license plates are the most important targets for many digital surveillance systems. The image quality is too poor to recognize the targets due to the uncontrollable effects such as poor light conditions or far distances from the concerned objects. The cameras are zoomed in to make the targets to be discernible. Image interpolation and super-resolution are two popular techniques for removing the blurry effects of the zoomed images. In this paper, the super-resolution approach is applied on the reconstructions of facial and license plate images. Training samples are collected and their pyramidal edge images are built. The intuition idea for enhancing image quality is to preserve the edge data of the high resolutional images. Face and license plate hallucination are constructed. Some experimental results are conducted to show the effectiveness of the proposed approach. Some conclusions and future works are given. <s> BIB020 </s> Super-resolution: a comprehensive survey <s> Color images <s> A typical dynamic reconstruction-based super-resolution video involves three independent processes: registration, fusion and restoration. Fast video super-resolution systems apply translational motion compensation model for registration with low computational cost. Traditional motion compensation model assumes that the whole spectrum of pixels is consistent between frames. In reality, the low frequency component of pixels often varies significantly. We propose a translational motion compensation model via frequency classification for video super-resolution systems. A novel idea to implement motion compensation by combining the up-sampled current frame and the high frequency part of the previous frame through the SAD framework is presented. Experimental results show that the new motion compensation model via frequency classification has an advantage of 2dB gain on average over that of the traditional motion compensation model. The SR quality has 0.25dB gain on average after the fusion process which is to minimize error by making use of the new motion compensated frame. <s> BIB021 </s> Super-resolution: a comprehensive survey <s> Color images <s> Designing effective image priors is of great interest to image super-resolution (SR), which is a severely under-determined problem. An edge smoothness prior is favored since it is able to suppress the jagged edge artifact effectively. However, for soft image edges with gradual intensity transitions, it is generally difficult to obtain analytical forms for evaluating their smoothness. This paper characterizes soft edge smoothness based on a novel SoftCuts metric by generalizing the Geocuts method . The proposed soft edge smoothness measure can approximate the average length of all level lines in an intensity image. Thus, the total length of all level lines can be minimized effectively by integrating this new form of prior. In addition, this paper presents a novel combination of this soft edge smoothness prior and the alpha matting technique for color image SR, by adaptively normalizing image edges according to their alpha-channel description. This leads to the adaptive SoftCuts algorithm, which represents a unified treatment of edges with different contrasts and scales. Experimental results are presented which demonstrate the effectiveness of the proposed method. <s> BIB022 </s> Super-resolution: a comprehensive survey <s> Color images <s> We present a variational approach to obtain high-resolution images from multiframe low-resolution video stills. The objective functional for the variational approach consists of a data fidelity term and a regularizer. The fidelity term is formed by adaptively mimicking l1 and l2 norms. The regularization uses the l1 norm of the framelet coefficients of a high-resolution image with a geometric tight framelet system constructed in this paper. The tight framelet system has abilities to detect multi-orientation and multi-order variations of an image. A two-phase iterative method for super-resolution reconstruction is proposed to construct a high-resolution image. The first phase is to get an approximation of the solution (i.e., the ideal image) using the steepest descent method. The second phase is to enhance the sparsity of the approximate solution by using the soft thresholding operator with variable thresholding parameters. Numerical results based on both synthetic data and real videos show that our algorithm is efficient in terms of removing visual artifacts and preserving edges in restored images. <s> BIB023 </s> Super-resolution: a comprehensive survey <s> Color images <s> In this paper, we propose a fusion technique based on framelets to obtain super resolution image from sub-pixel shifted, noisy, blurred low resolution images. This method has high advantages over all existing methods. A Tight frame filter bank provides symmetry and has a redundancy that allows for approximate shift invariance which leads to clear edges, high spatial information with effective denoising which was lacked in critically sampled discrete wavelet transform. They are also shorter and results in smoother scaling and wavelet functions. The reconstructed super resolution image obtained by this technique has high peak signal to noise ratio (PSNR) and low mean square error (MSE) than that obtained by wavelet based fusion method, which is evident through the experimental results. <s> BIB024 </s> Super-resolution: a comprehensive survey <s> Color images <s> In image reconstruction, dynamic super resolutionimage reconstruction algorithms have been investigated toenhance video frames sequentially, where explicit motionestimation is considered as a major factor in theperformance. This paper proposes a novel measurementvalidation method to attain robust image reconstructionresults under inaccurate motion estimation. In addition, wepresent an effective scene change detection methoddedicated to the proposed super resolution technique forminimizing erroneous results when abrupt scene changesoccur in the video frames. Representative experimentalresults show excellent performance of the proposedalgorithm in terms of the reconstruction quality andprocessing speed. <s> BIB025 </s> Super-resolution: a comprehensive survey <s> Color images <s> This paper introduces a new procedure to handle color in single image super resolution (SR). Most existing SR techniques focus primarily on enforcing image priors or synthesizing image details; less attention is paid to the final color assignment. As a result, many existing SR techniques exhibit some form of color aberration in the final upsampled image. In this paper, we outline a procedure based on image colorization and back-projection to perform color assignment guided by the super-resolution luminance channel. We have found that our procedure produces better results both quantitatively and qualitatively than existing approaches. In addition, our approach is generic and can be incorporated into any existing SR techniques. <s> BIB026 </s> Super-resolution: a comprehensive survey <s> Color images <s> This paper proposes a context-constrained hallucination approach for image super-resolution. Through building a training set of high-resolution/low-resolution image segment pairs, the high-resolution pixel is hallucinated from its texturally similar segments which are retrieved from the training set by texture similarity. Given the discrete hallucinated examples, a continuous energy function is designed to enforce the fidelity of high-resolution image to low-resolution input and the constraints imposed by the hallucinated examples and the edge smoothness prior. The reconstructed high-resolution image is sharp with minimal artifacts both along the edges and in the textural regions. <s> BIB027 </s> Super-resolution: a comprehensive survey <s> Color images <s> Feeding low-resolution and low-quality images, from inexpensive surveillance cameras, to systems like, e.g., face recognition, produces erroneous and unstable results. Therefore, there is a need for a mechanism to bridge the gap between on one hand low-resolution and low-quality images and on the other hand facial analysis systems. The proposed system in this paper deals with exactly this problem. Our approach is to apply a reconstruction-based super-resolution algorithm. Such an algorithm, however, has two main problems: first, it requires relatively similar images with not too much noise and second is that its improvement factor is limited by a factor close to two. To deal with the first problem we introduce a three-step approach, which produces a face-log containing images of similar frontal faces of the highest possible quality. To deal with the second problem, limited improvement factor, we use a learning-based super-resolution algorithm applied to the result of the reconstruction-based part to improve the quality by another factor of two. This results in an improvement factor of four for the entire system. The proposed system has been tested on 122 low-resolution sequences from two different databases. The experimental results show that the proposed system can indeed produce a high-resolution and good quality frontal face image from low-resolution video sequences. <s> BIB028 </s> Super-resolution: a comprehensive survey <s> Color images <s> As digital video cameras spread rapidly, critical locations such as airports, train stations, military compounds and airbases and public ''hot-spots'' are placed in the spotlight of video surveillance. Video surveillance provides visual information, in order to maintain security at the monitored areas. Indeed, super-resolution (SR) technique can be useful for extracting additional information from the captured video sequences. Generally, video surveillance cameras capture moving objects while cameras are moving in some cases, which means that many of the existing SR algorithms, that cannot cope with moving objects, are not applicable in this case. In this paper, we propose a SR algorithm that takes into account inaccurate registration at the moving regions and therefore copes with moving objects. We propose to adaptively weight each region according to its reliability where regions that have local motion and/or occlusion have different registration error level. Also, the regularization parameter is simultaneously estimated for each region. The regions are generated by segmenting the reference frame using watershed segmentation. Our approach is tested on simulated and real data coming from videos with different difficulties taken by a hand-held camera. The experimental results show the effectiveness of the proposed algorithm compared to four state-of-the-art SR algorithms. <s> BIB029 </s> Super-resolution: a comprehensive survey <s> Color images <s> In this paper, we propose a novel generic image prior-gradient profile prior, which implies the prior knowledge of natural image gradients. In this prior, the image gradients are represented by gradient profiles, which are 1-D profiles of gradient magnitudes perpendicular to image structures. We model the gradient profiles by a parametric gradient profile model. Using this model, the prior knowledge of the gradient profiles are learned from a large collection of natural images, which are called gradient profile prior. Based on this prior, we propose a gradient field transformation to constrain the gradient fields of the high resolution image and the enhanced image when performing single image super-resolution and sharpness enhancement. With this simple but very effective approach, we are able to produce state-of-the-art results. The reconstructed high resolution images or the enhanced images are sharp while have rare ringing or jaggy artifacts. <s> BIB030 </s> Super-resolution: a comprehensive survey <s> Color images <s> Nowadays, most of the image contents are recorded in high resolution (HR). Nevertheless, there is still a need of low resolution (LR) content presentation when recapturing is unviable. As high resolution content becomes commonplace (with HDTV penetration in US estimated at 65% for 2010) the viewers expectations on quality and presentation become higher and higher. Thus, in order to meet today?s consumer expectations the LR content quality has to be enhanced before being displayed. Content pre-processing/upscaling methods include interpolation and Super-Resolution Image Reconstruction (SRIR). SRIR hardware implementations are scarce, mainly due to high memory and performance requirements. This is the challenge we address. In this work an efficient super-resolution core implementation with high quality of the super-resolved outcome is presented. In order to provide high outcome quality the implementation is based on state-of-the-art software. The software algorithm is presented and its output quality compared with other state-of-the-art solutions. The results of the comparison show that the software provides superior output quality. When implemented in targeted FPGA device, the system operating frequency was estimated at 109 MHz. This resulted in a performance that allows dynamic 2x super resolution of QCIF YUV 4:2:0p sequences at a frame rate of 25 fps, leading to realtime execution using only on-chip device memory. Post-layout simulations with back-annotated time proved that the hardware implementation is capable of producing the same output quality as the base software. <s> BIB031 </s> Super-resolution: a comprehensive survey <s> Color images <s> Until now, neighbor-embedding-based (NE) algorithms for super-resolution (SR) have carried out two independent processes to synthesize high-resolution (HR) image patches. In the first process, neighbor search is performed using the Euclidean distance metric, and in the second process, the optimal weights are determined by solving a constrained least squares problem. However, the separate processes are not optimal. In this paper, we propose a sparse neighbor selection scheme for SR reconstruction. We first predetermine a larger number of neighbors as potential candidates and develop an extended Robust-SL0 algorithm to simultaneously find the neighbors and to solve the reconstruction weights. Recognizing that the k-nearest neighbor (k-NN) for reconstruction should have similar local geometric structures based on clustering, we employ a local statistical feature, namely histograms of oriented gradients (HoG) of low-resolution (LR) image patches, to perform such clustering. By conveying local structural information of HoG in the synthesis stage, the k-NN of each LR input patch is adaptively chosen from their associated subset, which significantly improves the speed of synthesizing the HR image while preserving the quality of reconstruction. Experimental results suggest that the proposed method can achieve competitive SR quality compared with other state-of-the-art baselines. <s> BIB032 </s> Super-resolution: a comprehensive survey <s> Color images <s> The neighbor-embedding (NE) algorithm for single-image super-resolution (SR) reconstruction assumes that the feature spaces of low-resolution (LR) and high-resolution (HR) patches are locally isometric. However, this is not true for SR because of one-to-many mappings between LR and HR patches. To overcome or at least to reduce the problem for NE-based SR reconstruction, we apply a joint learning technique to train two projection matrices simultaneously and to map the original LR and HR feature spaces onto a unified feature subspace. Subsequently, the k -nearest neighbor selection of the input LR image patches is conducted in the unified feature subspace to estimate the reconstruction weights. To handle a large number of samples, joint learning locally exploits a coupled constraint by linking the LR-HR counterparts together with the K-nearest grouping patch pairs. In order to refine further the initial SR estimate, we impose a global reconstruction constraint on the SR outcome based on the maximum a posteriori framework. Preliminary experiments suggest that the proposed algorithm outperforms NE-related baselines. <s> BIB033 </s> Super-resolution: a comprehensive survey <s> Color images <s> Reconstruction- and example-based super-resolution (SR) methods are promising for restoring a high-resolution (HR) image from low-resolution (LR) image(s). Under large magnification, reconstruction-based methods usually fail to hallucinate visual details while example-based methods sometimes introduce unexpected details. Given a generic LR image, to reconstruct a photo-realistic SR image and to suppress artifacts in the reconstructed SR image, we introduce a multi-scale dictionary to a novel SR method that simultaneously integrates local and non-local priors. The local prior suppresses artifacts by using steering kernel regression to predict the target pixel from a small local area. The non-local prior enriches visual details by taking a weighted average of a large neighborhood as an estimate of the target pixel. Essentially, these two priors are complementary to each other. Experimental results demonstrate that the proposed method can produce high quality SR recovery both quantitatively and perceptually. <s> BIB034 </s> Super-resolution: a comprehensive survey <s> Color images <s> In this letter, we present the first application of the Generalized Gaussian Markov Random Field (GGMRF) to the problem of video super-resolution. The GGMRF prior is employed to perform a maximum a posteriori (MAP) estimation of the desired high-resolution image. Compared with traditional prior models, the GGMRF can describe the distribution of the high-resolution image much better and can also preserve better the discontinuities (edges) of the original image. Previous work that used GGMRF for image restoration in which the temporal dependencies among video frames has not considered. Since the corresponding energy function is convex, gradient descent optimization techniques are used to solve the MAP estimation. Results show the super-resolved images using the GGMRF prior not only offers a good enhancement of visual quality, but also contain a significantly smaller amount of noise. <s> BIB035 </s> Super-resolution: a comprehensive survey <s> Color images <s> This paper discusses resolution enhancement of a set of images with varying exposure durations, having a high combined dynamic range. So far, little has been said in relation to the Human Visual System when it comes to Super-Resolution and High Dynamic Range fusion, unlike the case for traditional Super-Resolution where errors are measured with respect to human perception in the pixel domain. We propose a Super-Resolution method in the L*a*b* domain to bridge that gap and present some image reconstruction results. <s> BIB036
Dealing with color images has been discussed in several SR algorithms. The important factors here are the color space employed and the employment of information from different color channels. The most common color spaces in the reported SR works are shown in Table 11 . For employing the information of different channels, different techniques have been used. For example, in - BIB001 BIB003 BIB004 BIB008  BIB012 BIB013 BIB016 BIB018 BIB019 BIB028 BIB029 BIB030 , the images are first converted to YIQ color space (in BIB005 BIB032 BIB033 BIB034 to YCbCr). Since most of the energy in these representations is concentrated in the luminance component, Y, the SR algorithm is first applied to this component and the color components are simply interpolated afterwards. - BIB023 , the same process has been applied to all the three YCbCr channels of the images. - BIB024 BIB025  BIB035 , the same process has been applied to all the three RGB channels of the images. - BIB002 , the information regarding every two different channels of an RGB image are combined to improve the third one, then at the end, the three improved channels are combined to produce a new improved color image. - BIB006    BIB010 BIB011 BIB017 , inspired by the fact that most of the commercial cameras use only one CCD where every pixel is made sensitive to only one color and the other two color elements of the pixel are found by demosaicing techniques, additional regularization terms for luminance, chrominance, and nonhomogeneity of the edge orientation across the color channels in RGB color space are used. Similar terms have been used for the same purpose in BIB007 but the images are first converted to the YCbCr color space. BIB001 BIB004 BIB008  BIB012 BIB013 BIB016 BIB018 BIB019 BIB028 BIB029 BIB030 YCbCr BIB007 BIB005 BIB032 BIB033 BIB034 RGB BIB002 BIB006    BIB010 BIB011 BIB017 YUV  BIB009 BIB014  BIB020 BIB021 BIB026 BIB027 BIB031 L*a*b BIB036 - BIB015 BIB022 , alpha matting along with soft edge smoothness prior has been used. - BIB014 , the SR algorithm has been applied to the luminance component of the images in YUV color space, then the absolute value of the difference of the luminance component of each pixel in the super-resolved image from its four closest pixels in the LR image is computed as d 1 , d 2 , d 3 , and d 4 . Then, these weights are converted to normalized weights by and α is a partitive number which defines the penalty for a departure from luminance. Finally, the obtained w i s are used to linearly combine the color channels of the four neighboring pixels to obtain the color components of the super-resolved pixel.