Super-resolution: a comprehensive survey <s> Combined methods <s> Multiframe resolution enhancement ("superresolution") methods are becoming widely studied, but only a few procedures have been developed to work with compressed video, despite the fact that compression is a standard component of most image- and video-processing applications. One of these methods uses quantization-bound information to define convex sets and then employs a technique called "projections onto convex sets" (POCS) to estimate the original image. Another uses a discrete cosine transformation (DCT)-domain Bayesian estimator to enhance resolution in the presence of both quantization and additive noise. The latter approach is also capable of incorporating known source statistics and other reconstruction constraints to impose blocking artifact reduction and edge enhancement as part of the solution. We propose a spatial-domain Bayesian estimator that has advantages over both of these approaches. <s> BIB001 </s> Super-resolution: a comprehensive survey <s> Combined methods <s> We propose a Bayesian approach to image hallucination. Given a generic low resolution image, we hallucinate a high resolution image using a set of training images. Our work is inspired by recent progress on natural image statistics that the priors of image primitives can be well represented by examples. Specifically, primal sketch priors (e.g., edges, ridges and corners) are constructed and used to enhance the quality of the hallucinated high resolution image. Moreover, a contour smoothness constraint enforces consistency of primitives in the hallucinated image by a Markov-chain based inference algorithm. A reconstruction constraint is also applied to further improve the quality of the hallucinated image. Experiments demonstrate that our approach can hallucinate high quality super-resolution images. <s> BIB002 </s> Super-resolution: a comprehensive survey <s> Combined methods <s> Face recognition systems are very sensitive to the quality and resolution of their input face images. This makes such systems unreliable when working with long surveillance video sequences without employing some selection and enhancement algorithms. On the other hand, processing all the frames of such video sequences by any enhancement or even face recognition algorithm is demanding. Thus, there is a need for a mechanism to summarize the input video sequence to a set of key-frames and then applying an enhancement algorithm to this subset. This paper presents a system doing exactly this. The system uses face quality assessment to select the key-frames and a hybrid super-resolution to enhance the face image quality. The suggested system that employs a linear associator face recognizer to evaluate the enhanced results has been tested on real surveillance video sequences and the experimental results show promising results. <s> BIB003 </s> Super-resolution: a comprehensive survey <s> Combined methods <s> Reconstruction-based super-resolution algorithms are widely employed for enhancing the quality of low-resolution face images. However, these algorithms are very sensitive to the registration errors of their input images. The registration errors aggravate when working with face images coming from video sequences. The longer the video the bigger is the registration error (due to the motion of the subject). Furthermore, the improvement factor of these algorithms is limited by factors smaller than two. The proposed system in this paper deals with these two problems. In order to restrict the registration errors of the system a fuzzy-based face quality assessment is employed. To cope with the second problem, a hierarchy of different types of super-resolution algorithms is used to reach an improvement factor of four. The proposed system has been tested using real video sequences of different longs and the experimental results are promising. <s> BIB004 </s> Super-resolution: a comprehensive survey <s> Combined methods <s> This letter presents a new framework for improving the spatial resolution of infrared (IR) images based on the high-resolution visible image information. Edge regions in an IR image, which have a strong correlation with the aligned edges in the visible image, are sharpened by using their high frequency patches, which are locally estimated from the visible image. The estimation is performed on the basis of intensity correlations between two images. In addition, in order to improve the resolution in the uncorrelated edge regions and the texture regions where visible image information is not available, we adopt learning-based and reconstruction-based super resolution algorithms, respectively. Experimental results demonstrate that the proposed algorithm improves the spatial resolution compared with the existing upsampling algorithms. <s> BIB005 </s> Super-resolution: a comprehensive survey <s> Combined methods <s> Feeding low-resolution and low-quality images, from inexpensive surveillance cameras, to systems like, e.g., face recognition, produces erroneous and unstable results. Therefore, there is a need for a mechanism to bridge the gap between on one hand low-resolution and low-quality images and on the other hand facial analysis systems. The proposed system in this paper deals with exactly this problem. Our approach is to apply a reconstruction-based super-resolution algorithm. Such an algorithm, however, has two main problems: first, it requires relatively similar images with not too much noise and second is that its improvement factor is limited by a factor close to two. To deal with the first problem we introduce a three-step approach, which produces a face-log containing images of similar frontal faces of the highest possible quality. To deal with the second problem, limited improvement factor, we use a learning-based super-resolution algorithm applied to the result of the reconstruction-based part to improve the quality by another factor of two. This results in an improvement factor of four for the entire system. The proposed system has been tested on 122 low-resolution sequences from two different databases. The experimental results show that the proposed system can indeed produce a high-resolution and good quality frontal face image from low-resolution video sequences. <s> BIB006 </s> Super-resolution: a comprehensive survey <s> Combined methods <s> This paper presents a video super-resolution algorithm to interpolate an arbitrary frame in a low resolution video sequence from sparsely existing high resolution key-frames. First, a hierarchical block-based motion estimation is performed between an input and low resolution key-frames. If the motion-compensated error is small, then an input low resolution patch is temporally super-resolved via bi-directional overlapped block motion compensation. Otherwise, the input patch is spatially super-resolved using the dictionary that has been already learned from the low resolution and its corresponding high resolution key-frame pair. Finally, possible blocking artifacts between temporally super-resolved patches and spatially super-resolved patches are concealed using a specific de-blocking filter. The experimental results show that the proposed algorithm provides significantly better subjective visual quality as well as higher peak-to-peak signal-to-noise ratio than those by previous interpolation algorithms. <s> BIB007 </s> Super-resolution: a comprehensive survey <s> Combined methods <s> This paper proposes a new algorithm to perform single-frame image super-resolution (SR) of vehicle license plate (VLP) using soft learning prior. Conventional single-frame SR/interpolation methods such as bi-cubic interpolation often experience over-smoothing near the edges and textured regions. Therefore, learning-based methods have been proposed to handle these shortcomings by incorporating a learning term so that the reconstructed high-resolution images can be guided towards these models. However, existing learning-based methods employ a binary hard-decision approach to determine whether the prior models are fully relevant or totally irrelevant. This approach, however, is inconsistent with many practical applications as the degree of relevance for the prior models may vary. In view of this, this paper proposes a new framework that adopts a soft learning approach in license plate super-resolution. The method integrates image SR with optical character recognition (OCR) to perform VLP SR. The importance of the prior models is estimated through relevance scores obtained from the OCR. These are then incorporated as a soft learning term into a new regularized cost function. Experimental results show that the proposed method is effective in handling license plate SR in both simulated and real experiments. <s> BIB008
The above-mentioned methods for solving the SR problem have often been combined with each other, resulting in new groups of algorithms. Examples of such combinations can be found in -[31,38], where ML and POCS are combined, which allows incorporating non-linear a priori knowledge into the process. - BIB001  , MAP and POCS are combined and applied to compressed video. - BIB002 , MAP and IBP are combined. - [423, BIB003 BIB004  BIB005 BIB006 BIB007 BIB008 , where a reconstructionbased SR is followed by a learning-based SR.