Super-resolution: a comprehensive survey <s> Bilateral total variation (BTV) <s> Theoretical and practical limitations usually constrain the achievable resolution of any imaging device. Super-Resolution (SR) methods are developed through the years to go beyond this limit by acquiring and fusing several low-resolution (LR) images of the same scene, producing a high-resolution (HR) image. The early works on SR, ::: although occasionally mathematically optimal for particular models of data and noise, produced poor results when applied to real images. In this paper, we discuss two of the main issues related to designing a practical SR system, namely reconstruction accuracy and computational efficiency. Reconstruction accuracy refers to the problem of designing a robust SR method applicable to images from different imaging systems. We study a general framework for optimal reconstruction of images from grayscale, color, or color filtered (CFA) cameras. The performance of our proposed method is boosted by using powerful priors and is robust to both measurement (e.g. CCD read out noise) and system noise (e.g. motion estimation error). Noting that the motion estimation is often considered a bottleneck in terms of SR performance, we introduce the concept of "constrained motions" for enhancing the quality of super-resolved images. We show that using such constraints will enhance the quality of the motion estimation and therefore results in more accurate reconstruction of the HR images. We also justify some practical assumptions that greatly reduce the computational complexity and memory requirements of the proposed methods. We use efficient approximation of the Kalman Filter (KF) and adopt a dynamic point of view to the SR problem. Novel methods for addressing these issues are accompanied by experimental results on real data. <s> BIB001 </s> Super-resolution: a comprehensive survey <s> Bilateral total variation (BTV) <s> This paper explains and implements our efficient multi-frame super-resolution mosaicking algorithm. In this algorithm, feature points between images are matched using SIFT, and then random M-least squares is used to estimate the homography between frames. Next, separate frames are registered and the overlapping region is extracted. A generative model is then adopted and combined with maximum a posteriori estimation to construct the underdetermined sparse linear system. To solve the ill-posed large-scale inverse system, we derive and implement a new hybrid regularization (bilateral total variance Hubert) method. Cross validation is utilized to estimate the derivative of the blur kernel as well as the regularization parameter. Super-resolution is then applied to the individual sub-frames from the overlapping region. Finally, multi-band blending is used to stitch these resolution-enhanced frames to form the final image. The whole process is semi-real time (roughly 30 seconds for 35 frames) and the effectiveness of our algorithm is validated by applying it to real and synthetic UAV video frames. <s> BIB002 </s> Super-resolution: a comprehensive survey <s> Bilateral total variation (BTV) <s> In a typical superresolution algorithm, fusion error modeling, including registration error and additive noise, has a great influence on the performance of the super-resolution algorithms. In this letter, we show that the quality of the reconstructed high-resolution image can be increased by exploiting proper model for the fusion error. To properly model the fusion error, we propose to minimize a cost function that consists of locally and adaptively weighted L1- and L2-norms considering the error model. Binary weights are used so as to adaptively select L1- or L2-norm, based on the local errors. Simulation results demonstrate that proposed algorithm can overcome disadvantages of using either L1- or L2-norm. <s> BIB003 </s> Super-resolution: a comprehensive survey <s> Bilateral total variation (BTV) <s> A hybrid error model with L 1 and L 2 norm minimization criteria is proposed in this paper for image/video super-resolution. A membership function is defined to adaptively control the tradeoff between the L 1 and L 2 norm terms. Therefore, the proposed hybrid model can have the advantages of both L 1 norm minimization (i.e. edge preservation) and L 2 norm minimization (i.e. smoothing noise). In addition, an effective convergence criterion is proposed, which is able to terminate the iterative L 1 and L 2 norm minimization process efficiently. Experimental results on images corrupted with various types of noises demonstrate the robustness of the proposed algorithm and its superiority to representative algorithms. <s> BIB004 </s> Super-resolution: a comprehensive survey <s> Bilateral total variation (BTV) <s> Human faces in surveillance video images usually have low resolution and poor quality. They need to be reconstructed in super-resolution for identification. The traditional subspace-based face super-resolution algorithms are sensitive to light. For solving the problem, this paper proposes a face super-resolution method based on illumination invariant feature. The method firstly extracts the illumination invariant features of an input low resolution image by using adaptive L1–L2 total variation model and self-quotient image in logarithmic domain. Then it projects the feature onto non-negative basis obtained by Nonnegative Matrix Factorization(NMF) in face image database. Finally it reconstructs the high resolution face images under the framework of Maximum A Posteriori (MAP) probability. Experimental results demonstrate that the proposed method outperforms the compared methods both in subjective and objective quality under poor light conditions. <s> BIB005 </s> Super-resolution: a comprehensive survey <s> Bilateral total variation (BTV) <s> Although multi-frame super resolution has been extensively studied in past decades, super resolving real-world video sequences still remains challenging. In existing systems, either the motion models are oversimplified, or important factors such as blur kernel and noise level are assumed to be known. Such models cannot deal with the scene and imaging conditions that vary from one sequence to another. In this paper, we propose a Bayesian approach to adaptive video super resolution via simultaneously estimating underlying motion, blur kernel and noise level while reconstructing the original high-res frames. As a result, our system not only produces very promising super resolution results that outperform the state of the art, but also adapts to a variety of noise levels and blur kernels. Theoretical analysis of the relationship between blur kernel, noise level and frequency-wise reconstruction rate is also provided, consistent with our experimental results. <s> BIB006 </s> Super-resolution: a comprehensive survey <s> Bilateral total variation (BTV) <s> As digital video cameras spread rapidly, critical locations such as airports, train stations, military compounds and airbases and public ''hot-spots'' are placed in the spotlight of video surveillance. Video surveillance provides visual information, in order to maintain security at the monitored areas. Indeed, super-resolution (SR) technique can be useful for extracting additional information from the captured video sequences. Generally, video surveillance cameras capture moving objects while cameras are moving in some cases, which means that many of the existing SR algorithms, that cannot cope with moving objects, are not applicable in this case. In this paper, we propose a SR algorithm that takes into account inaccurate registration at the moving regions and therefore copes with moving objects. We propose to adaptively weight each region according to its reliability where regions that have local motion and/or occlusion have different registration error level. Also, the regularization parameter is simultaneously estimated for each region. The regions are generated by segmenting the reference frame using watershed segmentation. Our approach is tested on simulated and real data coming from videos with different difficulties taken by a hand-held camera. The experimental results show the effectiveness of the proposed algorithm compared to four state-of-the-art SR algorithms. <s> BIB007
where S k x and S l y shift f by k and l pixels in the x and y directions to present several scales of derivatives, 0 < α < 1 imposes a spatial decay on the results  , and P is the scale at which the derivatives are computed (so it computes the derivatives in multiple scales of resolution BIB001 ). It is discussed in BIB002 that this a priori term generates saturated data if it is applied to unmanned aerial vehicle data. Therefore, it has been suggested to combine it with the Hubert function, resulting in the following bilateral total variation Hubert: where A is the BTV regularization term as in Eq. (51) and α is obtained by α = median[|A − median|A||]. It is discussed in BIB002 that this term keeps the smoothness of the continuous regions and preserves edges in discontinuous regions. In  a locally adaptive version of BTV, LABTV, has been introduced to provide a balance between the suppression of noise and the preservation of image details  . To do so, instead of the l 1 norm of Eq. (51), an l p norm has been used, where p for every pixel is defined based on the difference between the pixel and its surroundings. In smooth regions where noise reduction is important, p is set to large values close to two and in non-smooth regions where edge preservation is important, p is set to small values close to one. The same idea of adaptive norms, but using different methods for obtaining the weights, has been employed in BIB003 BIB004 BIB005 BIB006  BIB007  .