Super-resolution: a comprehensive survey <s> Simultaneous methods <s> The author derives an estimate-maximise (EM) formulation of a Bayesian super-resolution algorithm for reconstructing scattering cross sections from coherent images. He generalises this result to obtain an autofocus/super-resolution method, which simultaneously autofocusses an imaging system and super-resolves its image data. Autofocus/super-resolution might be applied to the interpretation of airborne synthetic aperture radar images that are subject to defocussing effects. > <s> BIB001 </s> Super-resolution: a comprehensive survey <s> Simultaneous methods <s> We present a maximum a posteriori solution to the problem of obtaining a high-resolution image from a set of degraded low-resolution images of the same scene. The proposed algorithm has the advantage that no prior knowledge of blurring functions is required and it can handle unknown misregistrations between the input images. An efficient implementation scheme of alternating minimizations is presented together with experiments that demonstrate the performance of the algorithm. <s> BIB002 </s> Super-resolution: a comprehensive survey <s> Simultaneous methods <s> In many image restoration/resolution enhancement applications, the blurring process, i.e., point spread function (PSF) of the imaging system, is not known or is known only to within a set of parameters. We estimate these PSF parameters for this ill-posed class of inverse problem from raw data, along with the regularization parameters required to stabilize the solution, using the generalized cross-validation method (GCV). We propose efficient approximation techniques based on the Lanczos algorithm and Gauss quadrature theory, reducing the computational complexity of the GCV. Data-driven PSF and regularization parameter estimation experiments with synthetic and real image sequences are presented to demonstrate the effectiveness and robustness of our method. <s> BIB003 </s> Super-resolution: a comprehensive survey <s> Simultaneous methods <s> In this chapter the problem of reconstructing a high resolution image from multiple aliased and shifted by sub-pixel shifts low resolution images is considered. The low resolution images are possibly degraded by unknown blurs and their sub-pixel shifts are not known. This problem is described in the frequency and spatial domains. Algorithms for providing solutions to it are reviewed. In addition, two approaches are presented in detail for solving this low-to-high resolution problem. In the first of these two approaches registration and restoration is performed simultaneously using the expectation-maximization (EM) algorithm. The high resolution image is then reconstructed using regularized interpolation which is performed as a separate step. For this reason this approach is abbreviated as RR-I which corresponds to registration/restorationinterpolation. In the second of these approaches registration, restoration and interpolation are perfomed simultaneously using the EM algorithm. Therefore this approach is abbreviated as RRI which corresponds to registration/restoration/interpolation. Numerical experiments are presented that demonstrate the effectiveness of the two approaches. <s> BIB004 </s> Super-resolution: a comprehensive survey <s> Simultaneous methods <s> Super-resolution image reconstruction produces a high-resolution image from a set of shifted, blurred, and decimated versions thereof. Previously published papers have not addressed the computational complexity of this ill-conditioned large scale problem adequately. In this paper, the computational complexity of MAP-based multiframe super-resolution algorithms is studied, and a new fast algorithm, as well as methods for parallel image reconstruction is also presented. The proposed fast algorithm splits the multiple input low-resolution images into several subsets according to their translation relations, and then applies normal MAP algorithm to each subset, the reconstructed images are processed subsequently at a successive level until the desired resolution is achieved. Experiment results are also provided to demonstrate the efficiency of the proposed techniques. <s> BIB005 </s> Super-resolution: a comprehensive survey <s> Simultaneous methods <s> In traditional digital image restoration, the blurring process of the optic is assumed known. Many previous research efforts have been trying to reconstruct the degraded image or video sequence with either partially known or totally unknown point spread function (PSF) of the optical lens, which is commonly called the blind deconvolution problem. Many methods have been proposed in the application to image restoration. However, little work has been done in the super-resolution scenario. In this paper, we propose a generalized framework of regularized image/video iterative blind deconvolution / super-resolution (IBD-SR) algorithm, using some information from the more matured blind deconvolution techniques from image restoration. The initial estimates for the image restoration help the iterative image/video super-resolution algorithm converge faster and be stable. Experimental results are presented and conclusions are drawn. <s> BIB006 </s> Super-resolution: a comprehensive survey <s> Simultaneous methods <s> The process of combining, via mathematical software tools, a set of low-resolution images into a single high-resolution image is often referred to as super-resolution. Algorithms for super-resolution involve two key steps: registration and reconstruction. Most approaches proposed in the literature decouple these steps, solving each independently. This can be effective if there are very simple, linear displacements between the low-resolution images. However, for more complex, nonlinear, nonuniform transformations, estimating the displacements can be very difficult, leading to severe inaccuracies in the reconstructed high-resolution image. This paper presents a mathematical framework and optimization algorithms that can be used to jointly estimate these quantities. Efficient implementation details are considered, and numerical experiments are provided to illustrate the effectiveness of our approach. <s> BIB007 </s> Super-resolution: a comprehensive survey <s> Simultaneous methods <s> This paper proposes a new algorithm to address blind image super-resolution by fusing multiple blurred low-resolution (LR) images to render a high-resolution (HR) image. Conventional super-resolution (SR) image reconstruction algorithms assume either the blurring during the image formation process is negligible or the blurring function is known a priori. This assumption, however, is impractical as it is difficult to eliminate blurring completely in some applications or characterize the blurring function fully. In view of this, we present a new maximum a posteriori (MAP) estimation framework that performs joint blur identification and HR image reconstruction. An iterative scheme based on alternating minimization is developed to estimate the blur and HR image progressively. A blur prior that incorporates the soft parametric blur information and smoothness constraint is introduced in the proposed method. Experimental results show that the new method is effective in performing blind SR image reconstruction where there is limited information about the blurring function. <s> BIB008 </s> Super-resolution: a comprehensive survey <s> Simultaneous methods <s> We investigate new ways of improving the performance of Bayesian-based super-resolution image reconstruction by using a discontinuity adaptive image prior distribution based on robust statistics and a fast and efficient way of initializing the optimization process. The latter is an adapted normalized convolution (NC) technique that incorporates the uncertainty induced by registration errors. We present both qualitative and quantitative results on real video sequences and demonstrate the advantages of the proposed method compared to conventional methodologies. <s> BIB009 </s> Super-resolution: a comprehensive survey <s> Simultaneous methods <s> This work presents a new class of algorithms for super-resolution (SR) of image sequences. This class of algorithms estimates simultaneously all frames of a sequence by employing an iterative minimization of a regularized cost function. Similarly to other SR techniques, the proposed approach exploits the correlation among the frames of the sequence. This correlated information helps to improve the resolution of the captured images. By employing the motion information only in the prior term of the cost function, the proposed method achieves a better fidelity and more robust performance, when compared to other methods. This original approach provides a fidelity equivalent to the obtained by the simultaneous methods, while achieving a lower computational complexity. The isolation of the motion equations in the prior term allows an increased control over the motion errors, and as a consequence, provides an improved robustness. The performance of the proposed method is discussed and compared with other methods in the literature. In the comparative experiments, it is considered both the minimization over the Euclidean norm, which is employed to achieve low computational complexity, and the minimization over the Huber norm or norm, which is used to produce images with sharp edges and higher robustness to large motion errors. <s> BIB010 </s> Super-resolution: a comprehensive survey <s> Simultaneous methods <s> We present a novel method of Bayesian image super-resolution in which marginalization is carried out over latent parameters such as geometric and photometric registration and the image point-spread function. Related Bayesian super-resolution approaches marginalize over the high-resolution image, necessitating the use of an unfavourable image prior, whereas our method allows for more realistic image prior distributions, and reduces the dimension of the integral considerably, removing the main computational bottleneck of algorithms such as Tipping and Bishop's Bayesian image super-resolution. We show results on real and synthetic datasets to illustrate the efficacy of our method. <s> BIB011 </s> Super-resolution: a comprehensive survey <s> Simultaneous methods <s> Super resolution image reconstruction allows the recovery of a high-resolution (HR) image from several low-resolution images that are noisy, blurred, and down sampled. In this paper, we present a joint formulation for a complex super-resolution problem in which the scenes contain multiple independently moving objects. This formulation is built upon the maximum a posteriori (MAP) framework, which judiciously combines motion estimation, segmentation, and super resolution together. A cyclic coordinate descent optimization procedure is used to solve the MAP formulation, in which the motion fields, segmentation fields, and HR images are found in an alternate manner given the two others, respectively. Specifically, the gradient-based methods are employed to solve the HR image and motion fields, and an iterated conditional mode optimization method to obtain the segmentation fields. The proposed algorithm has been tested using a synthetic image sequence, the "Mobile and Calendar" sequence, and the original "Motorcycle and Car" sequence. The experiment results and error analyses verify the efficacy of this algorithm <s> BIB012 </s> Super-resolution: a comprehensive survey <s> Simultaneous methods <s> In this paper we propose a novel algorithm for super resolution based on total variation prior and variational distribution approximations. We formulate the problem using a hierarchical Bayesian model where the reconstructed high resolution image and the model parameters are estimated simultaneously from the low resolution observations. The algorithm resulting from this formulation utilizes variational inference and provides approximations to the posterior distributions of the latent variables. Due to the simultaneous parameter estimation, the algorithm is fully automated so parameter tuning is not required. Experimental results show that the proposed approach outperforms some of the state-of-the-art super resolution algorithms. <s> BIB013 </s> Super-resolution: a comprehensive survey <s> Simultaneous methods <s> Super-resolution reconstruction (SRR) produces a high-resolution image from multiple low-resolution images. Many image SRR algorithms assume that the blurring process, i.e., point spread function (PSF) of the imaging system is known in advance. However, the blurring process is not known or is known only to within a set of parameters in many practical applications. In this letter, we propose an approach for solving the joint blur identification and image SRR based on the principle similar to the variable projection method. The approach can avoid some shortcomings of cyclic coordinate descent optimization procedure. We also propose an efficient implementation based on Lanczos algorithm and Gauss quadrature theory. Experimental results are presented to demonstrate the effectiveness of our method. <s> BIB014 </s> Super-resolution: a comprehensive survey <s> Simultaneous methods <s> Accurate registration of the low resolution (LR) images is a critical step in image super resolution reconstruction (SRR). Conventional algorithms always use invariable motion parameters derived from registration algorithms, and carry on SRR without considering the registration errors in the disjointed method. In this paper we propose a new method that performs joint image registration and SRR based on regularized total least norm (RTLN), updating the motion parameters and HR image simultaneously. Not only translation but also rotation motion are considered, which makes the motion model more universal. Experimental results have shown that our approach is more effective and efficient than traditional ones. <s> BIB015 </s> Super-resolution: a comprehensive survey <s> Simultaneous methods <s> This letter proposes a nonlinear L 1-norm approach for joint image registration and super-resolution (SR). Image SR is the fusion of multiple low-resolution (LR) images to produce a high-resolution (HR) image. Conventional SR algorithms are sensitive to the initial registration error and outliers in the LR images. In view of this, we present a new SR method to address these problems using L 1-norm optimization in joint image registration and HR image reconstruction. Experimental results show that the proposed method is effective in handling these issues in the HR image reconstruction. <s> BIB016 </s> Super-resolution: a comprehensive survey <s> Simultaneous methods <s> Super-resolution reconstruction (SRR) has long been relying on very accurate motion estimation between the frames for a successful process. However, recent works propose SRR that bypasses the need for an explicit motion estimation. In this correspondence, we present a new framework that ultimately leads to the same algorithm as in our prior work. The contribution of this paper is two-fold. First, the suggested approach is much simpler and more intuitive, relying on the classic SRR formulation, and using a probabilistic and crude motion estimation. Second, the new approach offers various extensions not covered in our previous work, such as more general re-sampling tasks (e.g., de-interlacing). <s> BIB017 </s> Super-resolution: a comprehensive survey <s> Simultaneous methods <s> Regularization based super-resolution (SR) methods have been widely used to improve video resolution in recent years. These methods, however, only minimize the sum of difference between acquired low resolution (LR) images and observation model without considering video local structure. In this paper, we proposed an idea, which employs adaptive kernel regression on regularization based SR methods, to improve super-resolution performance. Arbitrary motions in input video are also considered and well modeled in our work. It is shown that the proposed idea can provide better visual quality as well as higher Peak Signal-to-Noise Ratio (PSNR) than approaches using regularized scheme or adaptive kernel regression alone. <s> BIB018 </s> Super-resolution: a comprehensive survey <s> Simultaneous methods <s> In this paper, we address the super resolution (SR) problem from a set of degraded low resolution (LR) images to obtain a high resolution (HR) image. Accurate estimation of the sub-pixel motion between the LR images significantly affects the performance of the reconstructed HR image. In this paper, we propose novel super resolution methods where the HR image and the motion parameters are estimated simultaneously. Utilizing a Bayesian formulation, we model the unknown HR image, the acquisition process, the motion parameters and the unknown model parameters in a stochastic sense. Employing a variational Bayesian analysis, we develop two novel algorithms which jointly estimate the distributions of all unknowns. The proposed framework has the following advantages: 1) Through the incorporation of uncertainty of the estimates, the algorithms prevent the propagation of errors between the estimates of the various unknowns; 2) the algorithms are robust to errors in the estimation of the motion parameters; and 3) using a fully Bayesian formulation, the developed algorithms simultaneously estimate all algorithmic parameters along with the HR image and motion parameters, and therefore they are fully-automated and do not require parameter tuning. We also show that the proposed motion estimation method is a stochastic generalization of the classical Lucas-Kanade registration algorithm. Experimental results demonstrate that the proposed approaches are very effective and compare favorably to state-of-the-art SR algorithms. <s> BIB019 </s> Super-resolution: a comprehensive survey <s> Simultaneous methods <s> In this paper, we propose an energy-based algorithm for motion-compensated video super-resolution (VSR) targeted on upscaling of standard definition (SD) video to high-definition (HD) video. Since the motion (flow field) of the image sequence is generally unknown, we introduce a formulation for the joint estimation of a super-resolution (SR) sequence and its flow field. Via the calculus of variations, this leads to a coupled system of partial differential equations for image sequence and motion estimation. We solve a simplified form of this system and, as a by-product, we indeed provide a motion field for super-resolved sequences. To the best of our knowledge, computing super-resolved flows has not been done before. Most advanced SR methods found in literature cannot be applied to general video with arbitrary scene content and/or arbitrary optical flows, as it is possible with our simultaneous VSR method. A series of experiments shows that our method outperforms other VSR methods when dealing with general video input and that it continues to provide good results even for large scaling factors up to 8 × 8. <s> BIB020 </s> Super-resolution: a comprehensive survey <s> Simultaneous methods <s> We describe a method for automatic determination of the regularization parameters for the class of simultaneous super-resolution (SR) algorithms. This method, proposed in (Zibetti et al., 2008c), is based on the joint maximum a posteriori (JMAP) estimation technique, which is a fast alternative to estimate the parameters. However, the classical JMAP technique can be unstable and may generate multiple local minima. In order to stabilize the JMAP estimation, while achieving a cost function with a unique global solution, we derive an improved solution by modeling the JMAP hyperparameters with a gamma prior distribution. In this work, experimental results are provided to illustrate the effectiveness of the proposed method for automatic determination of the regularization parameters for the simultaneous SR. Moreover, we contrast the proposed method to a reference method with known fixed parameters as well as to other parameter selection methods based on the L-curve. These results validate the proposed method as a very attractive alternative for estimating the regularization parameters. <s> BIB021
The reconstruction of the super-resolved image and the estimation of the involved parameters sometimes have been done simultaneously. For example, in BIB001 41, 44, 46, BIB004 BIB005 BIB007 BIB009   BIB010 BIB013 BIB015 BIB016 BIB018 BIB019 BIB020 BIB021 a MAP method has been used for simultaneous reconstruction of the super-resolved image and registration of the LR images. In these cases, the MAP formulation of Eq. (62) will be, as in [41, 44] : wherein W k can include all the decimation factors or only some of them, as in the blur kernel BIB001 46] and the motion parameters [41, 44] . In other groups of simultaneous SR algorithms, the blurring parameter is estimated at the same time that the reconstruction of the super-resolved image is being carried out. Examples of this group are: - [70] which uses a generalization of Papoulis' sampling theorem and the shifting property between consecutive frames; - [68, BIB003 which use a GCV-based method; - BIB006  BIB008 BIB002 BIB011  BIB014 BIB017 where a MAPbased method has been used. Finally, in a third group of simultaneous SR algorithms, all the involved parameters are estimated simultaneously, as in   BIB012 .