Super-resolution: a comprehensive survey <s> Fourier transform <s> A new view of the problem of continuing a given segment of the spectrum of a finite object is presented. Based on this, the problem is restated in terms of reducing a defined ‘error energy’ which is implicit in the truncated spectrum. A computational procedure, which is readily implemented on general purpose computers, is devised which must reduce this error. It is demonstrated that by so doing, resolution well beyond the diffraction limit is attained. The procedure is shown to be very effective against noisy data. <s> BIB001 </s> Super-resolution: a comprehensive survey <s> Fourier transform <s> A recently proposed iterative method for super-resolution is studied with the aid of prolate spheroidal wave functions. Perfect imagery is shown to be achieved in the limit of an infinite number of iterations. The case of a finite number of iterations is examined and possible extensions of the method are mentioned. <s> BIB002 </s> Super-resolution: a comprehensive survey <s> Fourier transform <s> In several applications it is required to reconstruct a high-resolution noise-free image from multipath frames of undersampled low-resolution noisy images. Using the aliasing relationship between the undersamples frames and the reference image, an algorithm based on weighted recursive least-squares theory is developed in the wavenumber domain. This algorithm is efficient because interpolation and noise removal are performed recursively, and is highly suitable for implementation via the massively parallel computational architectures currently available. Success in the use of the algorithm is demonstrated through various simulated examples. > <s> BIB003 </s> Super-resolution: a comprehensive survey <s> Fourier transform <s> The authors address the problem of reconstruction of a high-resolution image from a number of lower-resolution (possibly noisy) frames of the same scene where the successive frames are uniformly based versions of each other at subpixel displacements. In particular, two previously proposed methods, a frequency-domain method and a method based on projections onto convex sets (POCSs), are extended to take into account the presence of both sensor blurring and observation noise. A new two-step procedure is proposed, and it is shown that the POCS formulation presented for the high-resolution image reconstruction problem can also be used as a new method for the restoration of spatially invariant blurred images. Some simulation results are provided. > <s> BIB004 </s> Super-resolution: a comprehensive survey <s> Fourier transform <s> It is shown how the total least squares recursive algorithm for the real data FIR (finite impulse response) adaptive filtering problem can be applied to reconstruct a high-resolution filtered image from undersampled, noisy multiframes, when the interframe displacements are not accurately known. This is done in the wavenumber domain after transforming the complex data problem to an equivalent real data problem, to which the algorithm developed by C.E. Davila (Proc. ICASSP 1991 p.1853-6 of 1991) applies. The procedure developed also applies when the multiframes are degraded by linear shift-invariant blurs. All the advantages of implementation via massively parallel computational architecture apply. The performance of the algorithm is verified by computer simulations. > <s> BIB005 </s> Super-resolution: a comprehensive survey <s> Fourier transform <s> A noniterative method for superresolution is presented and analyzed. By exploiting the inherent structure of the discrete signal-processing environment, this algorithm reduces the problem of superresolution to that of solving a set of linear equations that relate known frequency samples to unknown time samples. It is shown that the algorithm always produces the correct solution if the known samples are error free and if the system is exactly determined or overdetermined. A technique to reduce the effect of measurement noise on the solution is presented also. It is shown that the noniterative algorithm will produce the same result as the Gerchberg–Papoulis algorithm [ Opt. Acta21, 709 ( 1974); IEEE Trans. Circuits Syst.CAS-22, 735 ( 1975)]. A simple, one-dimensional example is used to demonstrate the performance of both algorithms in the presence, and in the absence, of noise. The utility of the noniterative algorithm is demonstrated further with another one-dimensional example and then a two-dimensional example. <s> BIB006 </s> Super-resolution: a comprehensive survey <s> Fourier transform <s> A theoretical analysis is proposed to evaluate the robustness of the total least square (TLS) algorithm for image reconstruction from a sequence of undersampled noisy and blurred frames in the wavenumber domain. Since the data samples in the wavenumber domain are complex-valued, the results from the TLS theory developed for the case of real-valued data are adapted to the situation at hand. It is shown that the image quality can be improved as more frames become available. In the case of blurred frames, higher resolution images may be reconstructed using the TLS algorithm with post-deblurring. Finally, computer simulation results are provided to demonstrate the robustness of the TLS algorithm for image reconstruction. > <s> BIB007 </s> Super-resolution: a comprehensive survey <s> Fourier transform <s> Abstract It has been known for some time that temporal dependence (motion) plays a key role in the super-resolution (SR) reconstruction of a single frame (or sequence of frames). While the impact of global time-invariant translations is relatively well known, the general motion case has not been studied in detail. In this paper, we discuss SR reconstruction for both motion models from a frequency-domain point of view. A noniterative algorithm for SR reconstruction is presented using spatio-temporal filtering. The concepts of motion-compensated windows and sinc interpolation kernels are utilized, resulting in a finite impulse response (FIR) filter realization. In the simulations, we assume a priori knowledge of the motion (optical flow), which is commonly done throughout much of the SR reconstruction literature. The proposed process is localized in nature, and this enables the selective reconstruction of desired parts of a particular frame or sequence of frames. <s> BIB008 </s> Super-resolution: a comprehensive survey <s> Fourier transform <s> We are concerned with analysis of linear reconstruction-based super-resolution (SR) techniques. We propose a novel Fourier interpretation of an "archetype" linear SR algorithm and derive practical results on the expected performances of such techniques. <s> BIB009 </s> Super-resolution: a comprehensive survey <s> Fourier transform <s> In the present paper, we propose a new magnetic resonance fast scan technique using the phase-scrambling Fourier transform imaging (PSFT), in which the strategy of data acquisition is similar to that of the standard Fourier imaging technique. The Fourier transformed image in the PSFT has a phase modulation in which the local spatial frequency is increased as the position moves away from the center of the image. Therefore, the signal length of the PSFT signal can be expanded using the restraint that the reconstructed image is a real-value object in the super-resolution algorithm. Simulation and experimental studies using a 0.0187T MR scanner were performed for the new improving image resolution method. It was shown that the spatial resolution could be improved in proportion to the distance from the center of the image by the quadratic phase modulation function, irrespective of the amount of noise, and the maximum resolution improvement is 2.0 times, as compared to the standard imaging method. <s> BIB010 </s> Super-resolution: a comprehensive survey <s> Fourier transform <s> This paper addresses the reconstruction of high-resolution omnidirectional images from multiple low-resolution images with inexact registration. When omnidirectional images from low-resolution vision sensors can be uniquely mapped on the 2-sphere, such a reconstruction can be described as a transform-domain super-resolution problem in a spherical imaging framework. We describe how several spherical images with arbitrary rotations in the SO(3) rotation group contribute to the reconstruction of a high-resolution image with help of the spherical Fourier transform (SFT). As low-resolution images might not be perfectly registered in practice, the impact of inaccurate alignment on the transform coefficients is analyzed. We then cast the joint registration and super-resolution problem as a total least-squares norm minimization problem in the SFT domain. A l1-regularized total least-squares problem is considered and solved efficiently by interior point methods. Experiments with synthetic and natural images show that the proposed methods lead to effective reconstruction of high-resolution images even when large registration errors exist in the low-resolution images. The quality of the reconstructed images also increases rapidly with the number of low-resolution images, which demonstrates the benefits of the proposed solution in super-resolution schemes. Finally, we highlight the benefit of the additional regularization constraint that clearly leads to reduced noise and improved reconstruction quality. <s> BIB011
Gerchberg BIB001 and then Santis and Gori BIB002 introduced the first SR algorithms. These were iterative methods in the frequency domain, based on the Fourier transform BIB009 , which could extend the spectrum of a given signal beyond its diffraction limit and therefore increase its resolution. Though these algorithms were later reintroduced in BIB006 in a non-iterative form, based on singular value decomposition (SVD), they did not become as popular as the method of Tsai and Huang  . Tsai and Huang's system  was the first multiple-image SR algorithm in the frequency domain. This algorithm was developed for working on LR images acquired by Landsat 4 satellite. This satellite produces a set of similar but globally translated images, g k , of the same area of the earth, which is a continuous scene, f ; therefore, g k (m, n) = f (x, y), where x = m + Δ m k and y = n + Δ n k . These shifts, or translations, between the LR images were taken into account by the shifting property of the Fourier transformation: where F g k and F f are the co*ntinuous Fourier transforms of the kth LR image and the HR scene, respectively. The LR images are discrete samples of the continuous scene; there- where T m and T n are the sampling periods along the dimensions of the LR image. Thus, the discrete Fourier transform of the LR images, G k , and their continuous Fourier transform, F g k , are related through  BIB008 BIB010 BIB011 : where M and N are the maximum values of the dimensions of the LR images, m and n, respectively. It is assumed that the HR scene is band limited; therefore, putting the shifting property Eq. (15) into Eq. (16) and writing the results in matrix form results in  : in which relates the discrete Fourier transform of the LR images G to the continuous Fourier transform of the HR scene, F f . SR here is therefore reduced to finding F f in Eq. (17) which is usually solved by a least squares (LS) algorithm. The seminal work of Tsai and Huang  assumed ideal noise-free LR images with no blurring effects. Later on, an additive noise BIB003 BIB004 BIB007 68] and blurring effects BIB003 68] were added to Tsai and Huang's method  and Eq. (17) was rearranged as BIB003 BIB004 BIB007 : in which η is a noise term. From this model, Kim et al. BIB003 tried to minimize the following error, E, using an iterative algorithm: whereḞ f is an approximation of F f which minimizes Eq. (19) and † represents conjugate transpose BIB003 . Furthermore, BIB003 incorporated the a priori knowledge about the observed LR images into a recursive weighted least squares algorithm. In this case, Eq. (19) will be altered to: in which A is a diagonal matrix giving the a priori knowledge about the discrete Fourier transform of the available LR observations, G. In this case, those LR images which were known to have a higher signal-to-noise ratio are assigned greater weights. In BIB003 BIB004 it was assumed that the motion information was known beforehand. To reduce the errors of estimating the displacements between the LR images, BIB005 BIB007 512 ] used a recursive total least squares method. In this case, Eq. (17) becomes: where P is a perturbation matrix obtained from the estimation errors BIB005 BIB007 .