The spatial resolution of an image is fundamentally diffraction-limited by the wavelength of imaging light (or other modality, such as ultrasound or particle beam). In practice, in fact, the resolution of optical systems is poorer yet, and is typically specified in terms of a point spread function (PSF), which represents the impulse response of an imaging system to a point source. In full generality, the PSF is a complex function, i.e., encompassing both the phase and amplitude of the response. The PSF, generally, may vary with the position of the point within the field of view of an imaging system with respect to which the response is being characterized. An image, as obtained by an imaging system, may be represented as the actual source scene convolved with the PSF over the field of view.
Super-resolution (SR) refers to a class of techniques that provide for exceeding the resolution limit imposed by a particular imaging system, or, even by the physical diffraction limit. Super-resolution has extended the information capture abilities of imaging systems. With the increasing availability of cheaper, more computationally powerful processors, super-resolution algorithms are becoming more common. In order to achieve super-resolution, more information must be employed: either constraints on the geometry of the imaged scene, or else a multiplicity of images derived, for example, during the course of relative motion of the imaging system with respect to the imaged scene. Various SR techniques known in the art are described in Chaudhuri (ed.), Super-Resolution Imaging, (Springer 2001), and in Bannore, Iterative-Interpolation Super-Resolution Image Reconstruction (Springer, 2009), both of which are incorporated herein by reference.
Super-resolution has an inherent limitation, however: for an observed low-resolution image, there are many possible high-resolution images that the camera could have blurred to generate the observed low-resolution image. Super-resolution is an ill-posed inverse problem, a class of computationally challenging inference problems. All existing SR techniques that reconstruct a SR image from multiple images, derived, for example, in the course of motion, do so by performing a “blind” deconvolution, which is to say that the PSF is modeled at the same time that the image is being sharpened. Blind deconvolution concurrently approximates both the unknown high-resolution image and the blurring process, and is described, for example, in Takeda et al., “Removing Motion Blur with Space-Time Processing,” IEEE Trans. Image Proc. (in press, 2010, available at http://users.soe.ucsc.edu/˜milanfar/publications/journal/MotionDeblurringRevised.pdf), which is incorporated herein by reference.
Although existing super-resolution algorithms can perform well in laboratory conditions with very well described blurring conditions, real-world applications of super-resolution have lagged because of the complexity of real blurring and degradation conditions. Since the “inverse problem” is ill-posed and its solution is notoriously an unstable, iterative process, an efficient and stable technique for obtaining a super-resolved image from motion data is, thus, highly desirable.
Other super-resolution algorithms, such as the Pixon method, widely used in infrared astronomy and described in U.S. Pat. No. 5,912,993, require expansion of multi-resolution data in terms of a set of generalized image cells.
Several registration systems have been built that use motion data to achieve super-resolution imaging, however all of the prior art systems use external measures of camera position, such as accelerometers and tilt meters. These external measures suffer from drift problems, in which small errors are compounded with more camera motion. Indeed, the external measures of camera position are rarely as precise as a pixel size in high-resolution cameras. Therefore it would be desirable for there to be an efficient, stable and robust technique for performing image registration directly from the images themselves.