Recently, convolutional neural networks (CNN) have been successfully applied to many remote sensing problems. However, deep learning techniques for multi-image super-resolution from multitemporal unregistered imagery have received little attention so far. This work proposes a novel CNN-based technique that exploits both spatial and temporal correlations to combine multiple images. This novel framework integrates the spatial registration task directly inside the CNN, and allows to exploit the representation learning capabilities of the network to enhance registration accuracy. The entire super-resolution process relies on a single CNN with three main stages: shared 2D convolutions to extract high-dimensional features from the input images; a subnetwork proposing registration filters derived from the high-dimensional feature representations; 3D convolutions for slow fusion of the features from multiple images. The whole network can be trained end-to-end to recover a single high resolution image from multiple unregistered low resolution images. The method presented in this paper is the winner of the PROBA-V super-resolution challenge issued by the European Space Agency.
Concerning MISR, the first work was proposed by Tsai and Huang @cite_26 , who used a frequency-domain technique to combine multiple under-sampled images with sub-pixel displacements to improve the spatial resolution of Landsat TM acquisitions. Due to the drawbacks of the frequency-domain algorithms, like the difficulty to incorporate the prior information about HR images, many spatial-domain MISR techniques were proposed over the years @cite_38 . Typical spatial-domain methods include non-uniform interpolation @cite_29 , iterative back-projection (IBP) @cite_1 , projection onto convex sets (POCS) @cite_41 @cite_61 , regularized methods @cite_4 @cite_67 @cite_21 , and sparse coding @cite_27 @cite_17 .
@cite_61 @cite_38 @cite_67 @cite_26 @cite_4 @cite_41 @cite_29 @cite_21 @cite_1 @cite_27 @cite_17
2113397463 2124875329 2006262236 2165939075 2135063818 2102621306 2087380704 2590412282
We address the problem of reconstruction of a high-resolution image from a sequence of low-resolution images containing arbitrary relative motion, excluding occlusion effects. We develop a formulation that simultaneously takes into account blurring due to relative sensor-object motion, sensor integration, and additive noise. We propose a POCS-based algorithm for performing the high-resolution reconstruction, and provide experimental results. > This paper addresses the problem of recovering a super-resolved image from a set of warped blurred and decimated versions thereof. Several algorithms have already been proposed for the solution of this general problem. In this paper, we concentrate on a special case where the warps are pure translations, the blur is space invariant and the same for all the images, and the noise is white. We exploit previous results to develop a new highly efficient super-resolution reconstruction algorithm for this case, which separates the treatment into de-blurring and measurements fusion. The fusion part is shown to be a very simple non-iterative algorithm, preserving the optimality of the entire reconstruction process, in the maximum-likelihood sense. Simulations demonstrate the capabilities of the proposed algorithm. In this paper, we make contact with the field of nonparametric statistics and present a development and generalization of tools and results for use in image processing and reconstruction. In particular, we adapt and expand kernel regression ideas for use in image denoising, upscaling, interpolation, fusion, and more. Furthermore, we establish key relationships with some popular existing methods and show how several of these algorithms, including the recently popularized bilateral filter, are special cases of the proposed framework. The resulting algorithms and analyses are amply illustrated with practical examples Super-resolution reconstruction produces one or a set of high-resolution images from a set of low-resolution images. In the last two decades, a variety of super-resolution methods have been proposed. These methods are usually very sensitive to their assumed model of data and noise, which limits their utility. This paper reviews some of these methods and addresses their shortcomings. We propose an alternate approach using L sub 1 norm minimization and robust regularization based on a bilateral prior to deal with different data and noise models. This computationally inexpensive method is robust to errors in motion and blur estimation and results in images with sharp edges. Simulation results confirm the effectiveness of our method and demonstrate its superiority to other super-resolution methods. An algorithm based on spatial tessellation and approximation of each triangle patch in the Delaunay (1934) triangulation (with smoothness constraints) by a bivariate polynomial is advanced to construct a high resolution (HR) high quality image from a set of low resolution (LR) frames. The high resolution algorithm is accompanied by a site-insertion algorithm for update of the initial HR image with the availability of more LR frames till the desired image quality is attained. This algorithm, followed by post filtering, is suitable for real-time image sequence processing because of the fast expected (average) time construction of Delaunay triangulation and the local update feature. In this paper, we propose a super-resolution image reconstruction algorithm to moderate-resolution imaging spectroradiometer (MODIS) remote sensing images. This algorithm consists of two parts: registration and reconstruction. In the registration part, a truncated quadratic cost function is used to exclude the outlier pixels, which strongly deviate from the registration model. Accurate photometric and geometric registration parameters can be obtained simultaneously. In the reconstruction part, the L1 norm data fidelity term is chosen to reduce the effects of inevitable registration error, and a Huber prior is used as regularization to preserve sharp edges in the reconstructed image. In this process, the outliers are excluded again to enhance the robustness of the algorithm. The proposed algorithm has been tested using real MODIS band-4 images, which were captured in different dates. The experimental results and comparative analyses verify the effectiveness of this algorithm. Abstract Image resolution can be improved when the relative displacements in image sequences are known accurately, and some knowledge of the imaging process is available. The proposed approach is similar to back-projection used in tomography. Examples of improved image resolution are given for gray-level and color images, when the unknown image displacements are computed from the image sequence. A number of image super resolution algorithms based on the sparse coding have successfully implemented multi-frame super resolution in recent years. In order to utilize multiple low-resolution observations, both accurate image registration and sparse coding are required. Previous study on multi-frame super resolution based on sparse coding firstly apply block matching for image registration, followed by sparse coding to enhance the image resolution. In this paper, these two problems are solved by optimizing a single objective function. The proposed formulation not only has a mathematically interesting structure, called the double sparsity, but also yields comparable or improved numerical performance to conventional methods.