A survey of classical methods and new trends in pansharpening of multispectral images <s> Multiresolution Family <s> We describe a technique for image encoding in which local operators of many scales but identical shape serve as the basis functions. The representation differs from established techniques in that the code elements are localized in spatial frequency as well as in space. Pixel-to-pixel correlations are first removed by subtracting a lowpass filtered copy of the image from the image itself. The result is a net data compression since the difference, or error, image has low variance and entropy, and the low-pass filtered image may represented at reduced sample density. Further data compression is achieved by quantizing the difference image. These steps are then repeated to compress the low-pass image. Iteration of the process at appropriately expanded scales generates a pyramid data structure. The encoding process is equivalent to sampling the image with Laplacian operators of many scales. Thus, the code tends to enhance salient image features. A further advantage of the present code is that it is well suited for many image analysis tasks as well as for image compression. Fast algorithms are described for coding and decoding. <s> BIB001 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Multiresolution Family <s> This paper presents a technique for image pyramid generation, in which the reduction (expansion) factor between layers is any rational number M/L. The image pyramid generation is modeled as an interpolation and filtering followed by a decimation. The model enables frequency domain analysis of the image pyramid, as well as convenient design of the generating kernels. L(M) generating kernels are necessary to produce an image pyramid with reduction (expansion) factor M/L(L/M). A polyphase filter network scheme is used where the L(M) generating kernels can be produced by sampling one prototype low-pass filter with cutoff frequency at omega = pi /max(M,L). Using these polyphase filters, the frequency content of pyramid image decompositions can be adjusted with great flexibility. A systematic procedure is presented here for specifying the relative positions of spatial samples in successive pyramid levels-a complication that arises when generalizing from integer reduction factors to rational factors. Two types of low-pass filters are employed in this work for the prototype filter design: a binomial filter and an FIR linear phase filter. Illustrative examples are presented. > <s> BIB002 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Multiresolution Family <s> A new technique is developed for the merging and data fusion of two images. Two spatially registered images with differing spatial resolutions and color content are merged by combining multiresolution wavelet-decomposition components from each and then reconstructing the merged image by means of the inverse wavelet transform. The wavelet merger can employ a variety of wavelet bases, but in presentation of the concept, simple orthonormal sets—Haar and Daubechies wavelets—are explored. The wavelet technique is compared with the intensity–hue–saturation merging technique by means of multispectral and panchromatic test images. The results of the comparison show the wavelet merger performing better in combining and preserving spectral–spatial information for the test images. <s> BIB003 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Multiresolution Family <s> Several techniques have been developed to merge SPOT 10-m resolution panchromatic data (SPOT P) with simultaneously acquired 20-m resolution multispectral data (SPOT XS). Normally, the objective of these procedures is to create a composite image of enhanced interpretability. The current merging methods may not be satisfying; they can distort the spectral characteristics of the xs images and, as a result, the analysis becomes difficult. In this paper a method allowing the use of 10-m resolution xs data while conserving the spectral properties of the original 20-m data is presented. This method uses a multiresolution analysis procedure based upon the wavelets transform; it is applied to remotely sensed SPOT P and SPOT XS images of the river junction Arc-Is&re (France). This new method is compared with the IHS method and P+XS method. The wavelets method is the one which least distorts the spectral characteristics of the data. The distortions are minjmal and difficult to detect. <s> BIB004 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Multiresolution Family <s> The Nature of Remote Sensing: Introduction. Remote Sensing. Information Extraction from Remote-Sensing Images. Spectral Factors in Remote Sensing. Spectral Signatures. Remote-Sensing Systems. Optical Sensors. Temporal Characteristics. Image Display Systems. Data Systems. Summary. Exercises. References. Optical Radiation Models: Introduction. Visible to Short Wave Infrared Region. Solar Radiation. Radiation Components. Surface-Reflected. Unscattered Component. Surface-Reflected. Atmosphere-Scattered Component. Path-Scattered Component. Total At-Sensor. Solar Radiance. Image Examples in the Solar Region. Terrain Shading. Shadowing. Atmospheric Correction. Midwave to Thermal Infrared Region. Thermal Radiation. Radiation Components. Surface-Emitted Component. Surface-Reflected. Atmosphere-Emitted Component. Path-Emitted Component. Total At-Sensor. Emitted Radiance. Total Solar and Thermal Upwelling Radiance. Image Examples in the Thermal Region. Summary. Exercises. References. Sensor Models: Introduction. Overall Sensor Model. Resolution. The Instrument Response. Spatial Resolution. Spectral Resolution. Spectral Response. Spatial Response. Optical PSFopt. Image Motion PSFIM. Detector PSFdet. Electronics PSFel. Net PSFnet. Comparison of Sensor PSFs. PSF Summary for TM. Imaging System Simulation. Amplification. Sampling and Quantization. Simplified Sensor Model. Geometric Distortion. Orbit Models. Platform Attitude Models. Scanner Models. Earth Model. Line and Whiskbroom ScanGeometry. Pushbroom Scan Geometry. Topographic Distortion. Summary. Exercises. References. Data Models: Introduction. A Word on Notation. Univariate Image Statistics. Histogram. Normal Distribution. Cumulative Histogram. Statistical Parameters. Multivariate Image Statistics. Reduction to Univariate Statistics. Noise Models. Statistical Measures of Image Quality. Contrast. Modulation. Signal-to-Noise Ratio (SNR). Noise Equivalent Signal. Spatial Statistics. Visualization of Spatial Covariance. Covariance with Semivariogram. Separability and Anisotropy. Power Spectral Density. Co-occurrence Matrix. Fractal Geometry. Topographic and Sensor Effects. Topography and Spectral Statistics. Sensor Characteristics and Spectral Stastistics. Sensor Characteristics and Spectral Scattergrams. Summary. Exercises. References. Spectral Transforms: Introduction. Feature Space. Multispectral Ratios. Vegetation Indexes. Image Examples. Principal Components. Standardized Principal Components (SPC) Transform. Maximum Noise Fraction (MNF) Transform. Tasseled Cap Tranformation. Contrast Enhancement. Transformations Based on Global Statistics. Linear Transformations. Nonlinear Transformations. Normalization Stretch. Reference Stretch. Thresholding. Adaptive Transformation. Color Image Contrast Enhancement. Min-max Stretch. Normalization Stretch. Decorrelation Stretch. Color Spacer Transformations. Summary. Exercises. References. Spatial Transforms: Introduction. An Image Model for Spatial Filtering. Convolution Filters. Low Pass and High Pass Filters. High Boost Filters. Directional Filters. The Border Region. Characterization of Filtered Images. The Box Filter Algorithm. Cascaded Linear Filters. Statistical Filters. Gradient Filters. Fourier Synthesis. Discrete Fourier Transforms in 2-D. The Fourier Components. Filtering with the Fourier Transform. Transfer Functions. The Power Spectrum. Scale Space Transforms. Image Resolution Pyramids. Zero-Crossing Filters. Laplacian-of-Gaussian (LoG) Filters. Difference-of-Gaussians (DoG) Filters.Wavelet Transforms. Summary. Exercises. References. Correction and Calibration: Introduction. Noise Correction. Global Noise. Sigma Filter. Nagao-Matsuyama Filter. Local Noise. Periodic Noise. Distriping 359. Global,Linear Detector Matching. Nonlinear Detector Matching. Statistical Modification to Linear and Nonlinear Detector. Matching. Spatial Filtering Approaches. Radiometric Calibration. Sensor Calibration. Atmospheric Correction. Solar and Topographic Correction. Image Examples. Calibration and Normalization of Hyperspectral Imagery. AVIRIS Examples. Distortion Correction. Polynomial Distortion Models. Ground Control Points (GCPs). Coordinate Transformation. Map Projections. Resampling. Summary. Exercises References. Registration and Image Fusion: Introduction. What is Registration? Automated GCP Location. Area Correlation. Other Spatial Features. Orthrectification. Low-Resolution DEM. High-Resolution DEM. Hierarchical Warp Stereo. Multi-Image Fusion. Spatial Domain Fusion. High Frequency Modulation. Spectral Domain Fusion. Fusion Image Examples. Summary. Exercises. References. Thematic Classification: Introduction. The Importance of Image Scale. The Notion of Similarity. Hard Versus Soft Classification. Training the Classifier. Supervised Training. Unsupervised Training. K-Means Clustering Algorithm. Clustering Examples. Hybrid Supervised/Unsupervised Training. Non-Parametric Classification Algorithms. Level-Slice. Nearest-Mean. Artificial Neural Networks (ANNs). Back-Propagation Algorithm. Nonparametric Classification Examples. Parametric Classification Algorithms. Estimation of Model-Parameters. Discriminant Functions. The Normal Distribution Model. Relation to the Nearest-Mean Classifier. Supervised Classification Examples and Comparison to Nonparametric Classifiers. Segmentation. Region Growing. Region Labeling. Sub-Pixel Classification. The Linear Mixing Model. Unmixing Model. Hyperspectral Image Analysis. Visualization of the Image Cube. Feature Extraction. Image Residuals. Pre-Classification Processing and Feature Extraction. Classification Algorithms. Exercises. Error Analysis. Multitemporal Images. Summary. References. Index. <s> BIB005 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Multiresolution Family <s> Abstract To take advantage of the high spectral resolution of Landsat TM images and the high spatial resolution of SPOT panchromatic images (SPOT PAN), we present a wavelet transform method to merge the two data types. In a pyramidal fashion, each TM reflective band or SPOT PAN image was decomposed into an orthogonal wavelet representation at a given coarser resolution, which consisted of a low frequency approximation image and a set of high frequency, spatially-oriented detail images. Band-by-band, the merged images were derived by performing an inverse wavelet transform using the approximation image from each TM band and detail images from SPOT PAN. The spectral and spatial features of the merged results of the wavelet methods were compared quantitatively with those of intensity-hue-saturation (IHS), principal component analysis (PCA), and the Brovey transform. It was found that multisensor data merging is a trade-off between the spectral information from a low spatial-high spectral resolution sensor an... <s> BIB006 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Multiresolution Family <s> The standard data fusion methods may not be satisfactory to merge a high-resolution panchromatic image and a low-resolution multispectral image because they can distort the spectral characteristics of the multispectral data. The authors developed a technique, based on multiresolution wavelet decomposition, for the merging and data fusion of such images. The method presented consists of adding the wavelet coefficients of the high-resolution image to the multispectral (low-resolution) data. They have studied several possibilities concluding that the method which produces the best results consists in adding the high order coefficients of the wavelet transform of the panchromatic image to the intensity component (defined as L=(R+G+B)/3) of the multispectral image. The method is, thus, an improvement on standard intensity-hue-saturation (IHS or LHS) mergers. They used the "a trous" algorithm which allows the use of a dyadic wavelet to merge nondyadic data in a simple and efficient scheme. They used the method to merge SPOT and LANDSAT/sup TM/ images. The technique presented is clearly better than the IHS and LHS mergers in preserving both spectral and spatial information. <s> BIB007 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Multiresolution Family <s> This work compares two general and formal solutions to the problem of fusion of multispectral images with high-resolution panchromatic observations. The former exploits the generalized Laplacian pyramid, which is an oversampled structure obtained by subtracting from an image its lowpass version. The latter relies on an undecimated wavelet decomposition, which is another bandpass representation achieved from a conventional octave wavelet transform by omitting all decimators and upsampling the filterbank. Both the methods selectively perform spatial-frequencies spectrum substitution from an image to another. The major novelty is that both the decompositions are not critically-sampled, thus avoiding possible impairments in the fused images, due to missing cancellation of aliasing terms. Quantitative results are presented and discussed on SPOT data (XS+P). <s> BIB008 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Multiresolution Family <s> Intensity hue saturation (IHS) and wavelet decomposition are two distinct fusion methods used for enhancing the spatial resolution of multispectral images by exploiting a high-resolution panchromatic image. In this paper, a combination of the IHS transform and redundant wavelet decomposition is proposed as a general method for fusing multisensor images. The principle consists of transforming low-resolution multispectral images into IHS independent components. The low-resolution intensity component is fused with the high-resolution panchromatic image in the redundant wavelet domain through an appropriate model. Subsequently, the high-resolution intensity produced is substituted to the low-resolution intensity. High spatial resolution multispectral images are then obtained through an inverse IHS transformation. SPOT images are used to illustrate the superiority of this approach over the IHS fuser in terms of preservation of spectral properties. <s> BIB009 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Multiresolution Family <s> This paper deals with multi-spectral (MS) and panchromatic (Pan) image fusion based on redundant multiresolution analysis (MRA). Lower-resolution MS bands are sharpened by injecting high-pass details taken from the higher-resolution Pan image. Crucial point is modelling the relationships between detail coefficients of a generic MS band and the Pan image at the same resolution. Once calculated at the coarser resolution, where both types of data are available, such a model shall be extended to the finer resolution to weight the Pan details to be injected. Two injection models embedded in an “a trous” wavelet decomposition will be described and compared on a test set of very high resolution QuickBird MS+Pan data. One works on approximations and provides a partial unmixing of coarse MS pixels via high-resolution Pan. Another is based on spectral fidelity of the merged image to the original MS data. Fusion comparisons on spatially degraded data, of which higher-resolution true MS data are available for reference, show that the former yields better results than the latter, in terms of both spatial and spectral fidelity. <s> BIB010 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Multiresolution Family <s> Since Chavez proposed the highpass filtering procedure to fuse multispectral and panchromatic images, several fusion methods have been developed based on the same principle: to extract from the panchromatic image spatial detail information to later inject it into the multispectral one. In this paper, we present new fusion alternatives based on the same concept, using the multiresolution wavelet decomposition to execute the detail extraction phase and the intensity-hue-saturation (IHS) and principal component analysis (PCA) procedures to inject the spatial detail of the panchromatic image into the multispectral one. The multiresolution wavelet decomposition has been performed using both decimated and undecimated algorithms and the resulting merged images compared both spectral and spatially. These fusion methods, as well as standard IHS-, PCA-, and wavelet-based methods have been used to merge Systeme Pour l'Observation de la Terre (SPOT) 4 XI and SPOT 4 M images with a ratio 4:1. We have estimated the validity of each fusion method by analyzing, visually and quantitatively, the quality of the resulting fused images. The methodological approaches proposed in this paper result in merged images with improved quality with respect to those obtained by standard IHS, PCA, and standard wavelet-based fusion methods. For both proposed fusion methods, better results are obtained when an undecimated algorithm is used to perform the multiresolution wavelet decomposition. <s> BIB011 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Multiresolution Family <s> This paper addresses the modeling of wavelet coefficients for multispectral (MS) band sharpening based on undecimated multiresolution analysis (MRA). The coarse MS bands are sharpened by injecting highpass details taken from a high-resolution panchromatic (Pan) image. Besides the MRA, crucial point is modeling the relationships between detail coefficients of a generic MS band and the Pan image at the same resolution. Once calculated at the coarser resolution, where both types of data are available, such a model shall be extrapolated to the finer resolution in order to weight the Pan details to be injected. The goal is that the merged MS images are most similar to what the MS sensor would collect if it had the same resolution as the broadband Pan imager. Three injection models embedded in an ''a trous'' wavelet decomposition will be described and compared on a test set of very high-resolution QuickBird MS+Pan data. Two models work on approximation and detail coefficients, respectively, and provide a certain degree of unmixing of coarse MS pixels. The third model is based on spectral fidelity of the merged image to the (resampled) original MS data, that is, no unmixing is attempted. It is much simpler than the other two because it does not require calculation of local statistics. Fusion comparisons on spatially degraded data, of which higher-resolution true MS data are available for reference, show that the former two models yield better results than the latter, in terms of both radiometric and spectral fidelity. The latter, however, yields a reliable sharpening unaffected by local artifacts, regardless of landscape complexity. When local statistics of wavelet coefficients are used, the model estimated on approximation yields slightly better but considerably stabler results than that calculated starting from bandpass details. <s> BIB012 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Multiresolution Family <s> In the last few years, several researchers have proposed different procedures for the fusion of multispectral and panchromatic images based on the wavelet transform, which provide satisfactory high spatial resolution images keeping the spectral properties of the original multispectral data. The discrete approach of the wavelet transform can be performed with different algorithms, Mallat's and the ‘a trous’ being the most popular ones for image fusion purposes. Each algorithm has its particular mathematical properties and leads to different image decompositions. In this article, both algorithms are compared by the analysis of the spectral and spatial quality of the merged images which were obtained by applying several wavelet based, image fusion methods. All these have been used to merge Ikonos multispectral and panchromatic spatially degraded images. Comparison of the fused images is based on spectral and spatial characteristics and it is performed visually and quantitatively using statistical parameters ... <s> BIB013 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Multiresolution Family <s> Usual image fusion methods inject features from a high spatial resolution panchromatic sensor into every low spatial resolution multispectral band trying to preserve spectral signatures and improve spatial resolution to that of the panchromatic sensor. The objective is to obtain the image that would be observed by a sensor with the same spectral response (i.e., spectral sensitivity and quantum efficiency) as the multispectral sensors and the spatial resolution of the panchromatic sensor. But in these methods, features from electromagnetic spectrum regions not covered by multispectral sensors are injected into them, and physical spectral responses of the sensors are not considered during this process. This produces some undesirable effects, such as resolution overinjection images and slightly modified spectral signatures in some features. The authors present a technique which takes into account the physical electromagnetic spectrum responses of sensors during the fusion process, which produces images closer to the image obtained by the ideal sensor than those obtained by usual wavelet-based image fusion methods. This technique is used to define a new wavelet-based image fusion method. <s> BIB014 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Multiresolution Family <s> In this paper, we develop the nonsubsampled contourlet transform (NSCT) and study its applications. The construction proposed in this paper is based on a nonsubsampled pyramid structure and nonsubsampled directional filter banks. The result is a flexible multiscale, multidirection, and shift-invariant image decomposition that can be efficiently implemented via the à trous algorithm. At the core of the proposed scheme is the nonseparable two-channel nonsubsampled filter bank (NSFB). We exploit the less stringent design condition of the NSFB to design filters that lead to a NSCT with better frequency selectivity and regularity when compared to the contourlet transform. We propose a design framework based on the mapping approach, that allows for a fast implementation based on a lifting or ladder structure, and only uses one-dimensional filtering in some cases. In addition, our design ensures that the corresponding frame elements are regular, symmetric, and the frame is close to a tight one. We assess the performance of the NSCT in image denoising and enhancement applications. In both applications the NSCT compares favorably to other existing methods in the literature. <s> BIB015 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Multiresolution Family <s> Image fusion involves merging two or more images in such a way as to retain the most desirable characteristics of each. When a panchromatic image is fused with multispectral imagery, the desired result is an image with the spatial resolution and quality of the panchromatic imagery and the spectral resolution and quality of the multispectral imagery. Standard image fusion methods are often successful at injecting spatial detail into the multispectral imagery but distort the colour information in the process. Over the past decade, a significant amount of research has been conducted concerning the application of wavelet transforms in image fusion. In this paper, an introduction to wavelet transform theory and an overview of image fusion technique are given, and the results from a number of wavelet-based image fusion schemes are compared. It has been found that, in general, wavelet-based schemes perform better than standard schemes, particularly in terms of minimizing colour distortion. Schemes that combine standard methods with wavelet transforms produce superior results than either standard methods or simple wavelet-based methods alone. The results from wavelet-based methods can also be improved by applying more sophisticated models for injecting detail information; however, these schemes often have greater set-up requirements. <s> BIB016 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Multiresolution Family <s> This paper proposes a new fusion method that permits an adequate selection of information extracted from source images to obtain fused images with good spatial and spectral quality simultaneously. This method is based on a joint multiresolution multidirectional representation of the source images using a single directional spatial frequency low pass filter bank of low computational complexity, defined in the Fourier domain. The source images correspond to those captured by the IKONOS satellite (panchromatic and multispectral). The results obtained indicate that the proposed method provides, in a simple manner, objective control over the trade-off between high spatial and spectral quality of the fused images. <s> BIB017 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Multiresolution Family <s> High correlation among the neighboring pixels both spatially and spectrally in a multispectral image makes it necessary to use an efficient data transformation approach before performing pan-sharpening. Wavelets and principal component analysis (PCA) methods have been a popular choice for spatial and spectral transformations, respectively. Current PCA-based pan-sharpening methods make an assumption that the first principal component (PC) of high variance is an ideal choice for replacing or injecting it with high spatial details from the high-resolution histogram-matched panchromatic (PAN) image. This paper presents a combined adaptive PCA-contourlet approach for pan-sharpening, where the adaptive PCA is used to reduce the spectral distortion and the use of nonsubsampled contourlets for spatial transformation in pan-sharpening is incorporated to overcome the limitation of the wavelets in representing the directional information efficiently and capturing intrinsic geometrical structures of the objects. The efficiency of the presented method is tested by performing pan-sharpening of the high-resolution (IKONOS and QuickBird) and the medium-resolution (Landsat-7 Enhanced Thematic Mapper Plus) datasets. The evaluation of the pan-sharpened images using global validation indexes reveal that the adaptive PCA approach helps reducing the spectral distortion, and its merger with contourlets provides better fusion results. <s> BIB018 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Multiresolution Family <s> Image fusion is a process of producing a single image from a set of input images. Recently, the wavelet transform (WT) has been widely used in image fusion. However, the Contourlet transform give better results because it represents edges better than the wavelets transform. In this paper, fusion algorithms based on the contourlet transform are proposed. These algorithms are tested and compare to an existing similar algorithm using Synthesized and QuickBird images. The experimental results show the superiority of the proposed algorithms over the existing contourlet-based one. <s> BIB019 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Multiresolution Family <s> In this work, a technique is presented for the fusion of multi-spectral (MS) and hyperspectral (HS) images to enhance the spatial resolution of the latter. The technique works in the wavelet domain, and is based on a Bayesian estimation of the HS image, assuming a joint normal model for the images, and an additive noise imaging model for the HS image. An appropriate estimation strategy is also proposed. The technique is compared to its counterpart in the spatial domain, and validated for noisy conditions. Further, its performance is compared to several state-of-the-art pansharpening techniques, in the case where the MS image becomes a panchromatic image, and to some MS and HS image fusion techniques from the literature. <s> BIB020 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Multiresolution Family <s> Our framework is the synthesis of multispectral images (MS) at higher spatial resolution, which should be as close as possible to those that would have been acquired by the corresponding sensors if they had this high resolution. This synthesis is performed with the help of a high spatial but low spectral resolution image: the panchromatic (Pan) image. The fusion of the Pan and MS images is classically referred as pan-sharpening. A fused product reaches good quality only if the characteristics and differences between input images are taken into account. Dissimilarities existing between these two data sets originate from two causes-different times and different spectral bands of acquisition. Remote sensing physics should be carefully considered while designing the fusion process. Because of the complexity of physics and the large number of unknowns, authors are led to make assumptions to drive their development. Weaknesses and strengths of each reported method are raised and confronted to these physical constraints. The conclusion of this critical survey of literature is that the choice in the assumptions for the development of a method is crucial, with the risk to drastically weaken fusion performance. It is also shown that the Amelioration de la Resolution Spatiale par Injection de Structures concept prevents from introducing spectral distortion into fused products and offers a reliable framework for further developments. <s> BIB021 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Multiresolution Family <s> This article aims at explaining the ARSIS concept. By fusing two sets of images A and B, one with a high spatial resolution, the other with a low spatial resolution and different spectral bands, the ARSIS concept permits to synthesise the dataset B at the resolution of A that is as close as possible to reality. It is based on the assumption that the missing information is linked to the high frequencies in the sets A and B. It searches a relationship between the high frequencies in the multispectral set B and the set A and models this relationship. The general problem for the synthesis is presented first. The general properties of the fused product are given. Then, the ARSIS concept is discussed. The general scheme for the implementation of a method belonging to this concept is presented. Then, this article intends to help practitioners and researchers to better understand this concept through practical details about implementations. Two Multiscale Models are described as well as two Inter-Band Structure Models (IBSM). They are applied to an Ikonos image as an illustration case. The fused products are assessed by means of a known protocol comprising a series of qualitative and quantitative tests. The products are found of satisfactory quality. This case illustrates the differences existing between the various models, their advantages and limits. Tracks for future improvements are discussed. <s> BIB022 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Multiresolution Family <s> Fusion of multispectral and panchromatic remote sensing images is a procedure to obtain spatial resolution and quality of the panchromatic image as well as preserving spectral information of the multispectral image. In this paper, we present a new fusion method based on HSI (Hue-Saturation-Intensity) and Contourlet transform. First, we convert the multispectral image from the RGB color space into the HSI color space. Then, by applying Contourlet transform to the panchromatic image and the I component of the multispectral image, we utilize an improved fusion rule based on PCA for the low-frequency sub-images, and engage the maximum fusion rule for the high-frequency sub-images. Finally, a fusion image is obtained by the inverse HSI transform. The experimental results show that the proposed fusion method not only enhances the spatial resolution of the fusion image, but also preserves the spectral information of the original multispectral image. <s> BIB023 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Multiresolution Family <s> In this paper, a new fusion rule based on a pulse coupled neural network (PCNN) and the clarity of images is proposed for multi-band synthetic aperture radar (SAR) image fusion. By using a stationary wavelet-based nonsubsampled contourlet transform (SW-NSCT), we can calculate a flexible multiscale, multidirectional, anisotropy and shift-invariant representation of registered SAR images. A weighted fusion rule is performed on the low frequency subbands to calculate the fused lowpass band. For the fusion of high frequency directional subband images, a PCNN model is constructed, where the linking strength of each neuron is determined by the clarity of the decomposed subband images. The fusion approach exploits the advantages of both SW-NSCT in multiscale geometric representations and that of PCNN in the determination of fusion rules; as predicted, the obtained fusion image can preserve much more information regarding textures and edges of the images, compared to its counterparts. Some experiments are performed by comparing the new algorithm with other existing fusion rules and methods. The experimental results show that the proposed fusion approach is effective and can provide better performance in fusing multi-band SAR images than some current methods. <s> BIB024 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Multiresolution Family <s> With the fast development of remote sensor technologies, e.g. the appearance of Very High Resolution (VHR) optical sensors, SAR, LiDAR, etc., mounted on either airborne or spaceborne platforms, multi-source remote sensing data fusion techniques are emerging due to the demand for new methods and algorithms. The general fusion techniques have been well developed and applied in various fields ranging from satellite earth observation to computer vision, medical image processing, defence security and so on. Despite the fast development, the techniques remain challenging for multi-source data fusion within varying spatial and temporal resolutions. This article reviews current techniques of multi-source remote sensing data fusion and discusses their future trends and challenges through the concept of hierarchical classification, i.e., pixel/data level, feature level and decision level. This article concentrates on discussing optical panchromatic and multi-spectral data fusing methods. So far, the pixel level fus... <s> BIB025 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Multiresolution Family <s> Pansharpening is a technique that fuses the information of a low-resolution multispectral image and a high-resolution panchromatic image, usually remote sensing images, to provide a high-resolution multispectral image. In the literature, this task has been addressed from different points of view being one of the most popular wavelet-based algorithms. Recently, a new transform, the contourlet transform, has been proposed. This transform combines the advantages of the wavelets transform, with a more efficient directional information representation. The result is a flexible multiscale, multidirection and shift-invariant decomposition that can be efficiently implemented via the a’trous algorithm. In this chapter, we compare the wavelet-based pansharpening with existing contourlet-based approaches and propose a new pansharpening method based on the contourlet transform. The performance of the contourlet in general, and the proposed method in particular, is assessed numerically and visually for Landsat and SPOT images. <s> BIB026 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Multiresolution Family <s> Pansharpening is a technique that fuses a low resolution multispectral image and a high resolution panchromatic image, to obtain a multispectral image with the spatial resolution and quality of the panchromatic image while preserving spectral information of the multispectral image. In this paper, we present a new pansharpening method based on contourlet transform and Bayesian inference that generalizes the classical contourlet based algorithms. The experimental results show that the proposed method not only enhances the spatial resolution of the pansharpened image, but also preserves the spectral information of the original multispectral image. <s> BIB027
In order to extract or modify the spatial information in remote sensing images, spatial transforms represent also a very interesting tool. Some of these transforms use only local image information (i.e., within a relatively small neighborhood of a given pixel), such as convolution, while others use frequency content, such as the Fourier transform. Beside these two extreme transformations, there is a need for a data representation allowing the access to spatial information over a wide range of scales from local to global BIB005 . This increasingly important category of scale-space filters utilize multiscale decomposition techniques such as Laplacian pyramids BIB001 , wavelet transform  , contourlet transform BIB015 and curvelets transform. These techniques are used in pansharpening to decompose MS and PAN images in different levels in order to derive spatial details that are imported into finer scales of the MS images, highlight the relationship between PAN and MS images in coarser scales and enhance spatial details BIB025 . This is the idea behind the methods based on the successful ARSIS (from French "Amèlioration de la Résolution by Structure Injection") concept  . We will now describe each of the above multiresolution methods and their different types in detail. Multiresolution analysis based on the Laplacian pyramid (LP), originally proposed in BIB001 , is a bandpass image decomposition derived from the Gaussian pyramid (GP) which is a multiresolution (multiscale) image representation obtained through a recursive reductions of the image. LP is an oversampled transform that decomposes the image into nearly disjoint bandpass channels in the spatial frequency domain, without losing the spatial connectivity of its edges BIB008 . Figure 5 shows the concept of GP and its relation to LP. The generalized Laplacian pyramid (GLP) is an extension of the LP where a scale factor different of two is used BIB002 . An attractive characteristic of the GLP is that the low-pass reduction filter, used to analyze the PAN image, may be designed to match the MTF of the band into which the details extracted will be injected. The benefit is that the restoration of the spatial frequency content of the MS band is embedded into the enhancement procedure of the band itself, instead of being accomplished ahead of time. The steps for merging Landsat images using this GLP are described in Algorithm 4, where different injection methods can be used with GLP BIB012 BIB010 . In this context, injection means adding the details from the GLP to each MS band weighted by the coefficients obtained by the injection method. The Spectral Distortion Minimizing (SDM) injection model is both a spatially and spectrally varying model where the injected details at a pixel position must be parallel to the resampled MS vector at the same resolution. At the same time, the details are weighted to minimize the radiometric distortion measured as the Spectral Angle Mapper (SAM). In ContextBased Decision (CBD) injection model, the weights are calculated locally between the MS band resampled to the scale of the PAN image and an approximation of the PAN image at the resolution of the MS bands. Details are only injected if the local correlation coefficient between those images, calculated on a window of size N × N, is larger than a given threshold. The CBD model is uniquely defined by the set of thresholds, generally different for each band, and by the window size N, depending on the spatial resolutions and scale ratio of the images to be merged, as well as on the landscape characteristics, to avoid loss of local sensitivity BIB012 . Pansharpened images using wavelet/contourlet-based methods are shown in Figure 6 . Algorithm 4 General Laplacian Pyramid-based pansharpening The Ranchin-Wald-Mangolini (RWM) injection model BIB012 , unlike the SDM and CBD models, is calculated on bandpass details instead of approximations. RWM models the MS details as a space and spectral-varying linear combination of the PAN image coefficients. Another popular category of multiresolution pansharpening methods is the one based on Wavelet and Contourlet. Wavelet provide a framework for the decomposition of images into a hierarchy with a decreasing degree of resolution, separating detailed spatial information between successive levels BIB013 . The discrete approach of the wavelet transform, named discrete wavelet transform (DWT), can be performed using several different approaches, probably the most popular ones for image pansharpening being Mallat's BIB006  BIB004 BIB003 and the "a' trous" BIB022 BIB007 BIB009 algorithms. Each one has its particular mathematical properties and leads to different image decompositions. The first is an orthogonal, dyadic, non-symmetric, decimated, nonredundant DWT algorithm, while "a'trous" is a nonorthogonal, shift-invariant, dyadic, symmetric, undecimated, redundant DWT algorithm BIB013 . Redundant wavelet decomposition, as well as GLP, has an attractive characteristic: the low-pass reduction filter used to analyze the PAN image may be easily designed to match the MTF of the band to be enhanced. If the filters are correctly chosen, the extracted high spatial frequency components from the PAN image are properly retained, thus resulting in a greater spatial enhancement. It is important to note that undecimated, shift-invariant decompositions, and specifically "a' trous" wavelets, where sub-band and original image pixels correspond to the same locations, produce fewer artifacts and better preserve the linear continuity of features that do not have a horizontal or vertical orientation BIB016 and hence are better suited for image fusion. Contourlets provide a new representation system for image analysis BIB015 . The contourlet transform is so called because of its ability to capture and link the discontinuity points into linear structures (contours). The two-stage process used to derive the contourlet coefficients involves a multiscale transform and a local directional transform. First, a multiscale LP that detects discontinuities is applied. Then, a local directional filter bank is used to group these wavelet-like coefficients to obtain a smooth contour. Contourlets provide 2l directions at each scale, where l is the number of required orientations. This flexibility of having different numbers of directions at each scale makes contourlets different from other available multi-scale and directional image representations BIB018 . Similarly to wavelets, contourlets also have different implementations of the subsampled and non-subsampled BIB015 transforms. Algorithm 5 Wavelet/contourlet-based pansharpening 1. Forward transform the PAN and MS images using a sub-band and directional decomposition such as the subsampled or non-subsampled wavelet or contourlet transform. 2. Apply a fusion rule onto the transform coefficients. 3. Obtain the pansharpened image by performing the inverse transform. A number of pansharpening methods using the wavelet and, more recently, the contourlet transform have been proposed. In general, all the transform-based pansharpening methods follow the process in Algorithm 5. In the wavelet/contourlet-based approach, the MS and PAN images need to be decomposed multiple times in step 1 of Algorithm 5. Preliminary studies have shown that the quality of the pansharpened images produced by the wavelet-based techniques is a function of the number of decomposition levels. If few decomposition levels are applied, the spatial quality of the pansharpened images is less satisfactory. If an excessive number of levels is applied, the spectral similarity between the original MS and the pansharpened images decreases. Pradhan et al.  attempt in their work to determine the optimal number of decomposition levels for the wavelet-based pansharpening, producing the optimal spatial and spectral quality. The fusion rules in step 2 of the algorithm comprise, for instance, substituting the original MS coefficient bands by the coefficients of the PAN image or adding the coefficients of the PAN to the coefficients of the original MS bands, weighted sometimes by a factor dependent on the contribution of the PAN image to each MS band. It results in the different wavelet-and contourletbased pansharpening methods that will be described next. The additive wavelet/Contourlet method for fusing MS and PAN images uses the wavelet BIB013 /contourlet BIB017 transform in steps 1 and 3 in Algorithm 5, and for the fusion rule in step 2, it adds the detail bands of the MS image to those corresponding of the PAN image, having previously matched the MS histogram to one of the PAN image. The Substitutive Wavelet/Contourlet methods are quite similar to the additive ones, but instead of adding the information of the PAN image to each band of the MS image, these methods simply replace the MS detail bands with the details obtained from the PAN image (see BIB007 for wavelet and BIB023 for contourlet decomposition). A number of hybrid methods have been developed to attempt to combine the best aspects of classical methods and wavelet and contourlet transforms. Research has mainly focused on incorporating the IHS, PCA and BT into wavelet and contourlet methods. As we have seen, some of the most popular image pansharpening methods are based on the IHS transformation. The main drawback of these methods is the high distortion of the original spectral information present in the resulting MS images. To avoid this problem, the IHS transformation is followed by the additive wavelet or contourlet method in the so-called wavelet BIB013 and contourlet BIB019 BIB024 additive IHS pansharpening. If the IHS transform is followed by the substitutive wavelet method, the wavelet substitutive IHS  pansharpening method is obtained. Similarly to the IHS wavelet/contourlet methods, the PCA wavelet BIB011 BIB013 /contourlet BIB018 methods are based on applying substitutive wavelet/contourlet methods to the first principal component (PC1) instead of applying it to the bands of the MS image. Adaptive PCA has also been applied in combination with contourlets BIB018 . The WiSpeR BIB014 method can be considered as a generalization of different wavelet-based image fusion methods. It uses a modification of the non-subsampled additive wavelet algorithm where the contribution from the PAN image to each of the fused bands depends on a factor generated from both the sensor spectral response and the physical properties of the observed object. A new contourlet pan-sharpening method named CiSper was proposed in BIB026 that, similarly to WiSper, weights the contribution of the PAN image to each MS band, but it uses a different method to calculate these weights and uses the non-subsampling contourlet transform instead of the wavelet transform. In order to take advantage of multiresolution analysis, the use of pansharpening based on the statistics of the image on the wavelet/ contourlet domain has been suggested BIB027 BIB020 . Pansharpened images using wavelet-and contourlet-based methods are shown in Figure 6b -f. Some authors  BIB006 state that multisensor image fusion is a trade-off between the spectral information from an MS sensor and the spatial information from a PAN sensor and that wavelet transform fusion methods easily control this trade-off. The trade-off idea, however, is just a convenient simplification, as discussed in BIB021 , and ideal fusion methods must be able to simultaneously reach both spectral and spatial quality, and not one at the expense of the other. To do so, the physics of the capture process have to be taken into account, and the methods have to adapt to the local properties of the images.