A survey of classical methods and new trends in pansharpening of multispectral images <s> Pansharpening categories <s> A procedure is developed for combining high spatial resolution panchromatic data with lower resolution multispectral data in order to produce high spatial resolution digital data in multispectral form. Data simulating the French SPOT satellite were processed to resemble high altitude aerial photography, but image artifacts can hamper photointerpretative methods. <s> BIB001 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Pansharpening categories <s> The merging of multisensor image data is becoming a widely used procedure because of the complementary nature of various data sets. Ideally, the method used to merge data sets with high-spatial and high-spectral resolution should not distort the spectral characteristics of the high-spectral resolution data. This paper compares the results of three different methods used to merge the information contents of the Landsat Thermatic Mapper (TM) and Satellite Pour l'Observation de la Terre (SPOT) panchromatic data. The comparison is based on spectral characteristics and is made using statistical, visual, and graphical analyses of the results <s> BIB002 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Pansharpening categories <s> A generalized Component Substitution (COS) technique is presented for enhancing spatial resolution of multispectral bands using a higher resolution data set. The process involves a simple one-step linear transformation of the combined data space, and the procedure can be used for the implementation of the popular Intensity Hue Saturation (IHS) transformation technique of data merging. In the generalized procedure the weights used in the linear transformation are scene dependent and are determined from multivariate statistical techniques. <s> BIB003 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Pansharpening categories <s> Abstract Multispectral image data fusion is understood here as a process that generates synthetic images from a combination of primary images, by attempting to preserve the best characteristics of each primary image. The obtained product is important in helping the users of remote sensing in visual analysis. This paper describes a new method for multispectral image data fusion using a Bayesian framework. As an example, the method is applied to the synthesis of new bands for the SPOT satellite. The method comprises two steps: (a) a new interpolator for the multispectral bands, obtained through the orthogonality principle and leading to the estimator and its covariance.matrix, which will be used as a priori information for the second step; and (b) a new statistical synthesis formulation, also obtained through the orthogonality principle, using as observations the panchromatic and the multispectral bands. Experimental results with SPOT images over Guarulhos Airport, Sao Paulo, Brazil, are presented, includin... <s> BIB004 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Pansharpening categories <s> The Nature of Remote Sensing: Introduction. Remote Sensing. Information Extraction from Remote-Sensing Images. Spectral Factors in Remote Sensing. Spectral Signatures. Remote-Sensing Systems. Optical Sensors. Temporal Characteristics. Image Display Systems. Data Systems. Summary. Exercises. References. Optical Radiation Models: Introduction. Visible to Short Wave Infrared Region. Solar Radiation. Radiation Components. Surface-Reflected. Unscattered Component. Surface-Reflected. Atmosphere-Scattered Component. Path-Scattered Component. Total At-Sensor. Solar Radiance. Image Examples in the Solar Region. Terrain Shading. Shadowing. Atmospheric Correction. Midwave to Thermal Infrared Region. Thermal Radiation. Radiation Components. Surface-Emitted Component. Surface-Reflected. Atmosphere-Emitted Component. Path-Emitted Component. Total At-Sensor. Emitted Radiance. Total Solar and Thermal Upwelling Radiance. Image Examples in the Thermal Region. Summary. Exercises. References. Sensor Models: Introduction. Overall Sensor Model. Resolution. The Instrument Response. Spatial Resolution. Spectral Resolution. Spectral Response. Spatial Response. Optical PSFopt. Image Motion PSFIM. Detector PSFdet. Electronics PSFel. Net PSFnet. Comparison of Sensor PSFs. PSF Summary for TM. Imaging System Simulation. Amplification. Sampling and Quantization. Simplified Sensor Model. Geometric Distortion. Orbit Models. Platform Attitude Models. Scanner Models. Earth Model. Line and Whiskbroom ScanGeometry. Pushbroom Scan Geometry. Topographic Distortion. Summary. Exercises. References. Data Models: Introduction. A Word on Notation. Univariate Image Statistics. Histogram. Normal Distribution. Cumulative Histogram. Statistical Parameters. Multivariate Image Statistics. Reduction to Univariate Statistics. Noise Models. Statistical Measures of Image Quality. Contrast. Modulation. Signal-to-Noise Ratio (SNR). Noise Equivalent Signal. Spatial Statistics. Visualization of Spatial Covariance. Covariance with Semivariogram. Separability and Anisotropy. Power Spectral Density. Co-occurrence Matrix. Fractal Geometry. Topographic and Sensor Effects. Topography and Spectral Statistics. Sensor Characteristics and Spectral Stastistics. Sensor Characteristics and Spectral Scattergrams. Summary. Exercises. References. Spectral Transforms: Introduction. Feature Space. Multispectral Ratios. Vegetation Indexes. Image Examples. Principal Components. Standardized Principal Components (SPC) Transform. Maximum Noise Fraction (MNF) Transform. Tasseled Cap Tranformation. Contrast Enhancement. Transformations Based on Global Statistics. Linear Transformations. Nonlinear Transformations. Normalization Stretch. Reference Stretch. Thresholding. Adaptive Transformation. Color Image Contrast Enhancement. Min-max Stretch. Normalization Stretch. Decorrelation Stretch. Color Spacer Transformations. Summary. Exercises. References. Spatial Transforms: Introduction. An Image Model for Spatial Filtering. Convolution Filters. Low Pass and High Pass Filters. High Boost Filters. Directional Filters. The Border Region. Characterization of Filtered Images. The Box Filter Algorithm. Cascaded Linear Filters. Statistical Filters. Gradient Filters. Fourier Synthesis. Discrete Fourier Transforms in 2-D. The Fourier Components. Filtering with the Fourier Transform. Transfer Functions. The Power Spectrum. Scale Space Transforms. Image Resolution Pyramids. Zero-Crossing Filters. Laplacian-of-Gaussian (LoG) Filters. Difference-of-Gaussians (DoG) Filters.Wavelet Transforms. Summary. Exercises. References. Correction and Calibration: Introduction. Noise Correction. Global Noise. Sigma Filter. Nagao-Matsuyama Filter. Local Noise. Periodic Noise. Distriping 359. Global,Linear Detector Matching. Nonlinear Detector Matching. Statistical Modification to Linear and Nonlinear Detector. Matching. Spatial Filtering Approaches. Radiometric Calibration. Sensor Calibration. Atmospheric Correction. Solar and Topographic Correction. Image Examples. Calibration and Normalization of Hyperspectral Imagery. AVIRIS Examples. Distortion Correction. Polynomial Distortion Models. Ground Control Points (GCPs). Coordinate Transformation. Map Projections. Resampling. Summary. Exercises References. Registration and Image Fusion: Introduction. What is Registration? Automated GCP Location. Area Correlation. Other Spatial Features. Orthrectification. Low-Resolution DEM. High-Resolution DEM. Hierarchical Warp Stereo. Multi-Image Fusion. Spatial Domain Fusion. High Frequency Modulation. Spectral Domain Fusion. Fusion Image Examples. Summary. Exercises. References. Thematic Classification: Introduction. The Importance of Image Scale. The Notion of Similarity. Hard Versus Soft Classification. Training the Classifier. Supervised Training. Unsupervised Training. K-Means Clustering Algorithm. Clustering Examples. Hybrid Supervised/Unsupervised Training. Non-Parametric Classification Algorithms. Level-Slice. Nearest-Mean. Artificial Neural Networks (ANNs). Back-Propagation Algorithm. Nonparametric Classification Examples. Parametric Classification Algorithms. Estimation of Model-Parameters. Discriminant Functions. The Normal Distribution Model. Relation to the Nearest-Mean Classifier. Supervised Classification Examples and Comparison to Nonparametric Classifiers. Segmentation. Region Growing. Region Labeling. Sub-Pixel Classification. The Linear Mixing Model. Unmixing Model. Hyperspectral Image Analysis. Visualization of the Image Cube. Feature Extraction. Image Residuals. Pre-Classification Processing and Feature Extraction. Classification Algorithms. Exercises. Error Analysis. Multitemporal Images. Summary. References. Index. <s> BIB005 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Pansharpening categories <s> Abstract To take advantage of the high spectral resolution of Landsat TM images and the high spatial resolution of SPOT panchromatic images (SPOT PAN), we present a wavelet transform method to merge the two data types. In a pyramidal fashion, each TM reflective band or SPOT PAN image was decomposed into an orthogonal wavelet representation at a given coarser resolution, which consisted of a low frequency approximation image and a set of high frequency, spatially-oriented detail images. Band-by-band, the merged images were derived by performing an inverse wavelet transform using the approximation image from each TM band and detail images from SPOT PAN. The spectral and spatial features of the merged results of the wavelet methods were compared quantitatively with those of intensity-hue-saturation (IHS), principal component analysis (PCA), and the Brovey transform. It was found that multisensor data merging is a trade-off between the spectral information from a low spatial-high spectral resolution sensor an... <s> BIB006 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Pansharpening categories <s> This paper compares two general and formal solutions to the problem of fusion of multispectral images with high-resolution panchromatic observations. The former exploits the undecimated discrete wavelet transform, which is an octave bandpass representation achieved from a conventional discrete wavelet transform by omitting all decimators and upsampling the wavelet filter bank. The latter relies on the generalized Laplacian pyramid, which is another oversampled structure obtained by recursively subtracting from an image an expanded decimated lowpass version. Both the methods selectively perform spatial-fre- quencies spectrum substitution from an image to another. In both schemes, context dependency is exploited by thresholding the local correlation coefficient between the images to be merged, to avoid injection of spatial details that are not likely to occur in the target image. Unlike other multiscale fusion schemes, both the present decompositions are not critically subsampled, thus avoiding possible impairments in the fused images, due to missing cancellation of aliasing terms. Results are presented and discussed on SPOT data. <s> BIB007 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Pansharpening categories <s> Due to the performance limit of remote sensing systems, multispectral images have limited spatial resolution. Their spatial resolution can be improved by merging them with higher resolution image data. A fundamental problem frequently occurring in existing fusion processes, however, is the distortion of spectral information. This paper presents a spatially adaptive image fusion algorithm which produces visually natural images and retains the quality of local spectral information as well. High frequency information of the high resolution image to be inserted to the resampled multispectral images is controlled by adaptive gains to incorporate the difference of local spectral characteristics between the high and the low resolution images into the fusion. Each gain is estimated to minimize the l 2-norm of the error between the original and the estimated pixel values defined in a spatially adaptive window of which the weights are proportional to the spectral correlation measurements of the corresponding region... <s> BIB008 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Pansharpening categories <s> This paper addresses the modeling of wavelet coefficients for multispectral (MS) band sharpening based on undecimated multiresolution analysis (MRA). The coarse MS bands are sharpened by injecting highpass details taken from a high-resolution panchromatic (Pan) image. Besides the MRA, crucial point is modeling the relationships between detail coefficients of a generic MS band and the Pan image at the same resolution. Once calculated at the coarser resolution, where both types of data are available, such a model shall be extrapolated to the finer resolution in order to weight the Pan details to be injected. The goal is that the merged MS images are most similar to what the MS sensor would collect if it had the same resolution as the broadband Pan imager. Three injection models embedded in an ''a trous'' wavelet decomposition will be described and compared on a test set of very high-resolution QuickBird MS+Pan data. Two models work on approximation and detail coefficients, respectively, and provide a certain degree of unmixing of coarse MS pixels. The third model is based on spectral fidelity of the merged image to the (resampled) original MS data, that is, no unmixing is attempted. It is much simpler than the other two because it does not require calculation of local statistics. Fusion comparisons on spatially degraded data, of which higher-resolution true MS data are available for reference, show that the former two models yield better results than the latter, in terms of both radiometric and spectral fidelity. The latter, however, yields a reliable sharpening unaffected by local artifacts, regardless of landscape complexity. When local statistics of wavelet coefficients are used, the model estimated on approximation yields slightly better but considerably stabler results than that calculated starting from bandpass details. <s> BIB009 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Pansharpening categories <s> In this paper, we develop the nonsubsampled contourlet transform (NSCT) and study its applications. The construction proposed in this paper is based on a nonsubsampled pyramid structure and nonsubsampled directional filter banks. The result is a flexible multiscale, multidirection, and shift-invariant image decomposition that can be efficiently implemented via the à trous algorithm. At the core of the proposed scheme is the nonseparable two-channel nonsubsampled filter bank (NSFB). We exploit the less stringent design condition of the NSFB to design filters that lead to a NSCT with better frequency selectivity and regularity when compared to the contourlet transform. We propose a design framework based on the mapping approach, that allows for a fast implementation based on a lifting or ladder structure, and only uses one-dimensional filtering in some cases. In addition, our design ensures that the corresponding frame elements are regular, symmetric, and the frame is close to a tight one. We assess the performance of the NSCT in image denoising and enhancement applications. In both applications the NSCT compares favorably to other existing methods in the literature. <s> BIB010 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Pansharpening categories <s> This paper proposes a new fusion method that permits an adequate selection of information extracted from source images to obtain fused images with good spatial and spectral quality simultaneously. This method is based on a joint multiresolution multidirectional representation of the source images using a single directional spatial frequency low pass filter bank of low computational complexity, defined in the Fourier domain. The source images correspond to those captured by the IKONOS satellite (panchromatic and multispectral). The results obtained indicate that the proposed method provides, in a simple manner, objective control over the trade-off between high spatial and spectral quality of the fused images. <s> BIB011 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Pansharpening categories <s> This article aims at explaining the ARSIS concept. By fusing two sets of images A and B, one with a high spatial resolution, the other with a low spatial resolution and different spectral bands, the ARSIS concept permits to synthesise the dataset B at the resolution of A that is as close as possible to reality. It is based on the assumption that the missing information is linked to the high frequencies in the sets A and B. It searches a relationship between the high frequencies in the multispectral set B and the set A and models this relationship. The general problem for the synthesis is presented first. The general properties of the fused product are given. Then, the ARSIS concept is discussed. The general scheme for the implementation of a method belonging to this concept is presented. Then, this article intends to help practitioners and researchers to better understand this concept through practical details about implementations. Two Multiscale Models are described as well as two Inter-Band Structure Models (IBSM). They are applied to an Ikonos image as an illustration case. The fused products are assessed by means of a known protocol comprising a series of qualitative and quantitative tests. The products are found of satisfactory quality. This case illustrates the differences existing between the various models, their advantages and limits. Tracks for future improvements are discussed. <s> BIB012 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Pansharpening categories <s> Pansharpening is a technique that fuses the information of a low-resolution multispectral image and a high-resolution panchromatic image, usually remote sensing images, to provide a high-resolution multispectral image. In the literature, this task has been addressed from different points of view being one of the most popular wavelet-based algorithms. Recently, a new transform, the contourlet transform, has been proposed. This transform combines the advantages of the wavelets transform, with a more efficient directional information representation. The result is a flexible multiscale, multidirection and shift-invariant decomposition that can be efficiently implemented via the a’trous algorithm. In this chapter, we compare the wavelet-based pansharpening with existing contourlet-based approaches and propose a new pansharpening method based on the contourlet transform. The performance of the contourlet in general, and the proposed method in particular, is assessed numerically and visually for Landsat and SPOT images. <s> BIB013
Once the remote sensing images are pre-processed in order to satisfy the pansharpening method requirements, the pansharpening process is performed. The literature shows a large collection of these pansharpening methods developed over the last two decades as well as a large number of terms used to refer to image fusion. In 1980, Wong et al.  proposed a technique for the integration of Landsat Multispectral Scanner (MSS) and Seasat synthetic aperture radar (SAR) images based on the modulation of the intensity of each pixel of the MSS channels with the value of the corresponding pixel of the SAR image, hence named intensity modulation (IM) integration method. Other scientists evaluated multisensor image data in the context of co-registered  , resolution enhancement  or coincident  data analysis. After the launch of the French SPOT satellite system in February of 1986, the civilian remote sensing sector was provided with the capability of applying high-resolution MS imagery to a range of land use and land cover analyses. Cliche et al.  who worked with SPOT simulation data prior to the satellite's launch showed that simulated 10-m resolution color images can be produced by modulating each SPOT MS (XS) band with PAN data individually, using three different intensity modulation (IM) methods. Welch et al.  used the term "merge" instead of "integration" and proposed merging of SPOT PAN and XS data using the IntensityHue-Saturation (IHS) transformation, a method previously proposed by Haydn et al.  to merge Landsat MSS with Return Beam Vidicon (RBV) data and Landsat MSS with Heat Capacity Mapping Mission data. In 1988, Chavez et al.  used SPOT panchromatic data to "sharpen" Landsat Thematic Mapper (TM) images by high-pass filtering (HPF) the SPOT PAN data before merging it with the TM data. A review of the so-called classical methods, which include IHS, HPF, Brovey transform (BT)  and principal component substitution (PCS)  BIB002 , among others, can be found in  . In 1987, Price BIB001 developed a fusion technique based on the statistical properties of remote sensing images, for the combination of the two different spatial resolutions of the High Resolution Visible (HRV) SPOT sensor. Besides the Price method, the literature shows other pansharpening methods based on the statistical properties of the images, such as spatially adaptive methods BIB008 and Bayesian-based methods BIB004  . More recently, multiresolution analysis employing the generalized Laplacian pyramid (GLP) BIB007 BIB009 , the discrete wavelet transform  BIB006 and the contourlet transform BIB010 BIB011 BIB013 has been used in pansharpening using the basic idea of extracting the spatial detail information from the PAN image not present in the low-resolution MS image, to inject it into the later. Image fusion methods have been classified in several ways. Schowengerdt BIB005 classified them into spectral domain, spatial domain and scale-space techniques. Ranchin and Wald  classified them into three groups: projection and substitution methods, relative spectral contribution methods and those relevant to the ARSIS concept (from its French acronym "Amélioration de la Résolution Spatiale par Injection de Structures" which means "Enhancement of the spatial resolution by structure injections"). It was found that many of the existing image fusion methods, such as the HPF and additive wavelet transform (AWT) methods, can be accommodated within the ARSIS concept BIB012 , but Tu et al.  found that the PCS, BT and AWT methods could be also considered as IHS-like image fusion methods. Meanwhile, Bretschneider et al.  classified IHS and PCA methods as transformation-based methods, in a classification that also included more categories such as addition and multiplication fusion, filter fusion (which includes HPF method), fusion based on inter-band relations, wavelet decomposition fusion and further fusion methods (based on statistical properties). Fusion methods that involve linear forward and backward transforms had been classified by Sheftigara BIB003 as component substitution methods. Recently, two comprehensive frameworks that generalize previously proposed fusion methods such as IHS, BT, PCA, HPF or AWT and study the relationships between different methods have been proposed in   . Although it is not possible to find a universal classification, in this work we classify the pansharpening methods into the following categories according to the main technique they use: (1) Component Substitution (CS) family, which includes IHS, PCS and Gram-Schmidt (GS), because all these methods utilize, usually, a linear transformation and substitution for some components in the transformed domain. (2) Relative Spectral Contribution family, which includes BT, IM and P+XS, where a linear combination of the spectral bands, instead of substitution, is applied. (3) High-Frequency Injection family, which includes HPF and HPM, where these two methods inject highfrequency details extracted by subtracting a low-pass filtering PAN image from the original one. (4) Methods based on the statistics of the image, which include Price and spatially adaptive methods, Bayesian-based and super-resolution methods. (5) Multiresolution family, which includes generalized Laplacian pyramid, wavelet and contourlet methods and any combination of multiresolution analysis with methods from other categories. Note that although the proposed classification defines five categories, as we have already mentioned, some methods can be classified in several categories and, so, the limits of each category are not sharp and there are many relations among them. The relations will be explained when the categories are described.