A survey of classical methods and new trends in pansharpening of multispectral images <s> Quality assessment without a reference <s> This letter focuses on quality assessment of fusion of multispectral (MS) images with high-resolution panchromatic (Pan) observations. A new quality index suitable for MS imagery having four spectral bands is defined from the theory of hypercomplex numbers, or quaternions. Both spectral and radiometric distortion measurements are encapsulated in a unique measurement, simultaneously accounting for local mean bias, changes in contrast, and loss of correlation of individual bands, together with spectral distortion. Results are presented and discussed on very high-resolution QuickBird data, through comparisons between state-of-the-art and advanced MS+Pan merge algorithms. <s> BIB001 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> Quality assessment without a reference <s> Quality assessment of pansharpening methods is not an easy task. Quality-assessment indexes, like Q4, spectral angle mapper, and relative global synthesis error, require a reference image at the same resolution as the fused image. In the absence of such a reference image, the quality of pansharpening is assessed at a degraded resolution only. The recently proposed index of Quality Not requiring a Reference (QNR) is one among very few tools available for assessing the quality of pansharpened images at the desired high resolution. However, it would be desirable to cross the outcomes of several independent quality-assessment indexes, in order to better determine the quality of pansharpened images. In this paper, we propose a method to assess fusion quality at the highest resolution, without requiring a high-resolution reference image. The novel method makes use of digital filters matching the modulation transfer functions (MTFs) of the imaging-instrument channels. Spectral quality is evaluated according to Wald's spectral consistency property. Spatial quality measures interscale changes by matching spatial details, extracted from the multispectral bands and from the panchromatic image by means of the high-pass complement of MTF filters. Eventually, we highlight the necessary and sufficient condition criteria for quality-assessment indexes by developing a pansharpening method optimizing the QNR spatial index and assessing the quality of fused images by using the proposed protocol. <s> BIB002
Quantitative quality of data fusion methods can be provided when using reference images, usually obtained by degrading all available data to a coarser resolution and carrying out fusion from such data. A set of global indices capable of measuring the quality of pansharpened MS images and working at full scale without performing any preliminary degradation of the data have been recently proposed. The Quality with No Reference (QNR) index  comprises two indices, one pertaining to spectral distortion and the other to spatial distortion. As proposed in  and  , the two distortions may be combined to yield a unique global quality measure. While the QNR measure proposed in  is based on the UIQI index, the one proposed in  is based on the measure of the mutual information (MI) between the different images. The spectral distortion index defined in  can be derived from the difference of inter-band UIQI values calculated among all the fused MS bands and the resampled LR MS bands. The spatial distortion index defined in  is based on differences between the UIQI of each band of the fused image and the PAN image and the UIQI of each band of the LR MS image with a low-resolution version of the PAN image. This LR image is obtained by filtering the PAN image with a low-pass filter with normalized frequency cutoff at the resolution ratio between MS and PAN images, followed by decimation. The QNR index is obtained by the combination of the spectral and spatial distortion indices into one single measure varying from zero to one. The maximum value is only obtained when there is no spectral and spatial distortion between the images. The main advantage of the proposed index is that, in spite of lack of a reference data set, the global quality of a fused image can be assessed at the full scale of the PAN image. The QNR method proposed in  is based on the MI measure instead of UIQI. The mutual information between resampled original and fused MS bands is used to measure the spectral quality, while the mutual information between the PAN image and the fused bands yields a measure of spatial quality. Another protocol was proposed by Khan et al. BIB002 to assess spectral and spatial quality at full scale. For assessing spectral quality, the MTF of each spectral channel is used to low-pass filter the fused image. This filtered image, once it has been decimated, will give a degraded LR MS image. For comparing the degraded and original low-resolution MS images, the Q4 index BIB001 is used. Note that the MTF filters for each sensor are different and the exact filter response is not usually provided by the instrument manufacturers. However, the filter gain at Nyquist cutoff frequency may be derived from onorbit measurements. Using this information and assuming that the frequency response of each filter is approximately Gaussian shaped, MTF filters for each sensor of each satellite can be estimated. To assess the spatial quality of the fused image, the high-pass complement of the MTF filters is used to extract the high-frequency information from the MS images at both high (fused) and low (original) resolutions. In addition, the PAN image is downscaled to the resolution of the original MS image, and high-frequency information is extracted from high-and low-resolution PAN images. The UIQI value is calculated between the details of the each MS band and the details of the PAN image at the two resolutions. The average of the absolute differences in the UIQI values across scale of each band produces the spatial index.