A survey of classical methods and new trends in pansharpening of multispectral images <s> for instance). <s> The problem of determining a shaping filter for nonstationary colored noise is considered. The shaping filter transforms white noise into a possibly nonstationary random process (having no white noise component) with a specified covariance function. A set of conditions to be satisfied by the covariance function leads to the determination of a shaping filter. The shaping filter coefficients are simply related to the solution of a matrix Riccati equation. In order to formulate the Riccati equation, basic results concerning the mean-square differentiability of a random process are developed. If the Riccati equation can not be defined, an autonomous (zero-input) shaping filter may be easily determined. <s> BIB001 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> for instance). <s> Abstract Multispectral image data fusion is understood here as a process that generates synthetic images from a combination of primary images, by attempting to preserve the best characteristics of each primary image. The obtained product is important in helping the users of remote sensing in visual analysis. This paper describes a new method for multispectral image data fusion using a Bayesian framework. As an example, the method is applied to the synthesis of new bands for the SPOT satellite. The method comprises two steps: (a) a new interpolator for the multispectral bands, obtained through the orthogonality principle and leading to the estimator and its covariance.matrix, which will be used as a priori information for the second step; and (b) a new statistical synthesis formulation, also obtained through the orthogonality principle, using as observations the panchromatic and the multispectral bands. Experimental results with SPOT images over Guarulhos Airport, Sao Paulo, Brazil, are presented, includin... <s> BIB002 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> for instance). <s> This paper presents a novel maximum a posteriori estimator for enhancing the spatial resolution of an image using co-registered high spatial-resolution imagery from an auxiliary sensor. Here, we focus on the use of high-resolution panchromatic data to enhance hyperspectral imagery. However, the estimation framework developed allows for any number of spectral bands in the primary and auxiliary image. The proposed technique is suitable for applications where some correlation, either localized or global, exists between the auxiliary image and the image being enhanced. To exploit localized correlations, a spatially varying statistical model, based on vector quantization, is used. Another important aspect of the proposed algorithm is that it allows for the use of an accurate observation model relating the "true" scene with the low-resolutions observations. Experimental results with hyperspectral data derived from the airborne visible-infrared imaging spectrometer are presented to demonstrate the efficacy of the proposed estimator. <s> BIB003 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> for instance). <s> A maximum a posteriori (MAP) estimation method for improving the spatial resolution of a hyperspectral image using a higher resolution auxiliary image is extended to address several practical remote sensing situations. These include cases where: 1) the spectral response of the auxiliary image is unknown and does not match that of the hyperspectral image; 2) the auxiliary image is multispectral; and 3) the spatial point spread function for the hyperspectral sensor is arbitrary and extends beyond the span of the detector elements. The research presented follows a previously reported MAP approach that makes use of a stochastic mixing model (SMM) of the underlying spectral scene content to achieve resolution enhancement beyond the intensity component of the hyperspectral image. The mathematical formulation of a generalized form of the MAP/SMM estimate is described, and the enhancement algorithm is demonstrated using various image datasets. <s> BIB004 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> for instance). <s> A new super resolution Bayesian method for pansharpening multispectral images which incorporates sensor characteristics to model the observation process of both the panchromatic and the multispectral images is presented in this paper. The quality of the pansharpened images is assessed both qualitatively and quantitatively. Preliminary results are very promising: the method succeeded in preserving the spectral information while increasing the spatial resolution of the Landsat ETM+ multispectral image. Further research will focus on extending the method to fuse imagery acquired by other sensors. <s> BIB005 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> for instance). <s> In this paper we present a super resolution Bayesian methodology for pansharpening of multispectral images. By following the hierarchical Bayesian framework, and by applying variational methods to approximate probability distributions this methodology is able to: (a) incorporate prior knowledge on the expected characteristics of the multispectral images, (b) use the sensor characteristics to model the observation process of both panchromatic and multispectral images, (c) include information on the unknown parameters in the model in the form of hyperprior distributions, and (d) estimate the parameters of the hyperprior distributions on the unknown parameters together with the unknown parameters, and the high resolution multispectral image. Using real data, the pansharpened multispectral images are compared with the images obtained by other pansharpening methods and their quality is assessed both qualitatively and quantitatively. <s> BIB006 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> for instance). <s> A new remote sensing image fusion method based on statistical parameter estimation is proposed in this paper. More specially, Bayesian linear estimation (BLE) is applied to observation models between remote sensing images with different spatial and spectral resolutions. The proposed method only estimates the mean vector and covariance matrix of the high-resolution multispectral (MS) images, instead of assuming the joint distribution between the panchromatic (PAN) image and low-resolution multispectral image. Furthermore, the proposed method can enhance the spatial resolution of several principal components of MS images, while the traditional Principal Component Analysis (PCA) method is limited to enhance only the first principal component. Experimental results with real MS images and PAN image of Landsat ETM+ demonstrate that the proposed method performs better than traditional methods based on statistical parameter estimation, PCA-based method and wavelet-based method. <s> BIB007 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> for instance). <s> Currently, most optical Earth observation satellites carry both a panchromatic sensor and a set of lower spatialresolution multispectral sensors. In order to benefit from both sources of information, several pansharpening methods have been developed to produce a multispectral image at the spatial resolution of the panchromatic band. The aim of this paper is to suggest a novel approach to the pansharpening problem within a Bayesian framework. This Bayesian data fusion (BDF) method relies on statistical relationships between the various spectral bands and the panchromatic band without suffering from restricting modeling hypotheses. Furthermore, it allows the user to weight the spectral and panchromatic information with respect to either visual or quantitative criteria, which leads to adaptable results according to users' needs and study areas. The performance of this approach was compared to existing methods based on markedly different subset images from very high spatial resolution IKONOS images. Results showed that BDF yielded the highest spectral consistency. Furthermore, small details were adequately added to the pansharpened images with little artifact as compared to those created using wavelet-based methods. Finally, the method was fast and easy to implement owing to its straightforward formulation. As it does not have any intrinsic limitations on the type of data to be processed or the number of bands to be merged, it also appears to be very promising for optical/SAR or hyperspectral image fusion. <s> BIB008 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> for instance). <s> In this paper we propose a novel algorithm for the pansharpening of multispectral images based on the use of a Total Variation (TV) image prior. Within the Bayesian formulation, the proposed methodology incorporates prior knowledge on the expected characteristics of multispectral images, and uses the sensor characteristics to model the observation process of both panchromatic and multispectral images. Using real and synthetic data, the pansharpened multispectral images are compared with the images obtained by other pansharpening methods and their quality is assessed both qualitatively and quantitatively. <s> BIB009 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> for instance). <s> In this paper we propose a novel super-resolution based algorithm for the pansharpening of multispectral images. Within the Bayesian formulation, the proposed methodology incorporates prior knowledge on the expected characteristics of multispectral images; that is, it imposes smoothness within each band by means of the energy associated with the ?1 norm of vertical and horizontal first order differences of image pixel values and also takes into account the correlation among the bands of the multispectral image. The observation process is modeled using the sensor characteristics of both panchromatic and multispectral images. The method is tested on real and synthetic images, compared with other pansharpening methods, and the quality of the results assessed both qualitatively and quantitatively. <s> BIB010 </s> A survey of classical methods and new trends in pansharpening of multispectral images <s> for instance). <s> Multisensor data fusion methods for Gaussian noise are widely reported, while fusion approaches for non-Gaussian noise are seldom met in the literature. In this study, four statistical fusion methods are presented for a mixture of Gaussians noise. These four methods are the minimum variance approach, the maximum kurtosis approach, the minimum kurtosis approach, and the minimum mean absolute error approach. Preliminary numerical simulations demonstrate that the maximum kurtosis method shows the worst fusion performance, while the rest three methods shows equivalent better fusion performance. <s> BIB011
Bayesian methods model the degradation suffered by the original HRMS image, z, as the conditional probability distribution of the observed LRMS image, y, and the PAN image, x, given the original z, called the likelihood and denoted as p(y, x|z). They take into account the available prior knowledge about the expected characteristics of the pansharpened image, modeled in the socalled prior distribution p(z), to determine the posterior probability distribution p(z|y, x) by using Bayes law, where p(y, x) is the joint probability distribution. Inference is performed from the posterior distribution to draw estimates of the HRMS image, z. The main advantage of the Bayesian approach is to place the problem of pansharpening into a clear probabilistic framework BIB008 , although assigning suitable distributions for the conditional and prior distributions and the selection of an inference method are critical points that lead to different Bayesian-based pansharpening methods. As prior distribution, Fasbender et al. BIB008 assumed a noninformative prior p(z) ∝ 1, which gives equal probability to all possible solutions, that is, no solution is preferred as no clear information on the HRMS image is available. This prior has also been used by Hardie et al. BIB003 . In BIB002 , the prior information is carried out by an interpolation operator and its covariance matrix; both will be used as the mean vector and the covariance matrix, respectively, for a Bayesian synthesis process. In BIB006 , the prior knowledge about the smoothness of the object luminosity distribution within each band makes it possible to model the distribution of z using a simultaneous autoregressive model (SAR) as where C denotes the Laplacian operator and 1/a b is the variance of the Gaussian distribution of z b , b = 1, ..., B, with B being the number of bands in the MS image. More advanced models try to incorporate a smoothness constrain while preserving the edges in the image. Those models include adaptive SAR model  , Total Variation (TV) BIB009 , Markov Random Fields (MRF)  based models and Stochastic Mixing Models (SMM) BIB004 . Note that the described models do not take into account the correlations between the MS bands. In BIB010 , the authors propose a TV prior model to take into account spatial pixel relationships and a quadratic model to enforce similarity between the pixels in the same position in the different bands. It is usual to model the LRMS and PAN images as degraded versions of the HRMS image by two different processes: one modeling the LRMS image and usually described as where g s (z) represents a function that relates z to y and n s represents the noise of the LRMS image, and a second one that models how the PAN image is obtained from the HRMS image, which is written as where g p (z) represents a function that relates z to x and n p represents the noise of the PAN image. Note that, since the success of the pansharpening algorithm will be limited by the accuracy of those models, the physics of the sensor should be considered. In particular, the MTF of the sensor and the sensor's spectral response should be taken into account. The conditional distribution of the observed images given the original one, p(y, x|z), is usually defined as by considering that the observed LRMS image and the PAN image are independent given the HRMS image. This allows an easier formulation of the degradation models. However, Fasbender et al. BIB008 took into account that y and x may carry information of quite different quality about z and defined p(y, x|z) = p(y|z) , where the parameter w [0, 1] can be interpreted as the weight to be given to the panchromatic information at the expense of the MS information. Note that w = 0.5 leads back to Equation 12 while a value of zero or one means that we are discarding the PAN or the MS image, respectively. Different models have been proposed for the conditional distributions p(y|z) and p(x|z). The simpler model is to assume that g s (z) = z, so that y will be then y = z + n s BIB008 where n s~N (0, Σ s ). Note that in this case, y has the same resolution as z so an interpolation method has to be used to obtain y from the observed MS image. However, most of the authors consider the relation y = H z + n s where H is a matrix representing the blurring, usually represented by its MTF, the sensor integration function and the spatial subsampling and n s is the capture noise, assumed to be Gaussian with zero mean and variance 1/b, leading to the distribution This model has been extensively used  BIB004  , and it is the base for the so-called super-resolution-based methods  as the ones described, for instance, in  BIB009 . The degradation model in BIB002 can be also written in this way. A pansharpened image using the super-resolution method proposed in BIB009 is shown in Figure  3b . On the other hand, g p (z) has been defined as a linear regression model linking the MS pixels to the PAN ones, as estimated from both observed images, so that where a and l b , b = 1, 2, . .., B, are the regression parameters. Note that this model is used by IHS, PCA and Brovey methods to relate the PAN and HRMS images. Mateos et al. BIB005 (and also  BIB009  for instance) used a special case for g p (z), where a = 0 and l b ≥ 0, b = 1, 2, ..., B are known quantities that can be obtained from the sensor spectral characteristics (see Figure 4 for the Landsat 7 ETM+ spectral response) that represent the contribution of each MS band to the PAN image. In all those papers, the noise n p is assumed to be Gaussian with zero mean and covariance matrix C p and hence, Finally, we want to mention that a similar approach has been used in the problem of hyperspectral (HS) resolution enhancement in which a HS image is sharpened by a higher-resolution MS or PAN image. In this context, Eismann and Hardie  BIB004 and other authors later (see for instance BIB007 ) proposed to use the model x = S t z + n, where z is the HR original HS image, x is a HRMS or PAN image used to sharpen a LRHS image, S is the spectral response matrix, and n is assumed to be a spatially independent zero-mean Gaussian noise with covariance matrix C. The spectral response matrix is a sparse matrix that contains in each column the spectral response of a MS band of x. Note that in the case of pansharpening, the image x has only one band and the matrix S will be a column vector with components l b as in the model proposed in BIB005 . Once the prior and conditional distributions have been defined, Bayesian inference is performed to find an estimate of the original HRMS image. Different methods have been used in the literature to carry out the inference, depending on the form of the chosen distributions. Maximum likelihood (ML) BIB008 , linear minimum mean square error (LMMSE) BIB007 , maximum a posteriori (MAP) BIB003 , the variational approach  BIB009 and simulated annealing  are some of the techniques used. Bayesian methods usually end up with an estimation of the HRMS image z that is a convex combination of the LRMS image, upsampled to the size of the HRMS image by inverting the degradation model, the PAN image, and the prior knowledge about the HRMS image. The combination factors usually are pixel adaptive and related to the spectral characteristics of the images. Although all approaches already mentioned use the hypothesis of Gaussian additive noise for mathematical convenience, in practice, remote sensing imagery noise shows non-Gaussian characteristics BIB011 . In some applications, such as astronomical image restoration, Poisson noise is usually used, or a shaping filter BIB001 may be used in order to transform non-Gaussian noise into Gaussian. Recently, Niu et al. BIB011 proposed the use of a mixture of Gaussian (MoG) noise for multisensor fusion problems.