Multi-modal and frequency-weighted tensor nuclear norm for hyperspectral image denoising

Low-rankness is important in the hyperspectral image (HSI) denoising tasks. The tensor nuclear norm (TNN), defined based on the tensor singular value decomposition, is a state-of-the-art method to describe the low-rankness of HSI. However, TNN ignores some physical meanings of HSI in tackling denoising tasks, leading to suboptimal denoising performance. In this paper, we propose the multi-modal and frequency-weighted tensor nuclear norm (MFWTNN) and the non-convex MFWTNN for HSI denoising tasks. Firstly, we investigate the physical meaning of frequency components and reconsider their weights to improve the low-rank representation ability of TNN. Secondly, we consider the correlation among two spatial dimensions and the spectral dimension of HSI and combine the above improvements to TNN to propose MFWTNN. Thirdly, we use non-convex functions to approximate the rank function of the frequency tensor and propose the NonMFWTNN to relax the MFWTNN better. Besides, we adaptively choose bigger weights for slices mainly containing noise information and smaller weights for slices containing profile information. Finally, we develop the efficient alternating direction method of multiplier (ADMM) based algorithm to solve the proposed models, and the effectiveness of our models are substantiated in simulated and real HSI datasets.


I. INTRODUCTION
H YPERSPECTRAL image (HSI) has been widely used in many fields due to its wealthy spectral information of real scenes, e.g., target detection [1], unmixing [2], classification [3] and so on. Unfortunately, due to imaging conditions and the influence of weather, observed HSIs are often accompanied by the Gaussian noise, salt and pepper noise, and stripe noise. This not only reduces the quality of HSIs, but also hinders subsequent applications. Therefore, to improve the accuracy of subsequent applications, an important research topic is to recover the clear HSI from the observed HSI in the hyperspectral processing tasks [4,5].
Since the spectral bands are the imaging results of the same scene under different wavelengths, they have global correlation or low-rankness. Based on this prior, HSI can be unfolded into matrices along the spectral mode and then processed by matrix restoration methods. The low-rank matrix recoverybased method restores the clean HSI by minimizing the ranks of unfolding matrices, which is the most popular way [4,[6][7][8]. Zhang et al. [7] propose to disassemble the noisy HSI into the clean HSI and noise information and use the matrix nuclear norm as the relaxation of the rank function. To improve Sheng Liu, Xiaozhen Xie, and Wenfeng Kong are with College of Science, Northwest A&F University, Yangling 712100, China (e-mail: liu sheng@nwafu.edu.cn; e-mail: xiexzh@nwafu.edu.cn (Corresponding author); e-mail: kksama@nwafu.edu.cn. the low-rank performance of the model, some non-convex methods are introduced into the HSI denoising task. Xie et al. [9] use the log sum function, Chen et al. [10] use the γ function, Lou et al. [11] use the L 1 − L 2 function and Gao et al. [12] use the SCAD and MCP function as the nonconvex approximation of the rank function. Low-rank matrix decomposition is also an effective way to solve the low-rank minimization problem [13,14].
However, the unfolding operator destroys the highdimensional structure of HSI. Treating HSI as a third-order tensor can protect this structure, and tensor rank minimization is the most effective way to overcome the above shortcoming. Due to the non-unique definitions of the tensor rank, different tensor decompositions and the corresponding tensor ranks are proposed, e.g., the CANDECOMP/PARAFAC (CP) decomposition [15,16], Tucker decomposition [17,18] and tensor singular value decomposition (t-SVD) [19,20]. CP rank is defined as the minimum of rank-1 tensors by CP decomposition. Whereas, it is difficult to estimate when HSI is unknown. By Tucker decomposition, a tensor can be decomposed into core tensor and modal matrices. Then, Tucker rank is approximated by the sum of modal matrix nuclear norms (SNN) [18]. However, SNN has shortcomings in preserving low-rank structure, and its results are materially sub-optimal [21,22].
Based on t-SVD, the tensor tubal rank has attracted extensive attention recently [23][24][25][26]. Its convex relaxation is the tensor nuclear norm (TNN), which is defined as the sum of the matrix nuclear norms of each frontal slice in the Fourier transformed tensor [19,21]. The minimization of TNN can be quickly solved by convex optimization algorithms, and it is effective in characterizing the low-rankness of tensor. Therefore, it is widely used in HSI denoising task [24,27]. In the definition process of TNN, the information in the image domain of HSI is transformed into frequency information in the Fourier domain. In the Fourier transformed tensor, the lowfrequency slices mainly carry the profile information, while the high-frequency slices mainly carry detail and noise information. Besides, in each frontal slice of the Fourier transformed tensor, bigger singular values mainly contain information of clean data and smaller singular values mainly contain information of noise. Meanwhile, HSIs have correlations among different modes, but TNN lacks the flexibility to explore different correlations along with different modes [25]. Thus, it is advantageous to improve the flexibility of TNN when considering the prior information about the physical meaning of frequency, correlation among different modes, and singular value.
In this paper, to take full advantage of the above prior information and improve the capability and flexibility of model, we propose the multi-modal and frequency-weighted tensor nuclear norm (MFWTNN) and non-convex MFWTNN. In the proposed MFWTNN, we explore the physical meaning of frequency to explore the low-rankness of HSI in the Fourier domain and give an adaptive calculation method of frequency weights. In addition, we consider the correlation among the spatial modes and the spectral mode. Furthermore, based on MFWTNN, we take into account the physical meaning of singular values inside frequency slices and propose the nonconvex approximation of their nuclear norms. Finally, we apply them to the HSI denoising task. The main contributions of this paper are summarized as follows: • Motivated by Tucker decomposition, we consider the correlation among spectral and spatial modes of HSI and use the mode-p permutation of tensor instead of the mode-p unfolding matrix. And then we use their weighted sum to explore the low-rankness of HSI. Furthermore, we combine the advantages of multi modes with the improvement of TNN in the frequency domain and propose MFWTNN in HSI denoising. The MFWTNN is the fusion of TNN and SNN, which inherits the advantages of t-SVD and Tucker decomposition. • In each frontal slice of the Fourier transformed tensor, we choose the log sum function as a non-convex approximation to the rank function, which can provide a better approximation to rank function compared with nuclear norms in MFWTNN. Different from the onedimensional weights of the classical non-convex models, NonMFWTNN considers the weight of frequency direction and singular value direction, which is the twodimensional weight. • According to information types in different frequency slices in the Fourier transformed tensor, we adaptively choose bigger weights for slices mainly containing noise information and smaller weights for slices containing profile information, which can depress noise more and simultaneously preserve the profile information of clean HSI better. • We develop the efficient alternating direction method of multiplier based algorithm to solve the proposed models, and obtain the best restorative performance both on the simulated and real HSI dataset in comparison to all competing HSI denoised methods.

A. Notations
In this section, we follow the work in [28,29] and give some basic notations used about matrix and tensor. We use bold upper-case letter X and calligraphic letter X to denote matrix and tensor, respectively. For a 3rd-order tensor X ∈ R n1×n2×n3 , its (i, j, k)-th component is denoted as X (i, j, k). For X , Y ∈ R n1×n2×n3 , their inner product is defined as n3 k=1 x ijk y ijk . Then the Frobenius norm of a tensor X is computed as n3 k=1 |x ijk |. The k-th frontal slice of X is represented as X (k) = X (:, :, k). For one-dimensional vector v, its fast Fourier transform is denoted asv = fft(v) ∈ C N , and inverse transformation is represented as v = ifft(v). For three-dimensional tensor X , its fast Fourier transform is represented asX = fft(X , [], 3) and its inverse operation is X = ifft(X , [], 3). The mode-p permutation of X is defined as X p = permute(X , p), p = 1, 2, 3, where the mth mode-3 slice of X p is the m-th mode-p slice of X , i.e., X (i, j, k) = X 1 (j, k, i) = X 2 (k, i, j) = X 3 (i, j, k). Also, its inverse operation is X = ipermute(X p , p).

B. Problem Formulation
A clean HSI can be treated as a third-order tensor X ∈ R n1×n2×n3 and is usually assumed to be low-rank. Corrupted by mixed noise, its observed version can be modeled as where Y, S, N ∈ R n1×n2×n3 ; S denotes the sparse noise; N denotes the Gaussian white noise. HSI denoising aims to recover the clean HSI X from the observed HSI Y in (1). Under the framework of regularization theory, it can briefly be formulated as arg min where · 1 describes the sparse noise; · F describes the Gaussian noise; Rank(·) represents the rank of unknown ideal HSI; λ and τ are non-negative parameters.

C. Tensor Nuclear Norm
Since the tensor rank function is non-convex and discrete, its rank minimization problem is NP-hard [30]. To overcome computational difficulties, a usual approach is to approximate the non-convex rank function to the convex function and then substitute it into the minimization problem. Fazel [31] proves that the matrix nuclear norm is the convex envelope rank function on the unit sphere of the spectral norm. Thereafter, the nuclear norm is used as a convex replacement of the rank function. The rank minimization problem is transformed into nuclear norm minimization, and it is widely used in low-rank matrix recover based HSI denoising tasks [7].
Furthermore, HSI can be regarded as a third-order tensor. Then, based on t-SVD [19], Lu et al. [21] propose the tensor nuclear norm (TNN) as the relaxation function of the tensor rank function. TNN is defined as whereX (k) is the k th frontal slice ofX . Due to the property of fft, we have Based on (4), frequency component [26] is defined as if i = 2, · · · , n3+1 2 (5) Fig. 1 shows the relationship between the frontal slice and the frequency component. Due to TNN can be effective in expressing the low-rank characteristics, it has received extensive attention in the HSI denoising task. However, it focuses on the low-rankness in the spectral dimension. For HSI, its low-rank structure exists with multiple dimensions [29,32]. To connect the correlation along different dimensions, Liu et al. [18] use modal matrices of the tensor in different directions to explore low-rank structures, and then use SNN as the convex relaxation of the rank function. SNN is defined as where X p is the modal matrix that unfold X along the direction of the mode-p, α p is weighting parameter of modes and 3 p=1 α p = 1. However, this way will inevitably destroy the internal structure of HSI. Zheng et al. [25] propose the weighted sum of TNN (WSTNN). They consider the lowrankness of the three modes, which are two spatial modes and one spectral mode. WSTNN is defined as where X p is the mode-p permutation of X .
III. MULTI-MODAL AND FREQUENCY-WEIGHTED TENSOR NUCLEAR NORM AND ITS NON-CONVEX APPROXIMATION.

A. Multi-modal and Frequency-weighted Tensor Nuclear Norm
In the definition of TNN (3), fft plays a core important role in the process of transforming t-products into matrix multiplication in the Fourier domain. It makes TNN to have fast calculation capabilities. However, the role of fft is not just for fast computation in the framework of t-SVD. When Fourier matrix F n is treated as a transform matrix,X is the feature tensor of X . To improve the low-rank representation ability of TNN, Wang et al. [26] propose the frequency-filtered tensor nuclear norm (FTNN), which is defined as is the sum of two matrix nuclear norms, I denotes the number of frequency bands, and α k is a pre-defined parameter assigned to the k-th frequency band, whose value depends on the prior knowledge. They treat the FTNN as a frequency filtering method. α k characterizes the filtering coefficient for the kth frequency component. However, these parameters should be data-dependent, and the pre-weight parameters cannot well reflect the real physical meaning between filter coefficient and frequency component. For the HSI, it is difficult to give n3+1 2 pre-weight parameters in advance. Therefore, we reconsider these weight parameters based on the physical properties of HSI. For the clean Washington DC Mall dataset, sparse noise is added to form clean data, and then the zerofrequency component and non-zero frequency component of the noise data are separated. Fig. 2 shows the grayscale image of the 60th band. We can find an interesting phenomenon that just using the information of the zero frequency component can get satisfactory results without any processing and the noise is more concentrated in the non-zero frequency component. As the transform operator, fft plays the role of information separation, which makes the image information and noise information as separate as possible. The distribution of the nuclear norm shown in Fig. 2(e) is consistent with the observed phenomenon. When the HSI is polluted by noise, the variation in the high-frequency component is bigger than that in the low-frequency component. There is a natural idea that we can reduce the shrinkage of the low-frequency component and increase the penalty of the high-frequency component. Therefore, base on FTNN, we redefine the frequency-weighted tensor nuclear norm (FWTNN): Our weights are different from the weights of FTNN, which are not pre-weight but data dependent. They are discussed in Section III-C.
In the definition of FWTNN, by calculating the nuclear norm along the spectral direction, fft can capture low rank of the HSI spectrum. However, the low-rankness of HSI exists not only in the spectral dimension, but also in the spatial dimensions [18,32]. Therefore, fft is insufficient in exploring intra-mode and inter-mode correlations of HSI. Besides, in the HSI denoising task, the HSI band may be polluted by stripe noise in the same direction. In a low-rank model that only considers the spectral dimension, these stripe noises will be regarded as part of the low-rank image and are hard to remove. In the frequency domain, for the same HSI, the information expressed by the permuted versions X p in different directions is even more different. Therefore, motivated by SNN [18], it is necessary to consider the low-rankness of different dimensions for HSI denoising task. Different from SNN, to better retain the high-dimensional structures of HSI, its modep permutation X p to replace its mode-p unfolding matrix X p .
Based on the above analysis, we propose the multi-modal and frequency-weighted tensor nuclear norm (MFWTNN) as follows: is the k-th frontal slice of X p and its assigned weight is w p k ; α p > 0 and 3 p=1 α p = 1.

B. Non-convex Multi-modal and Frequency-weighted Tensor Nuclear Norm
In the same frequency band, its singular values are treated equally. However, the major information of HSI, such as smooth zones and profile, is contained in the larger singular value; the noise information of HSI is contained in the smaller singular value [9][10][11]33]. To combine both the frequency and the singular value prior information, we use the log norm to perform non-convex relaxation of the nuclear norm in the frequency band to more accurately describe the tensor rank function [9]. Thus, we propose the non-convex multi-modal and frequency-weighted tensor nuclear norm (NonMFWTNN) to more accurately describe the low-rankness of HSI. It is defined as follows: where log Fig. 3 shows the mechanism of Non-MFWTNN. As can be seen from Fig. 3, NonMFWTNN takes into account the relationship between the singular values inside and outside of frequency band. The weights of classic matrix / tensor non-convex models [9,10,29,34] are one-dimensional, which only consider the relationship between the singular values inside the matrix. The way is equivalent to pulling the core singular value matrix S p in Fig. 3(c) into a column vector and using the weights as the coefficient of this column vector. The weight of MFWTNN is to make a Cartesian product of the weight vector W F in the frequency direction and the weight vector W S in the singular value direction, which are twodimensional weights. If W F = 1, it degenerates to classical one-dimensional weights.

C. Frequency-weighted parameter
In the definition of MFWTNN and NonMFWTNN, the values of the frequency-weighted parameter w p k are important. The ideal w p k can take smaller values that reduce the penalty for the low-frequency slice nuclear norm to protect the main information of HSI in the low-frequency band, and take larger values that increase the penalty for the high-frequency slice nuclear norm to fully remove the noise in the high-frequency band. It can be seen from Fig. 4(a) that the nuclear norm is opposite to the distribution characteristics of w p k in the frequency domain. It takes larger values in the low-frequency band and takes smaller values in the high-frequency band. And the value of the nuclear norm decreases as the frequency increases. However, the calculation of the nuclear norm brings high cost, which requires singular value decomposition. In a finite dimensional space, any two norms are equivalent. This shows that when transforming from the nuclear norm to the Frobenius norm, they are equal under the normalization condition. Thus, the reciprocal of the Frobenius norm of each frequency slice matrix can be used as w p k .
where δ = 10 −6 . The purpose of using log function is to shrink the scale of the value without changing the trend of the Frobenius norm to prevent the extreme influence of the maximum value. The above method of calculating weights is feasible when the clean HSI is known. However, for HSI denoising tasks, we often only have noisy HSI, and clear HSI is unknown. It can be seen from Fig. 4(b) that the weights calculated by the denoised HSI is closer to the result obtained by the clean HSI than the noisy HSI. Further, as shown in Fig. 3(e) , as the number of iterations n continues to increase, the weights calculated by the denoised HSI will get closer to the result of clean HSI. Based on this, we propose an iterative calculation method to update w p k . Assuming that the nth iteration getsX where h p k,n = 1/(log X (k,n) p 2 F + δ). C 1 is the scaling factor after frequency normalization; C 2 is a smaller constant. Affected by noise, there is less useful information from the denoised HSI in the initial iteration. If it is not corrected, the denoised result will develop in a bad direction. Since TNN is a robust model, inspired by this, we introduce a constant C 2 to improve our weights. When C 1 =0, it degenerates into TNN.
Another reason we introduced C 2 is that noise exists in all frequency bands. The previous conclusion is that there is less noise information in the low-frequency part instead of none. C 2 can achieve a slight shrinkage of all frequency bands.

IV. HSI DENOISING VIA MFWTNN AND NONMFWTNN MINIMIZATION
A. Proposed model MFWTNN uses frequency components, modal information, and NonMFWTN uses the physical meaning of singular value distribution on the basis of MFWTNN. They can provide a better approximation to the tensor rank. Then we use MFWTNN and NonMFWTNN to replace the regularization term Rank in (2) and propose the HSI denoising model as follows: arg min arg min Introducing auxiliary variables, model (14) and (15) are equivalent to arg min arg min By augmented Lagrangian multiplier method, the Lagrangian function of model (16) can be written as

B. Algorithmic optimization
ADMM is an effective framework to solve these type of minimization problems [35]. When other variables are fixed at the n-th iteration, each variable in the Lagrangian function can be updated by solving its corresponding subproblem respectively at the (n + 1)-th iteration.
For Z p , p = 1, 2, 3, their corresponding subproblems can be written as The closed-form solution of (19) obtained from theorem 1 of [26] are as follows: Similarly, when (17), the sub-problem of Z p is rewritten as The closed-form solution of (21) obtained from theorem 1 are as follows: For X , its corresponding subproblem can be reformulated as X can be updated as follows: For S, its corresponding subproblem can be reformulated as It can be solved by the soft-thresholding operator [36] as: N can be updated as follows : For multipliers Γ p and Λ, they can be updated as follows:

C. Time Complexity Analysis
The HSI denoising models based on MFWTNN and NonMFWTNN can be solved by Algorithm 1. Further, we discuss the time complexity of this algorithm. Computing Z p , p = 1, 2, 3, in both MFWTNN-based solver and NonMFWTNN based solver, has a complexity of O(n 1 n 2 n 3 log(n 1 n 2 n 3 ) + 3 i=1 max(n i , n i+1 ) min 2 (n i , n i+1 )n i+2 ), where n 4 = n 1 , n 5 = n 2 . Calculating w k has a complexity of O( 3 i=1 min (n i , n i+1 ) n i+2 ). The cost of computing X , S, and N are all O(n 1 n 2 n 3 ). Then, by calculating the complexity of the above variables, the total complexity of the proposed algorithm can be obtained as O(n 1 n 2 n 3 log (n 1 n 2 n 3 ) + 3 i=1 max (n i , n i+1 ) min 2 (n i , n i+1 )n i+2 + 3 i=1 min (n i , n i+1 )n i+2 + 3n 1 n 2 n 3 ).

V. EXPERIMENT RESULTS AND DISCUSSION
In this section, we conduct experiments on simulated and real-world HSIs to substantiate the effectiveness of the proposed MFWTNN and NonMFWTNN model for HSI denoising. For evaluating the effectiveness of the models, our models are compared with four state-of-the-art HSI denoised methods i.e.BM4D [37], LRMR [7], LRTDTV [32], 3DLogTNN [29]. Due to the BM4D methods that are only suitable for removing Gaussian noise, we first use the RPCA [21] model to remove the sparse noise, and then use it to obtain the denoised results. Before denoising all models, all pixels in the HSI band are linearly mapped to [0,1]. In all experiments, the hyper-parameters in these comparative methods are manually adjusted according to the suggestions in relevant papers. In Section V-C, the parameters are discussed in detail in our proposed models. We use visual comparison and quantitative comparison to comprehensively evaluate the performance of different denoising methods. In quantitative comparisons, we use five quantitative picture quality indices (PQIs), i.e., PSNR [38], SSIM [39], FSIM [40], ERGAS [41] and SAM [42], to assess the denoised results. The better denoised results correspond to higher values in MPSNR, MSSIM, MFSIM, and lower values in EGRAS, MSAM.

A. Simulated HSI Data Experiments
In this section, we select two HSI datasets for the simulated experiments to show the robustness of our methods. One is filmed by the reflection optical system imaging spectrometer (ROSIS-03) from Pavia City Center dataset 1 . The size is 1096×1096, with a total of 102 bands. The reason we chose a sub-block with a size of 200×200×80 as a simulation dataset is because some bands are seriously polluted by noise and have lost the meaning of reference. The other one is collected by a HSI acquisition sensor (HYDICE) from Washington DC Mall dataset 2 . The size is 1208 × 307, with a total of 191 bands. we choose a sub-block with a size of 256 × 256 × 191 as a simulation dataset.
The observed HSIs are degraded by the hybrid noises, i.e. Gaussian noise, sparse noise, and stripe noise. Thus, to better simulate the degradation mechanism of HSI, we add various intensity of hybrid noises to the clean HSI. In the simulation experiments, the hybrid noises with eight different intensity levels are added to simulation dataset band by band. Let G and P denote the variance of Gaussian white noise and percentage of impulse noise, respectively. In noise cases 1-5, the same intensity noise is added to all the bands. In noise case 1, G=0.1 and P =0.2; In noise case 2, G=0.1 and P =0.3; In noise case 3, G=0.1 and P =0.4; In noise case 4, G=0.15 and P =0.2; In noise case 5, G=0.2 and P =0.2; In noise cases 6-8, the noise intensities are different for different bands. In noise case 6, G=0.1 and P is randomly selected from 0.2 to 0.4. In noise case 7, G is randomly selected from 0.1 to 0.3 and P =0.2; In noise case 8, G is randomly selected from 0.1 to 0.3 and P is randomly selected from 0.1 to 0.3. In addition, stripe noises are added to the 54-64 bands of the Pavia City Center and the 70-100 bands of the Washington DC Mall. In this part, we show the evaluation results of all denoising models including visual and quantitative quality on Pavia City Center and Washington DC Mall datasets. Table I reports the denoised results and CPU running time by all the compared methods under the condition of the eight noise cases on simulated dataset. When compared to the other methods, our proposed methods obtain the optimal PQIs among all the denoising models in most noise cases, indicating the advantage of the proposed methods in HSI denoising. Although LRTDTV also consider the low-rankness between different modes, it is affected by the matrix unfold operation, which destroy the connection between the modes. Therefore, LRTDTV models obtain the lower PQIs. BM4D and LRTDTV use non-local similarity information and spatial smoothing information respectively. These make the denoised result excessively smooth and lack texture details. Although 3DlogTNN considers the correlation between different modes, it is still not accurate enough to represent the low-rankness of HSI. MFWTNN use the physical meaning of singular values and frequency components to improve TNN respectively. But it is not the best descriptions of the low-rankness of HSI. NonMFWTNN inherits the advantages of 3DLogTNN and MFWTNN, and at the same time improves the ability to express low-rankness of HSI in terms of frequency components and singular values. Although the time complexity of our models is higher, our models have obtained better denoised results.
In Fig. 5, we show the denoised results of the Pavia City Center and Washington DC Mall dataset in terms of PSNR and SSIM under all noisy case experiments. As this figure shows, our methods obtain the optimal PSNR and SSIM values in most bands. In Fig. 6, we show the 50th band of the denoised results of all comparison methods in case 2 on Pavia City Center dataset. Also in Fig. 7, We show the 57th band of the denoised results in case 7 on Washington DC Mall dataset. As these grayscale images show, LRMR cannot completely remove some high-intensity noises and retain more noise. For BM4D and LRTDTV, although they can remove more noise, they also lose more details. This makes the denoised result too smooth. Compared with them, our proposed models can remove more noise while retaining more details. The spectral signatures are important indicators for HSI. To further compare the denoised image quality, we show the original and denoised spectral signatures curves. Therefore, we show the spectral curve of the pixel (170, 99) under case 1 and pixel (234,171) under case 6 in Fig. 8. It can be seen that spectral signatures curves are relative to ERGAS values in table I, and our models can obtain the optimal value.

B. Real HSI Data Experiments
In this subsection, we chose the real HSI dataset in this experiment. It is AVIRIS Indian Piness dataset 3 . It is filmed by the Airborne Visible Infrared Imaging Spectrometer (AVIRIS). The reason we chose a sub-block with a size of 145×145×220 3 https://engineering.purdue.edu/ as a real dataset is because some bands are seriously polluted by Gaussian white noise and impulse noise and have lost the meaning of reference. We evaluate our denoising model from visual evaluation and vertical average profile. In Fig.  9, we show the 105th band of the observed and denoised results in Indian dataset. It can be seen that it is seriously polluted by Gaussian and sparse noise, and the original ground features information cannot be distinguished. Although the competitive denoising methods restore the feature information of HSI, the denoised HSIs still lose detailed information or residual noise, resulting in missing or blurred texture details of HSI. Our models not only restore more feature information of HSI but also remove more local noise than the competitive methods. Therefore, these denoised HSIs have clearer texture information. In Fig. 10, we show the vertical mean profiles of the 50th band of Indian dataset. Due to the influence of hybrid noises, the vertical mean profiles of the observed HSI shows rapid fluctuations. This means that the smaller fluctuations in the vertical mean profiles, the higher quality of the denoised HSI. Obviously, the mean profile curves obtained by our model are the most stable and its fluctuation is the smallest. This result is relative to the facts reflected in Fig. 9.
C. Discussion 1) Parameter Analysis: In the NonMFWTNN model, five parameters need to be discussed, i.e., the modal weight parameter α, the regularization parameters λ, τ and frequency weight parameter C 1 , and C 2 . All experiments are done under Case 2.
The modal weight α is to control the ratio between different modes. The low-rank characteristics of the spatial dimensions are similar. To reduce the parameters, we set the weights of the two spatial dimensions to equal weights. Therefore, it is set to α = [1, 1, α 3 ]/(1 + 1 + α 3 ), which makes 3 i=1 α i = 1. We study the impact of α 3 which is selected from {0.1 − 1.0} on the robustness of the model, and its optimal value is obtained at 0.2. It can be observed from Fig. 11(a) that when α 3 takes a smaller value, the PSNR value of the denoised result is better. This is because the low-rank characteristic of the spectral mode is stronger than the spatial mode. The regularization parameter τ controls the weight of the Gaussian noise term. Its value is related to the intensity of Gaussian noise. Therefore, it is set to τ = τ n /σ, where σ is Gaussian noise intensity. We study the impact of τ n which is selected from {10 −5 − 1.6 × 10 −4 } on the robustness of the model, and its optimal value is obtained at 10 −4 . It can be observed from Fig. 11(c) that when τ ≤ 0.0001, the MPSNR of the denoised result is low and fluctuates greatly. The main reason is that when τ is small, the ability to remove Gaussian noise is weak, and there is a lot of remaining Gaussian noise. The regularization parameter λ controls the weight of the sparse noise term. Defining λ = λ s ( α1 √ max(n2,n3)n1 ).
As observed from Fig. 11(b), our model can obtain remarkable results when λ s which is selected from {0.006 − 0.016}. its optimal value is obtained at 0.011. Similar to τ , when λ is small, the noise removal effect is not good. The frequency weight parameter C 1 reflects the scaling ratio of the weight. Fig. 11(d) show the PSNR values when C 1 which is selected from {0.1 − 1.0}. It can be seen from these figures that when C 1 is small, the frequency weight becomes smaller, and the noise is more difficult to remove; when C 1 is large, the frequency weight becomes larger, and the detailed information of tensor is removed. its optimal value is obtained at 0.6. The frequency weight parameter C 2 reflects the initial weight of the iteration. Fig. 11(e) show the PSNR values when C 2 which is selected from {0.1 − 1.0}. its optimal value is obtained at 0.6. It shrinks equally of all frequency components. It inherits the advantages of TNN and ensures that noise can be removed correctly.
2) Convergence Analysis: Fig.12 shows the MPSNR value, the MSSIM value, and the Error value of the proposed model according to the iterations. As shown in Fig.12, the number of iterations increases, the value gradually stabilizes by the proposed method, which justifies the numerical convergence of the proposed method.

VI. CONCLUSION
In this paper, we proposed the multi-modal and frequencyweighted tensor nuclear norm (MFWTNN) and Non-convex MFWTNN to approximate the rank function and applied them to the HSI denoising task. The MFWTNN not only considered the correlation between two spatial modes and one spectral mode but also considered the frequency meaning of HSI in the Fourier domain. The MFWTNN is the fusion of TNN and SNN, which inherits the advantages of t-SVD and Tucker decomposition. And we gave the frequency weight adaptive calculation way. Since the calculation of weights depends on iteratively restored and observed HSI, our models are more robust to different degrees of noise. It can better characterize the low-rankness of HSI. Besides, based on the MFWTNN model, we also considered the physical meaning of the internal singular values of the frequency slice and proposed its nonconvex approximation. They powerfully improved capability and flexibility for describing low-rankness in HSIs. The ex-  periments conducted with both simulated and real HSI datasets showed that our proposed models based HSI denoising model were competitive methods to remove the hybrid noises.