the analysis of multivariate time series data is of increased interest and importance in the modern information age .although the methods and the associate theory for univariate time series analysis are well developed and understood , the picture for the multivariate cases is less complete . in spite of the fact that the conventional univariate time series models ( such as arma ) and the associated time - domain and frequency - domain methods have been formally extended to multivariate cases ,their usefulness is often limited .one may face serious issues such as the lack of model identification or flat likelihood functions .in fact vector arma models are seldom used directly in practice .dimension - reduction via , for example , reduced - rank structure , structural indices , scalar component models and canonical correlation analysis is more pertinent in modeling multivariate time series data . see  .  in this paperwe deal with the factor modeling for multivariate time series from a dimension - reduction viewpoint . differently from the factor analysis for independent observations , we search for the factors which drive the serial dependence of the original time series . early attempts in this direction include .more recent efforts focus on the inference when the dimension of time series is as large as or even greater than the sample size ; see , for example ,  and the references within .high - dimensional time series data are often encountered nowadays in many fields including finance , economics , environmental and medical studies .for example , understanding the dynamics of the returns of large numbers of assets is the key for asset pricing , portfolio allocation , and risk management .panel time series are commonplace in studying economic and business phenomena .environmental time series are often of a high dimension due to a large number of indices monitored across many different locations .our approach is from a dimension - reduction point of view .the model adopted can be traced back at least to that of .we decompose a high - dimensional time series into two parts : a dynamic part driven by , hopefully , a lower - dimensional factor time series , and a static part which is a vector white noise .since the white noise exhibits no serial correlations , the decomposition is unique in the sense that both the number of factors ( i.e. , the dimension of the factor process ) and the factor loading space in our model are identifiable .such a conceptually simple decomposition also makes the statistical inference easy .although the setting allows the factor process to be nonstationary ( see  ; also section [ sec31 ] below ) , we focus on stationary models only in this paper : under the stationary condition , the estimation for both the number of factors and the factor loadings is carried out in an eigenanalysis for a nonnegative definite matrix , and is therefore applicable when the dimension of time series is on the order of a few thousands .furthermore , the asymptotic properties of the proposed method are investigated under two settings : ( i ) the sample size goes to infinity while the dimension of time series is fixed ; and ( ii ) both the sample size and the dimension of time series go to infinity together . in particular , our estimators for zero - eigenvalues enjoy the faster convergence ( or slower divergence ) rates , from which the proposed ratio - based estimator for the number of factors benefits .in fact when all the factors are strong , the performance of our estimation for the number of factors improves when the dimension of time series increases .this phenomenon is coined as `` blessing of dimensionality . ''the new contributions of this paper include : ( i ) the ratio - based estimator for the number of factors and the associated asymptotic theory which underpins the `` blessing of dimensionality '' phenomenon observed in numerical experiments , and ( ii ) a two - step estimation procedure when the factors are of different degrees of strength .we focus on the results related to the estimation for the number of factors in this paper .the results on the estimation of the factor loading space under the assumption that the number of factors is known are reported in  .there exists a large body of literature in econometrics and finance on factor models for high - dimensional time series .however , most of them are based on a different viewpoint , as those models attempt to identify the _ common factors _ that affect the dynamics of most original component series . in analyzing economic and financial phenomena, it is often appealing to separate these common factors from the so - called idiosyncratic components : each idiosyncratic component may at most affect the dynamics of a few original time series .an idiosyncratic series may exhibit serial correlations and , therefore , may be a time series itself .this poses technical difficulties in both model identification and inference .in fact the rigorous definition of the common factors and the idiosyncratic components can only be established asymptotically when the dimension of time series tends to infinity ; see  .hence those factor models are only asymptotically identifiable . according to the definition adopted in this paper ,both `` the common factors '' and those serially correlated idiosyncratic components will be identified as factors .this is not ideal for the applications with the purpose to identify those common factors .however , this makes the tasks of model identification and inference much simpler .the rest of the paper is organized as follows .the model and the estimation methods are introduced in section [ sec3 ] .the sampling properties of the estimation methods are investigated in section [ sec5 ] .simulation results are inserted whenever appropriate to illustrate the various asymptotic properties of the methods . section [ sec6 ] deals with the cases when different factors are of different strength , for which a two - step estimation procedure is preferred .the analysis of two real data sets is reported in section [ sec7 ] .all mathematical proofs are relegated to the .if we are interested in the linear dynamic structure of only , conceptually we may think that consists of two parts : a static part ( i.e. , a white noise ) , and a dynamic component driven by , hopefully , a low - dimensional process .this leads to the decomposition : where is an latent process with ( unknown ) , is a unknown constant matrix , and is a vector white - noise process . when is much smaller than , we achieve an effective dimension - reduction , as then the serial dependence of is driven by that of a much lower - dimensional process .we call a factor process . the setting ( [ c1 ] )may be traced back at least to  ; see also its further development in dealing with cointegrated factors in  . since none of the elements on the rhs of ( [ c1 ] )are observable , we have to characterize them further to make them identifiable .first we assume that no linear combinations of are white noise , as any such components can be absorbed into  [ see condition ( c1 ) below ] .we also assume that the rank of is .otherwise ( [ c1 ] ) may be expressed equivalently in terms of a lower - dimensional factor .furthermore , since ( [ c1 ] ) is unchanged if we replace by for any invertible matrix  , we may assume that the columns of are orthonormal , that is , , where denotes the identity matrix .note that even with this constraint , and are not uniquely determined in ( [ c1 ] ) , as the aforementioned replacement is still applicable for any orthogonal .however , the factor loading space , that is , the -dimensional linear space spanned by the columns of , denoted by , is uniquely defined .we summarize into condition ( c1 ) all the assumptions introduced so far :  in model ( [ c1 ] ) , .if is white noise for a constant , then for any nonzero integers . furthermore .the key for the inference for model ( [ c1 ] ) is to determine the number of factors and to estimate the factor loading matrix , or more precisely the factor loading space .once we have obtained an estimator , say ,  , a natural estimator for the factor process is and the resulting residuals are the dynamic modeling for is achieved via such a modeling for and the relationship . a parsimonious fitting for be obtained by rotating  .such a rotation is equivalent to replacing by for an appropriate orthogonal matrix .note that , and the residuals ( [ c3 ] ) are unchanged with such a replacement.=1   an innovation expansion algorithm is proposed in  for estimating based on solving a sequence of nonlinear optimization problems with at most variables .although the algorithm is feasible for small or moderate only , it can handle the situations when the factor process is nonstationary .we outline the key idea below , as our computationally more efficient estimation method for stationary cases is based on the same principle .our goal is to estimate , or , equivalently , its orthogonal complement , where is a matrix for which forms a orthogonal matrix , that is , and [ see also ( c1 ) ] .it follows from ( [ c1 ] ) that implying that for any , is a white - noise process .hence , we may search for mutually orthogonal directions one by one such that the projection of on each of those directions is a white noise .we stop the search when such a direction is no longer available , and take as the estimated value of , where is the number of directions obtained in the search .this is essentially how  accomplish the estimation .it is irrelevant in the above derivation if is stationary or not .however , a much simpler method is available when , therefore also , is stationary :   is weakly stationary , and cov for any .  in most factor modeling literature , and are assumed to be uncorrelated for any and .condition ( c2 ) requires only that the future white - noise components are uncorrelated with the factors up to the present .this enlarges the model capacity substantially .put it follows from ( [ c1 ] ) and ( c2 ) that for a prescribed integer , define then is a nonnegative matrix .it follows from ( [ d2 ] ) that , that is , the columns of are the eigenvectors of corresponding to zero - eigenvalues .hence conditions ( c1 ) and ( c2 ) imply :  we take the sum in the definition of to accumulate the information from different time lags .this is useful especially when the sample size is small .we use the nonnegative definite matrix [ instead of to avoid the cancellation of the information from different lags .this is guaranteed by the fact that for any matrix , if and only if for all .we tend to use small , as the autocorrelation is often at its strongest at the small time lags . on the other hand , adding more terms will not alter the value of , although the estimation for with large  is less accurate .the simulation results reported in  also confirm that the estimation for and , defined below , is not sensitive to the choice of  .to estimate , we only need to perform an eigenanalysis on where denotes the sample covariance matrix of at lag .then the estimator for the number of factors is defined in ( [ d5 ] ) below .the columns of the estimated factor loading matrix are the orthonormal eigenvectors of corresponding to its largest eigenvalues .note that the estimator  is essentially the same as that defined in section 2.4 of , although a canonical form of the model is used there in order to define the factor loading matrix uniquely .due to the random fluctuation in a finite sample , the estimates for the zero - eigenvalues of are unlikely to be 0 exactly .a common practice is to plot all the estimated eigenvalues in a descending order , and look for a cut - off value such that the largest eigenvalue is substantially smaller than the largest eigenvalues .this is effectively an eyeball - test .the ratio - based estimator defined below may be viewed as an enhanced eyeball - test , based on the same idea as  .in fact this ratio - based estimator benefits from the faster convergence rates of the estimators for the zero - eigenvalues ; see proposition [ prop1 ] in section [ sec51 ] below , and also theorems [ thm1 ] and [ thm2 ] in section [ sec52 ] below .the other available methods for determining include the information criteria approaches of  and  , and the bootstrap approach of  , though the settings considered in those papers are different .  _a ratio - based estimator for . we define an estimator for the number of factors as follows : where are the eigenvalues of , and is a constant .  in practicewe may use , for example , .we can not extend the search up to , as the minimum eigenvalue of is likely to be practically 0 , especially when  is small and  is large .it is worthy noting that when  and  are on the same order , the estimators for eigenvalues are no longer consistent .however , the ratio - based estimator ( [ d5 ] ) still works well. see theorem [ thm2](iii ) below .the above estimation methods for and can be extended to those nonstationary time series for which a generalized lag- autocovariance matrix is well defined ( see , e.g. ,  ) .in fact , the methods are still applicable when the weak limit of the generalized lag- autocovariance matrix exists for , where is a constant .further developments on those lines will be reported elsewhere . for the factor modeling for high - dimensional volatility processes based on a similar idea, we refer to  .conventional asymptotic properties are established under the setting that the sample size tends to and everything else remains fixed .modern time series analysis encounters the situation when the number of time series is as large as , or even larger than , the sample size . then the asymptotic properties established under the setting when both and tend to are more relevant .we deal with these two settings in section [ sec51 ] and sections [ sec52][sec525 ] separately .we first consider the asymptotic properties under the assumption that and is fixed .these properties reflect the behavior of our estimation method in the cases when  is large and is small .we introduce some regularity conditions first .let be the eigenvalues of the matrix :   is strictly stationary and -mixing with the mixing coefficients  satisfying the condition that .furthermore , element - wisely . .section 2.6 of  gives a compact survey on the mixing properties of time series .the use of the -mixing condition in ( c3 ) is for technical convenience .note that is a nonnegative definite matrix .all its eigenvalues are nonnegative .condition ( c4 ) assumes that its nonzero eigenvalues are distinct from each other .while this condition is not essential , it substantially simplifies the presentation of the convergence properties in proposition [ prop1 ] below .let be a unit eigenvector of corresponding to the eigenvalue .we denote by the pairs of eigenvalue and eigenvector of matrix : the eigenvalues are arranged in descending order , and the eigenvectors are orthonormal .furthermore , it may go without explicit statement that may be replaced by in order to match the direction of for .[ prop1 ] let conditions hold . then as ( but fixed ) , it holds that :   and for , and   for .the proof of the above proposition is in principle the same as that of theorem 1 in  , and is therefore omitted .   to highlight the radically different behaviorwhen diverges together with , we first conduct some simulations : we set in model ( [ c1 ] ) , , are independent , and is an ar(1 ) process defined by we set the sample size , and 3200 , and the dimension fixed at half the sample size , that is , .let be defined as in ( [ d3 ] ) with .for each setting , we draw 200 samples. the boxplots of the errors , , are depicted in figure [ largep ] .note that for , since .   with and all the factor loading coefficients being 1 .]  the figure shows that those estimation errors do not converge to 0 .in fact those errors seem to increase when ( and also ) increases .therefore the classical asymptotic theory ( i.e. , and fixed ) such as proposition [ prop1 ] above is irrelevant when increases together with . in spite of the lack of consistency in estimating the eigenvalues , the ratio - based estimator for the number of factors ( ) defined in ( [ d5 ] ) works perfectly fine for this example , as shown in figure [ fig5 ] .in fact it is always the case that in all our experiments even when the sample size is as small as ; see figure [ fig5 ] ., with and all the factor loading coefficients being 1 . ]  to develop the relevant asymptotic theory , we introduce some notation first . for any matrix ,let be the square root of the maximum eigenvalue of , and be the square root of the smallest nonzero eigenvalue of .we write if and .recall and .some regularity conditions are now in order :  for a constant  ] are constants .we take as a measure of the strength of the factor .we call a strong factor when , and a weak factor when . since is fixed , it is also reasonable to assume that for , under conditions ( [ e3 ] ) and ( [ e2 ] ) with for all .( ii ) the condition assumed on in ( c6 ) requires that the correlation between ( ) and is not too strong .in fact under a natural condition that element - wisely , it is implied by ( [ e2 ] ) and the standardization [ hence now as a result of such standardization ] that .now we deal with the convergence rates of the estimated eigenvalues , and establish the results in the same spirit as proposition [ prop1 ] .of course the convergence ( or divergence ) rate for each estimator is slower , as the number of estimated parameters goes to infinity now .[ thm1 ] let conditions hold and . then as and , it holds that :   for , and   for .[ cor0 ] under the condition of theorem [ thm1 ] , it holds that   the proofs of theorem [ thm1 ] and corollary [ cor0 ] are presented in the .obviously when is fixed , theorem [ thm1 ] formally reduces to proposition [ prop1 ] .some remarks are now in order .[ remark2 ] ( i ) corollary [ cor0 ] implies that the plot of ratios , will drop sharply at .this provides a partial theoretical underpinning for the estimator defined in ( [ d5 ] ) .especially when all factors are strong ( i.e. , ) , .this convergence rate is independent of , suggesting that the estimation for may not suffer as increases .in fact when all the factors are strong , the estimation for may improve as increases .see remark [ remark3](iv ) in section [ sec525 ] below .\(ii ) unfortunately , we are unable to derive an explicit asymptotic expression for the ratios with , although we make the following conjecture : where is the number of lags used in defining matrix in ( [ d3 ] ) , and is any fixed integer .see also figure [ fig5 ] .further simulation results , not reported explicitly , also conform with ( [ e4 ] ) .this conjecture arises from the following observation : for , the largest eigenvalue of  is predominately contributed by the term which has a cluster of largest eigenvalues on the order of , where is the sample lag- autocovariance matrix for .see also theorem [ thm2](iii ) in section [ sec525 ] below .\(iii ) the errors in estimating eigenvalues are on the order of or , and both do not necessarily converge to 0 .however , since the estimation errors for the zero - eigenvalues is asymptotically of an order of magnitude smaller than those for the nonzero - eigenvalues .   to illustrate the asymptotic properties in section [ sec52 ] above , we report some simulation results .we set in model ( [ c1 ] ) , , and 3200 , and , and .all the elements of are generated independently from the uniform distribution on the interval $ ] first , and we then divide each of them by  to make all three factors of the strength  ; see ( [ e2 ] ) .we generate factor  from a vector - ar(1 ) process with independent innovations and the diagonal autoregressive coefficient matrix with 0.6 , and 0.3 as the main diagonal elements .we let in ( [ c1 ] ) consist of independent components and they are also independent across .we set in ( [ d3 ] ) and ( [ d4 ] ) .for each setting , we replicate the simulation 200 times ..3d1.3cc@ & & * 50 * & * 100 * & * 200 * & & & & * 3200 * + & & 0.165 & 0.680 & 0.940 & 0.995 & 1 & 1 & 1 + & & 0.410 & 0.800 & 0.980 & 1 & 1 & 1 & 1 + & & 0.560 & 0.815 & 0.990 & 1 & 1 & 1 & 1 + & & 0.590 & 0.820 & 0.990 & 1 & 1 & 1 & 1 + [ 4pt ] & & 0.075 & 0.155 & 0.270 & 0.570 & 0.980 & 1 & 1 + & & 0.090 & 0.285 & 0.285 & 0.820 & 0.960 & 1 & 1 + & & 0.060 & 0.180 & 0.490 & 0.745 & 0.970 & 1 & 1 + & & 0.090 & 0.180 & 0.310 & 0.760 & 0.915 & 1 & 1 +  table [ table1 ] reports the relative frequency estimates for the probability with and 0.5 .the estimation performs better when the factors are stronger .even when the factors are weak ( i.e. , ) , the estimation for is very accurate for .when the factors are strong ( i.e. , ) , we observe a phenomenon coined as `` blessing of dimensionality '' in the sense that the estimation for improves as the dimension increases .for example , when the sample size , the relative frequencies for are , respectively , 0.68 , 0.8 , 0.815 and 0.82 for 80 and 120 .the improvement is due to the increased information on from the added components of when increases . when , the columns of are -vectors with the norm [ see ( [ e2 ] ) ]. hence we may think that many elements of are now effectively 0 .the increase of the information on the factors is coupled with the increase of `` noise '' when increases .indeed , table [ table1 ] shows that when factors are weak as , the estimation for does not necessarily improve as increases .   with two strong factors ( ) and one weak factor ( ) and , .]  we also experiment with a setting with two strong factors ( with ) and one weak factor ( with ) . then the ratio - based estimator tends to take two values , picking up the two strong factors only. however figure [ fig11 ] indicates that the information on the third weak factor is not lost .in fact , tends to take the second smallest value at . in this casea two - step estimation procedure should be employed in order to identify the number of factors correctly ; see section [ sec6 ] below .the rates in theorem [ thm1 ] can be further improved , if we are prepared to entertain some additional conditions on in model ( [ c1 ] ) .such an improvement is relevant as the condition that , required in theorem [ thm1 ] , is sometimes unnecessary .for example , in table [ table1 ] , the ratio - based estimator works perfectly well when and is sufficiently large ( e.g. , ) , even though .furthermore , in relation to the phenomenon of `` blessing of dimensionality '' exhibited in table [ table1 ] , theorem [ thm1 ] fails to reflect the possible improvement on the estimation for when increases ; see also remark [ remark2](i ) .we first introduce some additional conditions on :  let denote the component of .then are independent for different and , and have mean 0 and common variance .the distribution of each is symmetric .furthermore , , and for all and , where is a constant independent of .all the eigenvalues of are uniformly bounded as .the moment condition in ( c8 ) implies that are sub - gaussian .condition ( c9 ) imposes some constraint on the correlations among the components of .when all components of are independent , ( c7)(c9 ) hold .see also conditions ( i)(iv ) of  .[ thm2 ] let conditions hold , and .then as , the following assertions hold :   for ,   for ,   for .if in addition holds , the rate in above can be further improved to   [ cor1 ] under the conditions of theorem [ thm2 ] , it holds that if in addition also holds , .the proofs of theorem [ thm2 ] and corollary [ cor0 ] are given in the .[ remark3 ] ( i ) by comparing with theorem [ thm1 ] , the error rate for nonzero  in theorem [ thm2 ] is improved by a factor , the error rate for zero - eigenvalues is by a factor at least .however , those estimators themselves may still diverge , as illustrated in figure [ largep ] .( ii ) theorem [ thm2](iii ) is an interesting consequence of the random matrix theory .the key message here is as follows : for the eigenvalues corresponding purely to the matrix , their magnitudes adjusted for converge at a super - fast rate .the matrix is a part of in ( [ d4 ] ) , where is the sample lag- autocovariance matrix for .in particular , when all the factors are strong ( i.e. , ) , the convergence rate is .such a super convergence rate never occurs when is fixed .\(iii ) condition is mild , and is weaker than condition required in theorem [ thm1 ] . for example , when , this condition is implied by the condition .\(iv ) with additional condition ( c9 ) , when all factors are strong .this shows that the speed at which converges to 0 increases when increases .this property gives a theoretical explanation why the identification for becomes easier for larger when all factors are strong ( i.e. , ) .see table [ table1 ] .in this section , we outline a two - step estimation procedure. we will show that it is superior than the one - step procedure presented in section [ sec4 ] for the determination of the number of factors as well as for the estimation of the factor loading matrices in the presence of the factors with different degrees of strength .a similar procedure is described in  to improve the estimation for factor loading matrices in the presence of small eigenvalues , although they gave no theoretical underpinning on why and when such a procedure is advantageous .consider model ( [ c1 ] ) with strong factors with strength and weak factors with strength , where . now( [ c1 ] ) may be written as where , with , consists of strong factors , and consists of weak factors . like model ( [ c1 ] ) in section [ sec31 ] , and are not uniquely defined , but only is .hereafter corresponds to a suitably rotated version of the original  in model ( [ e6 ] ) , where now  contains all the eigenvectors of  corresponding to its nonzero eigenvalues .refer to ( [ d3 ] ) for the definition of  .  to present the two - step estimation procedure clearly ,let us assume that we know and first . using the method in section [ sec4 ], we first obtain the estimator for the factor loading matrix , where the columns of are the orthonormal eigenvectors of corresponding to its largest eigenvalues . in practicewe may identify using , for example , the ratio - based estimation method ( [ d5 ] ) ; see figure [ fig11 ] .we carry out the second - step estimation as follows .let for all .we perform the same estimation for data now , and obtain the estimated factor loading matrix for the weak factors . combining the two estimators together , we obtain the final estimator for as   theorem [ thm3 ] below presents the convergence rates for both the one - step estimator and the two - step estimator .it shows that converges to at a faster rate than .the results are established with known and . in practicewe estimate and using the ratio - based estimators .see also theorem [ thm4 ] below .we introduce some regularity conditions first .let , , and for :  for , , , and . for any .the condition on in ( c5) is an analogue to condition ( c5 ) .see remark [ remark1](i ) in section [ sec52 ] for the background of those conditions .the order of will be specified in the theorems below .the order of is not restrictive , since is the largest possible order when .see also the discussion in remark [ remark1](ii ) .condition ( c6) replaces condition ( c6 ) . herewe impose a strong condition to highlight the benefits of the two - step estimation procedure .see remark [ remark4](iii ) below .put   [ thm3 ] let conditions , , , and hold .let and , as .then it holds that furthermore , if , in addition , and , where and are defined as follows :   note that .theorem [ thm3 ] indicates that between and , the latter is more difficult to estimate , and the convergence rate of an estimator for is determined by the rate for .this is intuitively understandable as the coefficient vectors for weak factors effectively contain many zero - components ; see ( [ e2 ] ) .therefore a nontrivial proportion of the components of may contain little information on weak factors . when , is dominated by . the condition for is imposed to control the behavior of the to the largest eigenvalues of under this situation .if this is not valid , those eigenvalues can become very small and give a bad estimator for , and thus . under this condition , the structure of the autocovariance for the strong factors , and the structure of the cross - autocovariance between the strong and weak factors , are not similar .recall that and are the largest eigenvalue of , respectively , defined in ( [ d3 ] ) and defined in ( [ d4 ] ) .we define matrices  and in the same manner as  and but with replaced by defined in ( [ e7 ] ) , and denote by  and the largest eigenvalue of , respectively ,  and .the following theorem shows the different behavior of the ratio of eigenvalues under the one - step and two - step estimation .readers who are interested in the explicit rates for the eigenvalues are referred to lemma [ lemma ] in the .[ thm4 ] under the same conditions of theorem [ thm3 ] , the following assertions hold :  for or , . for , . and provided   and provided .[ remark4 ] ( i ) theorem [ thm4](i ) and ( ii ) imply that the one - step estimation is likely to lead to .for instance , when , then theorem [ thm4](ii ) says that has a faster rate of convergence than as long as .figure [ fig11 ] shows exactly this situation .\(ii ) theorem [ thm4](iii ) implies that the two - step estimation is more capable to identify the additional factors than the one - step estimation . in particular , if , always has a faster rate of convergence than .unfortunately we are unable to establish the asymptotic properties for for , and for , though we believe that conjectures similar to ( [ e4 ] ) continue to hold .\(iii ) when and/or the cross - autocovariances between different factors and the noise are stronger , the similar and more complex results can be established via more involved algebra in the proofs .we illustrate our method using two real data sets .[ example1 ] we first analyze the daily returns of 123 stocks in the period 2 january 200211 july 2008 .those stocks were selected among those included in the s&p500 and were traded every day during the period .the returns were calculated in percentages based on the daily close prices .we have in total observations with .we apply the eigenanalysis to the matrix defined in ( [ d4 ] ) with .the obtained eigenvalues ( in descending order ) and their ratios are plotted in figure [ figspeig ] .it is clear that the ratio - based estimator ( [ d5 ] ) leads to , indicating two factors .@   for example [ example1].,title="fig : " ] + ( a ) +  for example [ example1].,title="fig : " ] + ( b )  varying the value of between 1 and 100 in the definition of leads to little change in the ratios , and the estimate remains unchanged .figure [ figspeig](a ) shows that is close to 0 for all .figure [ figspeig](b ) indicates that the ratio is close to 1 for all large , which is in line with conjecture ( [ e4 ] ) .the first two panels of figure [ figspts ] display the time series plots of the two component series of the estimated factors defined as in ( [ c2 ] ) .their cross - autocorrelations are presented in figure [ figspfacacf ] .although each of the two estimated factors shows little significant autocorrelation , there are some significant cross - correlations between the two series .the cross - autocorrelations of the three residual series for are not significantly different from 0 , where is the unit eigenvector of corresponding to its largest eigenvalue .if there were any serial correlations left in the data after extracting the two estimated factors , those correlations are most likely to show up in those three residual series .figure [ figspeig ] may suggest the existence of a third and weaker factor , though there are hardly any significant autocorrelations in the series .in fact and .note that now is not necessarily a consistent estimator for although ; see theorem [ thm1](ii ) and corollary [ cor0 ] . to investigate this further, we apply the two - step estimation procedure presented in section [ sec6 ] . by subtracting the two estimated factors from the above , we obtain the new data [ see ( [ e8 ] ) ] .we then calculate the eigenvalues and their ratios of the matrix .the minimum value of the ratios is , which is closely followed by and .there is no evidence to suggest that ; see theorem [ thm4 ] .this reinforces our choice .  .]  with as large as 123 , it is difficult to gain insightful interpretation on the estimated factors by looking through the coefficients in [ see ( [ c2 ] ) ] . to link our fitted factor model with some classical asset pricing theory in finance , we wonder if the market index ( i.e. , the s&p500 index ) is a factor in our fitted model , or more precisely , if it can be written as a linear combination of the two estimated factors .when this is true , , where is the vector consisting of the returns of the s&p500 index over the same time period , and denotes the projection matrix onto the orthogonal complement of the linear space spanned by the two component series , which is a 1640-dimensional subspace in .this s&p500 return series is plotted together with the two component series in figure [ figspts ] .it turns out that is not exactly 0 but , that is , the 97.7% of the s&p500 returns can be expressed as a linear combination of the two estimated factors .thus our analysis suggests the following model for daily returns of the 123 stocks : where denotes the return of the s&p500 on the day , is another factor , and is a vector white - noise process .figure [ figspts ] shows that there is an early period with big sparks in the two estimated factor processes .those sparks occurred around 24 september 2002 when the markets were highly volatile and the dow jones industrial average had lost 27% of the value it held on 1 january 2001 .however , those sparks are significantly less extreme in the returns of the s&p500 index ; see the third panel in figure [ figspts ] .in fact the projected s&p500 return is the linear combination of those two estimated factors with the coefficients ( ) .two observations may be drawn from the opposite signs of those two coefficients : ( i ) there is an indication that those two factors draw the energy from the markets with opposite directions , and ( ii ) the portfolio s&p500 index hedges the risks across different markets .[ example2 ] we analyze a set of monthly average sea surface air pressure records ( in pascal ) from january 1958 to december 2001 ( i.e. , 528 months in total ) over a grid in a range of longitude in the north atlantic ocean .let denote the air pressure in the month at the location , where and .we first subtract each data point by the monthly mean over the 44 years at its location : , where , representing the 12 different months over a year .we then line up the new data over grid points as a vector , so that is a -variate time series with .we have observations . to fit the factor model ( [ c1 ] ) to , we calculate the eigenvalues and the eigenvectors of the matrix defined in ( [ d4 ] ) with .let denote the eigenvalues of .the ratios are plotted against in the top panel of figure [ fig13 ] which indicates the ratio - based estimate for the number of factor is ; see ( [ d5 ] ) . however , the second smallest ratio is .this suggests that there may exist two weaker factors in addition ; see theorem [ thm4](ii ) and also figure [ fig11 ] .we adopt the two - step estimation procedure presented in section [ sec6 ] to identify the factors of different strength . by removing the factor corresponding to the largest eigenvalue of ,  the ratio of the eigenvalues of ( the top panel ) and ( the bottom panel ) , against , for example [ example2 ] . ]the resulting `` residuals '' are denoted as ; see ( [ e7 ] ) .now we repeat the factor modeling for data , and plot the ratios of eigenvalues of matrix in the second panel of figure [ fig13 ] .it shows clearly the minimum value at 2 , indicating further two ( weaker ) factors . combining the above two steps together , we set in the fitted model .we repeated the above calculation with in ( [ d4 ] ) .we still find three factors with the two - step procedure , and the estimated factors series are very similar to the case when .this is consistent with the simulation results in  , where they showed empirically that the estimated factor models are not sensitive to the choice of .  .]  [ fig14 ]  we present the time series plots for the three estimated factors in figure [ fig14 ] , where is a matrix with the first column being the unit eigenvector of corresponding to its largest eigenvalue , and the other two columns being the orthonormal eigenvectors of corresponding  . ]  to its two largest eigenvalues ; see ( [ e8 ] ) and also ( [ c2 ] ) . they collectively account for 85.3% of the total variation in which has components .in fact each of the three factors accounts for , respectively , 57.5% , 18.2% and 9.7% of the total variation of .figure [ fig15 ] depicts the factor loading surfaces of the three factors .some interesting regional patterns are observed from those plots .for example , the first factor is the main driving force for the dynamics in the north and especially the northeast .the second factor influences the dynamics in the east and the west in the opposite directions , and has little impact in the narrow void between them .the third factor impacts mainly the dynamics of the southeast region .we also notice that none of those factors can be seen as idiosyncratic components as each of them affects quite a large number of locations .  :sample cross - correlation functions for the three estimated factors . ]  : sample cross - correlation functions for three residual series .50 represents grid position ( 10 , 5 ) , 100 for ( 10 , 10 ) and 400 for ( 10 , 40 ) . ]figure [ fig16 ] presents the sample cross - correlations for the three estimated factors .it shows significant , though small , autocorrelations or cross - correlations at some nonzero lags .figure [ fig17 ] is the sample cross - correlations for three residuals series selected from three locations for which one is far apart from the other two spatially , showing little autocorrelations at nonzero lags .this indicates that our approach is capable to identify the factors based on serial correlations .finally we note that the bic method of  yields the estimate for this particular data set .we suspect that this may be due to the fact that  requires all the eigenvalues of be uniformly bounded when . this may not be the case for this particular data set , as the nearby locations are strongly spatially correlated , which may lead to very large and also very small eigenvalues for .indeed , for this data set , the three largest eigenvalues of are on the order of , and the three smallest eigenvalues are practically 0 . since the typical magnitude of is from our analysis , we have done simulations ( not shown here ) showing that the typical largest eigenvalues for , if is weakly correlated white noise , should be around to , and the smallest around to when and .such a huge difference in the magnitude of the eigenvalues suggests strongly that the components of the white - noise vector are strongly correlated . our method does not require the uniform boundedness of the eigenvalues of  .proof of theorem [ thm1 ] we present some notational definitions first .we denote by the largest eigenvalue of and the corresponding orthonormal eigenvector , respectively .the corresponding population values are denoted by and for the matrix .hence and .we also have   we show some intermediate results now . with conditions ( c3 ) and ( c5 ) and the fact that is white noise , we have \\[-8pt ] { \|{\widehat}{\bolds{\sigma}}_{x\epsilon}(k ) - { \bolds{\sigma}}_{x\epsilon}(k)\|},\qquad { \|{\widehat}{\bolds{\sigma}}_{\epsilon x}(k)\| } & = & o_p(p^{1-\delta/2}n^{-1/2 } ) , \nonumber\end{aligned}\ ] ] where . then following the proof of theorem 1 of , we have the following for : \\[-8pt ] & & \qquad\hphantom{\mbox{where } } { \|{\widehat}{\bolds{\sigma}}_y(k ) - { \bolds{\sigma}}_y(k)\| } = o_p\bigl(p^{1-\delta}n^{-1/2 } + p^{1-\delta/2}n^{-1/2 } + { \|{\widehat}{\bolds{\sigma}}_\epsilon(k)\|}\bigr)\nonumber\\ & & \qquad\hphantom{\mbox{where } { \|{\widehat}{\bolds{\sigma}}_y(k ) - { \bolds{\sigma}}_y(k)\|}}= o_p\bigl(p^{1-\delta/2}n^{-1/2 } + { \|{\widehat}{\bolds{\sigma}}_\epsilon(k)\|}\bigr).\nonumber\end{aligned}\ ] ] now , where denotes the frobenius norm of . hence from ( [ app1 ] ) , \\[-8pt ] { \|{\widehat}{{\mathbf m}}- { { \mathbf m}}\| } & = & o_p(p^{1-\delta } \cdot pn^{-1/2 } ) = o_p(p^{2-\delta}n^{-1/2 } ) .\nonumber\end{aligned}\ ] ]  for the main proof , consider for , the decomposition \\[-8pt ] & & \qquad\hphantom{\mbox{where } } i_3 = ( { \widehat}{\bolds{\gamma}}_j - { \mathbf a}_j)'{{\mathbf m}}{\mathbf a}_j,\qquad i_4 = { \mathbf a}_j'({\widehat}{{\mathbf m}}- { { \mathbf m}}){\widehat}{\bolds{\gamma}}_j , \nonumber\\ & & \qquad\hphantom{\mbox{where } } i_5 = { \mathbf a}_j'{{\mathbf m}}({\widehat}{\bolds{\gamma}}_j - { \mathbf a}_j).\nonumber\end{aligned}\ ] ] for , since where , and by ( [ app1 ] ) , together with ( [ app2 ] ) we have that so that , which proves theorem [ thm1](i ) .now consider .define following the same proof of theorem 1 of  , we can actually show that , so that .  noting for ,consider the decomposition \\[-8pt ] & & \hphantom{\qquad\mbox{where } } k_2 = 2{\widehat}{\bolds{\gamma}}_j'({\widetilde}{{\mathbf m}}- { { \mathbf m}})({\widehat}{\bolds{\gamma}}_j - { \mathbf a}_j),\nonumber\\ & & \hphantom{\qquad\mbox{where } } k_3 = ( { \widehat}{\bolds{\gamma}}_j - { \mathbf a}_j)'{{\mathbf m}}({\widehat}{\bolds{\gamma}}_j - { \mathbf a}_j ) . \nonumber\end{aligned}\ ] ] using ( [ app2 ] ) , similarly , using ( [ app1 ] ) and ( [ app2 ] ) , and , we can show that hence , and the proof of the theorem is completed .proof of corollary [ cor0 ] the proof of theorem 1 of  has shown that ( in the notation of this paper ) but we also have where the last equality sign follows from in ( [ app1 ] ) .hence we have for .letting for , we then have from theorem [ thm1](i ) .but since implying that , we have .hence we must have for .this implies that for , and together with theorem [ thm1](ii ) , this completes the proof of the corollary .  in the following , we use to denote the largest singular value of a matrix , so that .we use to denote the largest eigenvalue of .proof of theorem [ thm2 ] the first part of the theorem is actually theorem 2 of  .we prove the other parts of the theorem . from equation( 22 ) of  , the sample lag- autocovariance matrix for satisfies   note that ( [ app1 ] ) together with ( [ app3 ] ) implies since .we also have , similar to the proof of theorem [ thm1 ] .  with these , for , using decomposition ( [ appd1 ] ) , we have which is theorem [ thm2](i ) . for ,using decomposition ( [ appd2 ] ) , we have hence , which is theorem [ thm2](ii ) .  for part ( iii ), we define so that and .we define similarly , and . then we can write where , .it is easy to see that so that rank .this implies that then by theorem 3.3.16(a ) of  , for , where the last equality sign follows from ( [ app3 ] ) .this proves theorem [ thm2](iii ) .we prove theorem [ thm2](ii) now . using lemma 3 of  , with the same technique as in the proof of theorem 1 in their paper , we can write   with the definition of as in the proof of theorem [ thm1 ] , we can write , the largest eigenvalue of , as the element of the diagonal matrix , where .but from ( [ app15 ] ) , we also have , hence further , by using neumann series expansions of and , we see that the largest order term of is contributed from , since from ( [ app15 ] ) we have .hence the rate of can be analyzed using the element of .some notation first .define the column vector of ones , and since is finite and and are stationary , for convenience in this proof we take the sample lag- autocovariance matrix for , and the cross lag- autocovariance matrix between and to be respectively , for , and where .then where some tedious algebra ( omitted here ) shows that the dominating term of the above product is . defining and the first column of , the element of the said term is then in the last line we used , by noting that where is from ( [ app16 ] ) and is assumed in condition ( c5 ) . with condition ( c9 ) , we can show that , since where we used the markov inequality with the maximum eigenvalue of  , and the fact that .hence the element of has rate , which is also the rate of  for .this completes the proof of the theorem .we outline the proofs of theorems [ thm3 ] and [ thm4 ] below. detailed proofs can be found in the supplemental article ( lam and yao  ) .outline proof of theorem [ thm3 ] first , under model ( [ e6 ] ) and defined in ( [ d3 ] ) , with conditions ( c1)(c4 ) , ( c5) , ( c6) , we can show that the rates of the eigenvalues of are given by for model ( [ e6 ] ) , and defined in section [ sec6 ] by in ( [ e7 ] ) , we have   we can not use lemma 3 of  to prove this theorem for the one - step estimation , since the condition gives a restrictive condition on the growth rate of , and also restricts the range of allowed .instead , we use theorem 4.1 of  . write for , where , is the orthogonal complement of , and is diagonal with where contains for and contains for . with , define where if we denote .define . if we can show that \\[-8pt ] & & \eqntext{\mbox{with } \gamma_{ij } = \operatorname{sep}\left ( { { \mathbf d}}_i + { { \mathbf e}}_{ii } , \pmatrix { { { \mathbf d}}_j + { { \mathbf e}}_{jj } & { { \mathbf e}}_{j3 } \cr { { \mathbf e}}_{3j } & { { \mathbf e}}_{33 } } \right),}\end{aligned}\ ] ] then condition ( 4.2 ) in  is satisfied asymptotically , so that we can use their theorem 4.1 to conclude that for , since we can show that , we have .we can also show that using ( [ appo1 ] ) . hence ( [ appo3 ] ) is satisfied , and ( [ appo4 ] ) implies that also , we can show that , implying that .we can also show that using ( [ appo1 ] ) , provided . hence ( [ appo3 ] ) is satisfied since we assumed , and so ( [ appo4 ] ) implies that which completes the proof for the one - step estimation .for the two - step estimation , write , where is the orthogonal complement of , and is diagonal with .the matrix contains for , so that ( [ appo2 ] ) implies .we can show that , hence , as .hence we can use lemma 3 of  to conclude that since we can show that , we then have which completes the proof of the theorem .to prove theorem [ thm4 ] , we need two lemmas first .[ lemma ] under the same conditions and notations of theorem [ thm3 ] , the following assertions hold :  for , .  for , provided , .  for , ,provided , .  for , .  for ,  , .if in addition condition holds , then for , , provided , .the proof of this lemma is left in the supplementary materials for this paper . together with ( [ appo1 ] ) and ( [ appo2 ] ) , we have the following lemma .[ lemma2 ] let conditions , , , and hold .then as with , and with , the same as in theorem [ thm3 ] and , we have furthermore , if and , we have   if for , for , and , we have   for the two - step procedure , let conditions , , , and hold and .then we have   we only need to find the asymptotic rate for each and .the rate of each ratio can then be obtained from the results of lemma [ lemma ] .  for , from lemma [ lemma ] , , and hence from ( [ appo1 ] ) .consider the case .for , since , we have , and hence the other case is proved similarly .for , to make sure will not be zero or close to zero , we need where as in ( [ appo1 ] ) . hence we need , which is equivalent to the condition . with this condition satisfied , then for .since for , we then have all other rates can be proved similarly , and thus are omitted .proof of theorem [ thm4 ] with lemma [ lemma2 ] , theorem [ thm4](i ) is obvious . for theorem [ thm4](ii ) ,note that the range of and the rates given in the theorem ensure that .hence lemma [ lemma2 ] implies a better rate of convergence for no matter whether condition ( c9 ) holds or not .we can use a similar argument to prove part ( iii ) , and details are thus omitted .we are grateful to the joint editor professor peter bhlmann , the associate editor and the two referees for their helpful comments and suggestions .

Abstract: this paper deals with the factor modeling for high - dimensional time series based on a dimension - reduction viewpoint . under stationary settings , the inference is simple in the sense that both the number of factors and the factor loadings are estimated in terms of an eigenanalysis for a nonnegative definite matrix , and is therefore applicable when the dimension of time series is on the order of a few thousands . asymptotic properties of the proposed method are investigated under two settings : ( i ) the sample size goes to infinity while the dimension of time series is fixed ; and ( ii ) both the sample size and the dimension of time series go to infinity together . in particular , our estimators for zero - eigenvalues enjoy faster convergence ( or slower divergence ) rates , hence making the estimation for the number of factors easier . in particular , when the sample size and the dimension of time series go to infinity together , the estimators for the eigenvalues are no longer consistent . however , our estimator for the number of the factors , which is based on the ratios of the estimated eigenvalues , still works fine . furthermore , this estimation shows the so - called `` blessing of dimensionality '' property in the sense that the performance of the estimation may improve when the dimension of time series increases . a two - step procedure is investigated when the factors are of different degrees of strength . numerical illustration with both simulated and real data is also reported .  .  .


recently questions like , _ why go to so much effort to acquire all the data when most of what we get will be thrown away?_;_can we not just directly measure the part that will not end up being thrown away ? _ , that were paused by donoho and others , triggered a new way of sampling or sensing called compact ( `` compressed '' ) sensing ( cs ) .  in csthe task is to estimate or recover a sparse or compressible vector from a measurement vector .these are related through the linear transform . here, is a sparse vector and . in the seminal papers - , estimated from , by solving a convex optimization problem , .others have used greedy algorithms , like subspace pursuit ( sp) , orthogonal matching pursuit ( omp ) to solve the problem . in this paperthe focus is rather on the convex optimization methods .and we consider the noisy measurement system and the linear relation becomes    here , and are as in above where as the noise term , .there exists a large body of work on how to efficiently obtain an estimate for . andthe performances of such estimators are measured using metrics like restricted isometric property ( rip ) , mutual coherence ( mc ) , yet there is apparently no consensus on the bounds in using such metrics .the tool used in this paper gives performance bounds of large size cs systems .+ generally the linear model is used to describe a multitude of linear systems like code division multiple access ( cdma ) and multiple antenna systems like mimo , to mention just a few .tools from statistical mechanics have been employed to analyze large cdma and mimo systems  , and on in this paper the same wisdom is applied to analyze the performance of estimators used in cs .guo and et al in used a bayesian framework for statistical inference with noisy measurements and characterize the posterior distribution of individual elements of the sparse signal by describing the mean mean square error(mse ) exactly . to do so , they consider in a large system and applied the decoupling principle using tools from statistical mechanics .+ one can find also works that have used the tools from statistical mechanics to analyse cs system performances . to mention some , in as stated above , guo and etal used the tools to describe the minimum mean square error ( mmse ) estimator , in rangan and others used the maximum a posterior(map ) estimator of cs systems .these are referred as replica mmse claim and replica map claim in .  in - authors have used belief propagation and message passing algorithms for probabilistic reconstruction in cs using replica methods including rs .especially , in one finds excellent work about phase diagrams in cs systems while generalizes replica analysis using free random matrices .kabashima and et .al in , ganguli and sompolinsky in and takeda and kabashima - have shown statistical mechanical analysis of the cs by considering the noiseless recovery problem and they indicated that rsb analysis is needed in the phase regimes where the rs solution is not stable . in this paperthe performance of those cs estimators , considered as map estimator , is shown for the noisy problem by using the replica method including rs and rsb as in - , where the rsb ansatz gives better solution when the replica symmetry ( rs ) solution is unstable .this work is kind of an extension of from mimo systems to the cs systems .+ the paper is organized as follows . in section [ sec : problem setting ] the estimator in cs system are presented and redefined using the bayesian framework , and based on that we present our basis of analysis in section [ sec : analysis ] which is the replica method from the statistical physics and apply it on the different cs estimators which are presented generally as a map estimator . in section [ sec : particular example ] we showed our analysis using a paricular example , and section [ sec : conclusion ] presents conclusion and of future work .beginning with a given vector of measurements and measurement matrix , assuming noisy measurement with being i.i.d .gaussian random variables with zero mean and covariance matrix , estimating the sparse vector is the problem that we are considering where these variables are related by the linear model .various methods for estimating may be used . the classical approach to solving inverse problems of such type is by least squares ( ls ) estimator in which no prior information is used and its closed form is which performs very badly for the cs estimation problem we are considering since it does not find the sparse solution . another approachto estimate is via the solution of the unconstrained optimization problem    where is a regularizing term , for some non - negative . by taking ,emphasis is made on a solution with lp norm , and is defined as a penalizing norm .when , we get    this is penalizing the least square error by the l2 norm and this performs badly as well , since it does not introduce sparsity into the problem . when , we get the l0 norm , which is defined as    the number of the non zero intries of , which actually is a partial norm since it does not satisfy the triangle inequality property , but can be treated as norm by defining it as in , and get the l0 norm regularizing estimator which gives the best solution for the problem at hand since it favors sparsity in . nonetheless , it is an np- hard combinatorial problem .instead , it has been a practice to approximate it using l1 penalizing norm to get the estimator which is a convex approximation to the l0 penalizing solution [ l0 ] . the best solution for estimate of the sparse vector given by the zero - norm regularized estimator which is a hard combinatorial problem .these estimators , - , can equivalently be presented as solutions to constrained optimization problem - .this constrained optimization version of is known as the l1 penalized l2 minimization called lasso ( least absolute shrinkage and selection operator ) or bpdn(basis persuit denoising ) , which can be set as quadratic programing ( qp ) and quadratic constrained linear programing ( qcpl ) optimization problems . in the following subsection the above estimators are presented as a map estimator in bayesian framework .equivalently , the estimator of in can generally be presented as map estimator under the bayesian framework .assume a prior probability distribution for to be where the cost function is some scalar - valued , non negative function with and such that for sufficiently large , is finite as in . andlet the assumed variance of the noise be given by where is system parameter which can be taken as where is the assumed variance for each component of .note that we incorporate the sparsity in the prior pdf via . by the probability density function of given given by and prior distribution of by , the posterior distribution for the measurement channel according to bayes law is   then the map estimator can be shown to be now , as we choose different penalizing function in ( [ map ] ) we get the different estimators defined above in equations , , and but this time under the bayesian framework as a map estimator .  1 .linear estimators : when reduces to which is the lmmse estimator .lasso estimator : when we get the lasso estimator and becomes 3 .zero - norm regularization estimator : when , we get the zero - norm regularization estimator and becomes   whether these minimization problems are solvable or not the replica analysis results can provide the asymptotic performances of all the above estimators via replica method as showed in , , , and .we apply rs ansatz as used by mller and et al in and rsb ansatz as used by zaidel and et al on vector precoding for mimo .actually , this work is an extension of the rsb analysis to mimo systems done in to the cs system .the performance of the bayesian estimators like mmse and map can be done using the pdf of the error vector .the error is random and it should be centered about zero for the estimator to perform well .kay showed in that way ( see section 11.6 in ) the performance analysis of mmse estimator .we believe in general that inference for the asymptotic performance of map estimators is best done with statistical mechanical tools including rsb assumption and this is done in the sense of the mean square error ( mse ) .the posterior distribution is a sufficient statistics to estimate  and the denominator is called the normalizing factor or evidence in bayesian inference according to and partition function in statistical mechanics . actually , it is this connection , which gives the ground to apply the tools , which are used in statistical mechanics .so the task of evaluating the above estimators for the sparse vector can be translated to the statistical physics framework . and let us justify first how the analysis using statistical mechanical tool is able to do it .+  + define the gibbs - boltzmann distribution as    where is a constant known as the inverse temperature in the terminology of physical systems . for small , the prior probability becomes flat , and for large , the prior probability has sharp modes . , which is an expression of the total energy of the system , is called the hamiltonian in physics literature and is the partition function given by    often the hamiltonian can be given by a quadratic form like    with being a random matrix of dimension .then the minimum average energy per component of can be given by for our system that we considered to address , which is given by or equivalently by , the hamiltonian becomes compared to , the hamiltonian in has regularizing term in addition to the quadratic form , which is the energy of the error , in which the regularizing term is accountable for addressing the problem in the cs .the gibbs - boltzman distribution is a solution to or to in general , after plugging and since they are equivalent problems .the normalizing factor ( aslo called the partition function ) of this distribution is central for calculating many important variables and we shall begin from this term to analyse the cs estimators performance . + assuming that and being drawn from the same discrete set ( we shall later provide an example from such a set ). the partition function of the posterior distribution given in becomes },\ ] ] by using and .the posterior distribution depends on the predetermined random variables and called quenched states in physics literature , .that is , we use fixed states instead of for the large system limit , as , while maintaining fixed .we then calculate the nth moment of the partition function with respect to the predetermined variables , replicas , hence the name replica method came from .the replicated partition function is then given by },\ ] ] where . andafter substituting , it becomes } .\ ] ]  averaging over the noise first , we get } ,  \end{aligned}\ ] ] where and it is assumed to decompose into and is a diagonal matrix while is orthogonal matrix assumed to be drawn randomly from the uniform distribution defined by the haar measure on the orthogonal group . for more clarity on this one can see in . and is given by further averaging what we get on the right side of over the cross correlation matrix , by assuming the eigenvalue spectrum of to be self - averaging , we get } \biggr ) \\&=  \sum\limits_{\{\x^{a}\}}e^ {  \frac { -\beta\gamma } { \sigma_{u}^2 }  \sum\limits_{a=1}^{n }  f(\x^{a } ) } \underset{\j }  { \operatorname{e } } \biggl(e^{-\beta \bigl [ \frac { 1}{2 } tr \j \l(n )  \bigr ] } \biggr )  \end{aligned}\ ] ]  the inner expectation in is the harish -chandra -itzykoson - zuber integral ( again see in and and the references therein ) .the plan here is to evaluate the fixed - rank matrices as .further following the explanation in becomes where is the r - transform of the limiting eigenvalue distribution of the matrix j ( see , definition 1 in of r - transform or in and for better understanding of r - transform ) and denote the eigenvalues of the matrix , with defined through } ,  % \q_{ab}\equiv - \frac{1}{n } \sum\limits_{i=1}^{n }  { x_{i}^{a}}^t  x_{i}^{b } \hspace { 5 mm } for  \hspace { 5 mm }  a , b=0,1 , \cdots , n\ ] ] for . after applying replica trick ,the average free energy can be given by and the energy of the error can be calculated from the average free energy as where we get by using one of the assumptions used in replica calculations , after interchanging the order of the limits we assumed we get the same result .further , for we have since the additive exponential terms of order have no effect on the results when taking saddle point integration in the limiting regime as due to the factor outside the logarithm in any such terms are dropped further for notational simplicity as in .  in order to findthe summation in we employed the procedure in and the dimensional space spanned by the replicas is split into subshells , defined through matrix  the limit able us to use saddle point integration .hence we can have the following general result as similar to but extended in this work with the term , which pertains to cs , where we have given the expression that helps to evaluate the performances of the cs estimators using equation ._ [ prop : the limiting energy ] the energy from , for any inverse temperature , any structure of consistent with , and any r - transform such that is well - defined , is given by , \ ] ] where is the solution to the saddle point equation   see appendix [ app : prof of propostion the limiting energy ] .further , to get specific results we need to assume simple structure onto the cross correlation matrix at the saddle point .so we assume two different assumptions for the entries of called ansatz : replica symmetry(rs ) and replica symmetric breaking ( rsb ) ansatz .then compare the above limiting energy for the different estimators considered in this paper using the two types of ansatz for the cs system . that is the main purpose that we want to show in this paper .and we took the structures similar to :  1 .replica symmetry ansatz : 2 .one replica symmetry breaking ansatz : +   applying these assumptions we found some results as given in the following subsections . in the first subsectionwe assume the rs ansatz which can be considered as the extension of . in the last two subsections we assume rsb ansatz as an extension of to cs .consider the lasso estimator given in , which is equivalent to the solution of the main unconstrained optimization problem in penalized sense .its performance can be expressed in terms of the limiting energy penalty per component using two macroscopic variables and given by  where     and is refering about integration over gaussian measure , while refers to integration over the pdf of ( see appendix b ) . under rsansatz assumptions we then get the following statement .  _ _ [ prop : the limiting energy for the lasso estimator in rs ] given the lasso estimator in and the macroscopic variables and , in addition given the conditions in proposition 1 , the energy in simplifies to   see appendix b.   moving to the very purpose of the present paper , we use rsb ansatz instead of rs and we repeat what has been done in the above subsections . the limiting energy in this case involves four macroscopic variables like , , , and , which can be given by the following fixed point equations as and , as showed in appendix d , and using the compact notation as in . let and its normalized version       where and where the other variables , , and , are given by },\\ f_{1}&\displaystyle_{\longrightarrow } ^{n\rightarrow 0 }  \frac { 1}{\sigma_{u}^2 } \sqrt { q_{1 } r'(\frac{-b_{1}-\mu_{1}p_{1 } } { \sigma_{u}^2 } ) } % f_{1}&\displaystyle_{\longrightarrow } ^{n\rightarrow 0}\sqrt{\frac{1}{\beta } \biggl[\frac{\sigma_{0}^2}{\sigma_{u}^4}r(\frac{-b_{1}-\mu_{1 } p_{1 } } { \sigma_{u}^2 } ) + \frac{(\sigma_{u}^2\beta q_{1}+\sigma_{0}^2(b_{1}+\mu_{1}p_{1}))}{\sigma_{u}^6}r'(\frac{-b_{1}-\mu_{1}p_{1 } } { \sigma_{u}^2 } ) \biggr]},\end{aligned}\ ] ] then the following two statements are the extensions of the propositions in to cs problems ._ _ [ prop : the limiting energy for the lasso estimator in rsb ] given the lasso estimator in and suppose the random matrix satisfies the decomposability property .then under some technical assumptions , including one - step replica symmetry breaking , and the macroscopic variables given by the above fixed point equations , the effective energy penalty per component converges in probability as , , , to    see appendices d.   the lasso estimation is considered as the convex relaxation of the zero - norm regularizing estimation . since the latter is a non - convex problem its performance is better evaluated when we use rsb ansatz .so extending proposition to this estimator we get the following statement ._ _ [ prop : zero - norm regularizing estimator with 1rsb ansatz ] given the zero - norm regularizing estimator in and suppose the random matrix satisfies the decomposability property . then under some technical assumptions , including one - step replica symmetry breaking , the effective energy penalty per component converges in probablity as , , , to   see appendix d.assume the original vector follows a bernoulli - gaussian mixture distribution .so following the bayesian framework analysis in section [ sec : analysis ] , let be composed of random variables with each component obeying the pdf where , with being the number of non zero entries of . with out loss of generality , let , and vary between and . also lets assume that the entries of the measurement matrix follow i.i.d .gaussian random variable of mean zero and variance 1/m .in addition let be such that the signal to noise ratio is .we have simulated equations ( 2.7 ) and ( 2.8 ) .figure 1 shows mse versus of the two estimators , where we se that the penalizing estimator , lmmse , is not as good as the penalizing estimator in general .figure 2 shows mse versus of the two estimators and we see that lmmse is not sensitive to the sparsity of the vector as compared to the penalizing estimator .note that we have plotted the -penalizing estimator using different algorithms : lasso , l1-ls , log - bar .  in both figures, we see that the least square estimator is not good for the compressive sensing problem .in addition , we also observed that simulating the penalizing estimator is hard .however , it is possible to apply statistical physics tools , including replica methods , to analayze the performances of all the estimators mentioned above , including zero norm estimator . in ,median square error was used to compare the different estimators given by - as shown here in figure 3 .what we do here is that we include 1rsb ansatz analysis of the performance of the cs estimators as each of them are presented here as a map estimator .actually it is one of the conjuctures made by mller and others that the performance of map estimators is best done using one step rsb .and we showed it here via the minimized energy expressions as given in the propositions by the equations , , and .considering the macroscopic variables given by and and pluging the assumed distributions above and simplyfying it one more step , the fixed point equations become using these macroscopic variables in we find the limiting energy numerically which is given under propostion [ prop : the limiting energy for the lasso estimator in rs ] and the result is shown in figure [ fig : the limitting energy comparison ] .considering the same bernoulli - gaussian mixture distribution assumed in this section we consider the macroscopic variables which arises from one step replica symmetry breaking ( 1rsb ) ansatz . then the minimum energy per component as , while is finite ratio , which are given by and are dependent up on four macroscopic variables given by - .the ther first are simplified further as follows :  we can further simplify - as follows   it is possible to simplify these results further and give numerical results .but this is deferred for further work .we expect that the free energy from the rsb ansatz to be greater than the free energy from the rs ansatz for the zero - norm regularizing , which can be seen from the analytical terms which have more parameters in. however , for lasso these free energy , hence the energy error , will be quite similar since for convex minimization problems there is one global minimum and rs ansats is sufficient enough to produce the solution .in this paper we have used the replica method to analyze the performance of the estimators used in compressed sensing which can be generalized as map estimators . and the performance of map estimators can well be shown using replica method including one - step replica breaking ansatz . it is a philosophical standpoint that 1rsb enough to analyze the estimators like map .we have only showed here for one particular example for the cs problem , i.e. for bernoulli - gaussian distribution .one may be interested to verify it using different examples .in addition we have only compared the estimators performance based on the free energy , but one can also use other metrics such as comparing the input / out put distribution using replica analysis as it is done in . the main result of this paper is analytical analysis for the performance of the estimators used in cs and many things can be extended including efficient algorithms in implementing the numerical analysis .we are grateful to lars lundheim , rodrigo vicente de miguel and benjamin m. zaidel for interesting discussions and suggestions .in classical probability theory ( cpt ) one is concerned with the densities , moments and comulants of elements of random matrices . where as in random matrix theory ( rmt ) also called ( free random variable calculus ) , one is engaed in finding the spectral densities , moments and cumilants ( by professor maciej a. novak ) . asfourier transfom is the generating function for the moments in cpt , green s function ( also called stieltjes transform ) is the generating function for the spectral moments defined as where is random matrix and is of the same size unit matrix , are the eigenvalues , and is the spectral moment .the integral is over the support set of the eigenvalues .the generating function for the cumulants of the cpt is given by the logarithm of the fourier transfom .in similar maner to the above section we can define the generating function for spectral cumulants .it is called the r - transform ( voiculescu,1986 ) .it is given by    where are the spectral cumulants of the random matrix .we can relate r - transform with greens s function as follows : the spectral density of the matrix converges almost surely to the marchenko - pastur law as  . andthe r - transform of this matrix is given by    and its derivative with respect to z becomes where is system load .the avarage energy penality can be derived from the average free energy given in where is given by .using as the splitting of the space , we get where is the integration measure ,  \end{aligned}\ ] ]  denotes probability weight of the subshell composed of dirac - functions in the real line .this procedure is a change of integration variables in multiple dimensions where the integration of an exponential function over the replicas has been replaced by integration over the variables . to evaluate we follow , and represent the dirac measure using the fourier transform as where and this gives   where assuming , which is the sparsity enforcer as described above in lasso estimator , and after doing some rearrangements , the inner expectation of can be given by now defining we can get following the i.i.d .assumption for the component of the sparse vector , and applying the strong law of large numbers as we get where , is vector of dimention .next we apply the saddle point integration concept on the remaining part of , i.e. , as the integrand will be dominated by the exponential term with maximal exponent . hence in only the subshell that corresponds to this extremal value of the correlation between the vectors is relevant for the calculation of the integral . therefore , at the saddle point we have the following equations with partial derivatives being zero ( see the proof in appendix b of ) : =\0  \hspace{7 mm }  and\ ] ] =\0 .\ ] ] and from the former we get and from the later , using we finally get      the variables , , , , , ,,,, , and are called the macroscopic variables and they are all functions of n. they all can be calculated from the saddel point equations that we shortly will derive .first let us try to prove propostion [ prop : the limiting energy for the lasso estimator in rs ] using the ansatz in and .we do it using equations , and and we apply the saddelpoint integration rule . what matters most becomes the argument of the exponential in .so we first find , , and in addition we will find the macroscopic parametrs mentioned before since our limiting energy penality expressions for the different estimators considered in this paper are calculated interms of the macroscopic variables .hence using and we get and using and again we get from ( b.4 ) to ( b.7 ) we apply completing the square on the exponential of the argument and the hubbard - stratonovich transform , where is gaussian measure defined as before , to linearize the exponential argument .and we finally transformed the problem to a singele integral and a single summation problem . to evaluate we should first find the eigenvalues of the matrix l(n ) . under the rsansatz the matrix l(n ) has three types of eigenvalues :  and , and the numbers of degeneracy for each are 1 , n-1 , and n - n , respectively.thus we get the integral in is dominated by the maximum argument of the exponential function .therefore , the derivative of with respect to and must vanish as . plugging and into and taking the partial derivatives we get  respectively . after algebraic simplification and solving for and get }.\end{aligned}\ ] ] and with the limit for }.\end{aligned}\ ] ] by substituting into and doing the partial derivative of with respect to and and equating to zero we get ,  where so collecting the macroscopic variables in , , and and sending we have }\end{aligned}\ ] ]  and the fixed point equations , and further can be simplified via the saddle point integration rule in the limit as putting together the results above we have and the average free energy becomes r\bigl ( \frac{-(b_{0}+n\beta q_{0 } ) } { \sigma_{u}^2+n\sigma_{0}^2 } \bigr )  \\\nonumber & + \frac{-(b_{0}+n\beta q_{0 } ) } { ( \sigma_{u}^2+n\sigma_{0}^2)}\bigl [ -\frac{\bigl(\beta q_{0}(\sigma_{u}^2+n\sigma_{0}^2 ) -(b_{0}+n\beta q_{0 } ) \sigma_{0}^2\bigr ) } { ( \sigma_{u}^2+n\sigma_{0}^2)^2 } \bigr]r'\bigl ( \frac{-(b_{0}+n\beta q_{0 } ) } { ( \sigma_{u}^2+n\sigma_{0}^2 ) } \bigr ) \\ & + \int_{0}^{\frac{b_{0}}{\sigma_{u}^2 } } r(-w)dw  -   \int_{\mathbb{r } }  \int_{\mathbb{r } }  \frac { \zeta^n \ln \zeta } { \zeta^n }  dz df_{x^{0}}(x^{0 } ) \bigg\}\\ \nonumber & = \frac{-b_{0 } } { \sigma_{u}^2 }  r\bigl ( \frac{-b_{0 } } { \sigma_{u}^2 } \bigr)+  \frac{b_{0}\bigl ( \beta q_{0}\sigma_{u}^2 -b_{0}\sigma_{0}^2  \bigr )  } { \sigma_{u}^6 }  r'\bigl ( \frac{-b_{0 } } { \sigma_{u}^2 } \bigr ) \\ & + \int_{0}^{\frac{b_{0}}{\sigma_{u}^2 } } r(-w)dw  - \int_{\mathbb{r } } \int_{\mathbb{r } }  \ln \zeta  dz df_{x^{0}}(x^{0}).\end{aligned}\ ] ] coming back to the main goal , the solution for the main unconstrained optimization problem is given by the extremum of , it is calculated through the free energy by sending as follows this proves propostion [ prop : the limiting energy for the lasso estimator in rs ] . and to prove propostion [ prop : the limiting energy for the zero - norm estimator in rs ] what we need is to use the zero norm regularizing term instead of the l1 norm , i.e. using in , and the result will be as in which differ from through the calculation of the macroscopic varables which depend on the distributions of the components of .   turning to lasso estimator with rsb ansatz we first use and to get to evaluate we should first find the eigenvalues of the matrix . under the rsb ansatz the matrix has four types of eigenvalues :  , and , and the numbers of degeneracy for each are 1 , , , and , respectively .hence further with entries of being rsb ansatz will have more involved terms than the rs ansatzs .i.e. , using the hubbard - stratonovich transform we can express as in ( c.f .[ , ( 66)- ( 70 ) ] ) as follows + \beta^2g_{1}^2\sum\limits_{l=0}^{\frac{n \beta}{\mu}-1}\bigl| \sum\limits_{a=1}^{\frac{\mu}{\beta } } ( x^{0}- x_{a+\frac{l \mu_{1}}{\beta}})\bigr|^2  }  \nonumber\\ & \hspace{50 mm }  \cdot dz df_{x^{0}}(x^{0})\nonumber\\  & =  \int \log  \int_{\mathbb c } \biggl [ \int_{\mathbb c }  \biggl (  \sum\limits_{\{\x \in \chi \ } } \mbox{ } \mbox { ( , , ) }\biggr ) ^{\frac{\mu_{1}}{\beta } }  \mbox{ } \biggr]^ { \frac { n \beta } { \mu_{1 } } }  \mbox{ } \end{aligned}\ ] ] where due to the partial dervative of with respect to the macroscopic variables , , and vanishes as by definition of the sadel point approximation . and pluging and in and calculating the partial derivatives and seting them to zero and after some algebraic manipulation we get the folowing set of equations solving for , , we get },\\ f_{1}&=\sqrt{\frac{1}{n\beta } \biggl[\frac{1}{\sigma_{u}^2}r(\frac{-b_{1}-\mu_{1 } p_{1 } } { \sigma_{u}^2 } ) -\frac{1}{\sigma_{u}^2+n\sigma_{0}^2}r(\frac{-b_{1}-\mu_{1}p_{1 } -n\beta q_{1 } } { \sigma_{u}^2+n\sigma_{0}^2 } ) \biggr]},\end{aligned}\ ] ] and further with the limits }.\end{aligned}\ ] ] and as we can simplify it further as also due to the partial derivatives of with respect to , , and , must also vanish as .this produces the following set of equations while taking .  in addition when we takethe partial derivative of with respect of is vanishes and yields at the limit as  \nonumber\\ & \hspace{60 mm }  \cdot \mbox{ } \end{aligned}\ ] ] so as these fixed point equations can be simplified as follows :  where puting together the results again as in and doing again the steps ( b.34 ) to ( b.38 ) for the rsb case \nonumber\\  & -\log m ( q_{1 } , p_{1 } , f_{1 } , \mu_{1 } ) \biggr\}\\ & = \underset{\beta \rightarrow \infty } { \operatorname{lim } } \frac{1 } { \beta } \biggl\ {  ( \frac{b_{1}+\mu_{1 } p_{1 }  } { \sigma_{u}^2 } ) r(\frac{-b_{1}-\mu_{1 } p_{1 }  } { \sigma_{u}^2 } ) \\  & +  ( \frac{b_{1}+\mu_{1 } p_{1 }  } { \sigma_{u}^2})\frac { ( \beta q_{1 } \sigma_{u}^2 -(b_{1}+\mu_{1 } p_{1 } ) \sigma_{0}^2 ) } { \sigma_{u}^4 }  r'(\frac{-b_{1}-\mu_{1 } p_{1 }  } { \sigma_{u}^2 } )  \nonumber  \\ & + \frac{\beta}{\mu_{1}}\int_{0}^{\frac{b_{1}+\mu_{1 } p_{1}}{\sigma_{u}^2 }  } r(-w)dw +  ( 1-\frac { \beta}{\mu_{1}})\int_{0}^{\frac{b_{1}}{\sigma_{u}^2 }  } r(-w)dw  \nonumber\\ & + \bigl [ b_{1}(\beta f_{1}^2+\beta g_{1}^2- e_{1 } ) + \mu_{1 } ( q_{1 }  + p_{1 } ) ( \beta f_{1}^2+\beta g_{1}^2-\frac{\beta}{\mu_{1 } } e_{1 } ) - \mu_{1 } q_{1}\beta f_{1}^2  \bigr ]  \nonumber\\  &   - \frac{\beta}{\mu_{1 } } \int \log  \int_{\mathbb c }  \int_{\mathbb c }  \biggl (  \sum\limits_{\{\x \in \chi \ } } \mbox{ } \mbox { ( , , ) }  \biggr ) ^{\frac{\mu_{1}}{\beta } }  \mbox{ }  \mbox{ } \biggr \ }  \\ & = \frac {  q_{1 } } { \sigma_{u}^2 } ( \frac{b_{1}+\mu_{1 } p_{1 }  } { \sigma_{u}^2 } ) r(\frac{-b_{1}-\mu_{1 } p_{1 }  } { \sigma_{u}^2 } )  + \frac{1}{\mu_{1}}\int_{0}^{\frac{b_{1}+\mu_{1 } p_{1}}{\sigma_{u}^2 }  } r(-w)dw -  \frac { 1}{\mu_{1}}\int_{0}^{\frac{b_{1}}{\sigma_{u}^2 }  } r(-w)dw  \nonumber \nonumber \\ & + \bigl [ ( b_{1}+ \mu_{1 } ( q_{1 }  + p_{1 } ) ) ( f_{1}^2 + g_{1}^2)- e_{1}(q_{1 }  + p_{1 } ) - \mu_{1 } q_{1 } f_{1}^2  \bigr ] \nonumber\\ &  -\underset{\beta \rightarrow \infty } { \operatorname{lim } } \frac{1 } { \beta } \biggl\{\frac{\beta}{\mu_{1}}\int  \log  \int_{\mathbb c }  \int_{\mathbb c }  \biggl (  \sum\limits_{\{\x \in \chi \ } } \mbox{ } \mbox { ( , , ) }  \biggr ) ^{\frac{\mu_{1}}{\beta } }  \mbox{ }  \mbox{ } \biggr \ } \\ % \frac{f_{1}+\mu p_{1 }  } { \sigma_{u}^2 } r(\frac{-f_{1}-\mu p_{1 }  } { \sigma_{u}^2})+ \frac{\beta  q_{1 } } { \sigma_{u}^2 } r'(\frac{-f_{1}-\mu p_{1 }  } { \sigma_{u}^2 } ) % % & = \underset{\beta \rightarrow \infty } { \operatorname{lim } } \frac{1 } { \beta } \biggl\{+  - \frac{(f_{1}+\mu p_{1 } ) \sigma_{0}^2 } { \sigma_{u}^4 }  r'(\frac{-f_{1}-\mu p_{1 }  } { \sigma_{u}^2 } )  + \beta n q_{1 } } { \sigma_{u}^2+n\sigma_{0}^2 } } r(-w)dw + \frac{\beta}{\mu_{1}}\int_{0}^{\frac{b_{1}+\mu_{1 } p_{1}}{\sigma_{u}^2 }  } r(-w)dw  % &  +  ( 1-\frac { \beta}{\mu_{1}})\int_{0}^{\frac{b_{1}}{\sigma_{u}^2 }  } r(-w)dw - \underset{n \rightarrow 0}{\operatorname{lim } }  \frac{\partial } { \partial n}\bigl\{\log m ( q_{1 } , p_{1 } , f_{1 } , \mu_{1 } ) \bigr\ } \nonumber\\&+ \bigl[\beta^2 f_{1}^2 ( b_{1}+\mu_{1 } p_{1 } )  + \beta^2 g_{1}^2 ( b_{1}+\mu_{1}(q_{1 }  + p_{1 } ) -\beta e_{1}(q_{1 }  + p_{1 } + \frac{b_{1}}{\beta } )  \bigr ] \biggr\ }  \\ & = \frac {  1 } { \sigma_{u}^2 } ( q_{1 } + p_{1 } + \frac{b_{1}}{\mu_{1}})r(\frac{-b_{1}-\mu_{1 } p_{1 }  } { \sigma_{u}^2 } )  -\frac{b_{1}}{\mu_{1}\sigma_{u}^2}r(-\frac{b_{1 } } { \sigma_{u}^2 } ) \nonumber  \\ &   \hspace{10mm}+  q_{1 } ( \frac{b_{1}+\mu_{1 } p_{1 }  } { \sigma_{u}^2 } )  r'(\frac{-b_{1}-\mu_{1 } p_{1 }  } { \sigma_{u}^2})\\ & = \scriptstyle{\frac {  1 } { \sigma_{u}^2 } ( q_{1 } + p_{1 } + \frac{b_{1}}{\mu_{1}})r(\frac{-b_{1}-\mu_{1 } p_{1 }  } { \sigma_{u}^2 } )  -\frac{b_{1}}{\mu_{1}\sigma_{u}^2}r(-\frac{b_{1 } } { \sigma_{u}^2 } ) }  \nonumber\\ &  \scriptstyle { +  q_{1 } ( \frac{b_{1}+\mu_{1 } p_{1 }  } { \sigma_{u}^2 } )  r'(\frac{-b_{1}-\mu_{1 } p_{1 }  } { \sigma_{u}^2 } )   }  \end{aligned}\ ] ]  1 d. donoho,_compressed sensing_,1em plus 0.5em minus 0.4emieee trans . inform .theory , vol .4 , pp . 1289 - 1306 , 2006 .e. candes , j. romberg , and t. tao,_robust uncertainty principles : exact signal reconstruction from highly incomplete frequency information , _1em plus 0.5em minus 0.4emieee trans .theory , vol .489 - 509 , feb . 2006 .m. figueiredo , r. nowak , and s. wright,_gradient projection for sparse reconstruction : application to compressed sensing and other inverse problems _, 1em plus 0.5em minus 0.4emieee j. sel .top . signal process .586 - 597 , 2007 .d. guo , d. baron , and s. shamai,_a single - letter characterization of optimal noisy compressed sensin _ ,1em plus 0.5em minus 0.4emin proceedings of the forty- .seventh annual allerton conference on communication , control , and .computing , monticello , il , october 2009 .tanaka , t.,_a statistical mechanics approach to large system analysis of cdma multiuser detectors_,1em plus 0.5em minus 0.4em information theory , ieee transactions on , vol.48 , no.11 , pp.2888 - 2910 , nov 2002 .ralf r. mller , giusi alfano , benjamin m. zaidel , rodrigo de miguel,_applications of large random matrices in communications engineering_,1em plus 0.5em minus 0.4em invited for arxiv:1310.5479v1 [ cs.it ] 21 oct 2013 .s. sarvotham , d. baron , and r. g. baraniuk , , _ _ compressed sensing reconstruction via belief propagation,__1em plus 0.5em minus 0.4emtech .tree0601 , rice university , houston , tx 77005 , usa sundeep rangan , _ estimation with random linear mixing , belief propagation and compressed sensing_,1em plus 0.5em minus 0.4emarxiv:1001.2228v2 [ cs.it ] 18 may 2010 .florent krzakala , marc m ezard , francois sausset , yifan sun and lenka zdeborov a,_probabilistic reconstruction in compressed sensing : algorithms , phase diagrams , and threshold achieving matrices _ ,1em plus 0.5em minus 0.4emarxiv:1206.3953v1 [ cond-mat.stat-mech ] 18 jun 2012 .david l. donoho , adel javanmard and andrea montanari : _ information - theoretically optimal compressed sensing via spatial coupling and approximate message passing _arxiv:1112.0708v2 [ cs.it ]19 jan 2013 arian maleki , laura anitori , zai yang , and richard baraniuk,_asymptotic analysis of complex lasso via complex approximate message passing_,1em plus 0.5em minus 0.4emarxiv:1108.0477v2 [ cs.it ] 6 mar 2013 .antonia tulino , giuseppe caire , sergio verdu and shlomo shamai ( shitz),_support recovery with sparsely sampled free random matrices_,1em plus 0.5em minus 0.4emarxiv:1208.5269v1 [ cs.it ] 27 aug 2012 .y. kabashima , t. wadayama , and t. tanaka , j. stat .( 2009 ) l09003 .s ganguli , h sompolinsky - physical review letters , 2010 - keck.ucsf.edu k takeda , y kabashima - arxiv preprint arxiv:1001.4361 , 2010 - arxiv.org k. takeda , s. uda and y. kabashima1 _ analysis of cdma systems that are characterized by eigenvalue spectrum _ ,1em plus 0.5em minus 0.4em europhys .76 1193 , 2006 .koujin takeda , atsushi hatabu and yoshiyuki kabashima _ statistical mechanical analysis of the linear vector channel in digital communication_,1em plus 0.5em minus 0.4em j. phys . a : math .40 14085 , 2007 .r. r. muller , d. guo , and a. l. moustakas,_vector precoding in high dimensions : a replica analysis_,1em plus 0.5em minus 0.4em ieee journal on selected areas in communications , vol .530 - 540 , april 2008 .rodrigo de miguel , ralf mueller , _ on convex vector precoding for multiuser mimo broadcast channels_,1em plus 0.5em minus 0.4emieee transactions on signal processing , vol . 57 , issue:11 , june 2009 .benjamin zaidel , ralf mueller , aris moustakas , rodrigo de miguel _ vector precoding for gaussian mimo broadcast channels : impact of replica symmetry breaking._,1em plus 0.5em minus 0.4emarxiv:1001.3790v3 [ cs.it ] j 25 feb , 2011. s. m. kay , _ fundamentals of statistical signal processing : estimation theory_,1em plus 0.5em minus 0.4emenglewood cliffs , nj : prentice - hall , 1993 .david j.c .mackay,_information theory , inference , and learning algorithms._,1em plus 0.5em minus 0.4emuniversity of cambridge , 2003 .

Abstract: compressive sensing ( cs ) is a new methodology to capture signals at lower rate than the nyquist sampling rate when the signals are sparse or sparse in some domain . the performance of cs estimators is analyzed in this paper using tools from statistical mechanics , especially called replica method . this method has been used to analyze communication systems like code division multiple access ( cdma ) and multiple input multiple output ( mimo ) systems with large size . replica analysis , now days rigorously proved , is an efficient tool to analyze large systems in general . specifically , we analyze the performance of some of the estimators used in cs like lasso ( the least absolute shrinkage and selection operator ) estimator and zero - norm regularizing estimator as a special case of maximum a posteriori ( map ) estimator by using bayesian framework to connect the cs estimators and replica method . we use both replica symmetric ( rs ) ansatz and one - step replica symmetry breaking ( 1rsb ) ansatz , clamming the latter is efficient when the problem is not convex . this work is more analytical in its form . it is deferred for next step to focus on the numerical results .   solomon a.tesfamicael ( tesfamic.ntnu.no - spring 2010