low - density parity - check ( ldpc ) codes based on sparse graphs have emerged as a focal point in the theory of error correcting codes , used in noisy channel communication , largely because they are amenable to low complexity decoding and at the same time have a good performance ( measured as the gap to shannon s capacity ) .an important class of low complexity decoders are the message passing iterative decoders . in this framework , in order to decode a bit attached to a node of the graph , one unravels a computational tree ( or covering tree ) and iteratively updates messages ( suitable functions of the channel output observations ) passed along the edges of the computational tree .we refer to the recent book for the state of the art of this general theory .one would also like to be able to compare sub - optimal message passing decoders with the optimal or ideal decoder .the later is based on the posterior probability distribution supported on code - bits and is optimal in the sense that it is known to minimize the bit - error - rate among all decoders ( it is also called map decoder , and this is the terminology that we adopt in this paper ) .a priori the comparison of decoders is not easily done since the map decoder is in general computationaly complex .one of the most important low complexity message passing decoders is the belief propagation ( bp ) algorithm .it is well known that for a code whose graph is a tree , the bp algorithm has the same performance as the map decoder .this essentially comes from the fact that on a tree the computational graph of a node matches the original graph itself .however , codes based on tree graphs have poor performance and one needs to consider graphs with loops or cycles . with cycles in the original graph ,the messages on the computational tree are no longer independent , and it is not a priori clear , if and why , the bp algorithm should retain any close relationship to the map decoder .a fundamental theoretical tool that allows to analyze the bp algorithm is density evolution ( de ) first developed in . from de onecan for example obtain a noise threshold above which reliable communication is not possible with bp decoding .the analysis proceeds by taking first a very large block length and looking at iterations of the bp decoder .eventualy one considers the asymptotics .however , in the practical use of the decoder one fixes ( large ) and the iterations are performed till one reaches an acceptably small bit - error - rate .this corresponds to the asymptotics .the practical success of density evolution relies on the equivalence of these two limiting procedures , an open problem in general .it is fair to say that these issues have been resolved over the binary erasure channel ( bec ) ( the analysis however has not proceeded from the point of view of the correlations of the map decoder ) .an important tool in the analysis over the bec has been the " extended bp extrinsic information transfer ( exit ) curve ( which is a suitable continuation of the bit - error - rate curve under bp decoding ) .recently it was shown that the bit - error - rate of the map decoder can be obtained from that of the extended bp exit curve by a maxwell construction just as in the theory of first order phase transitions , . this construction allows to compute the map noise threshold , and to compare it to the bp noise threshold .the validity of the exchange of limits and ( for the bp decoder ) can also be derived for the bec using natural monotonicity properties of the decoder .  for the case of transmission over more general channelsvery little is known about these issues .indeed there one lacks the combinatorial methods available for the bec , and radically new methods have to be used .convenient measures of the performance , which generalize the exit curves , are the so - called gexit curves ( see the next section for their precise definition ) .it is believed that in terms of these , the results obtained for the bec still hold .in particular , the gexit curves for the bp and the map decoder should match for high and low noise regimes away from the phase transition thresholds .such conjectures are supported by spin glass theory calculations ( e.g the replica and cavity methods ) which provide conjectural but analytic formulas .one - sided bounds have been derived for the gexit curves by the ( information theoretical ) method of physical degradation and also by using correlation inequalities valid for spin glasses .related bounds on the conditional input - output entropy have also been derived , by using `` interpolation methods '' first developed in the mathematical theory of spin glasses , , .as it turns out , all these bounds match the replica expressions and are therefore believed to be the best possible . in the interpolation method has been extended to obtain the converse bounds for a class of poissonian ldpc codes over the bec , thus recovering combinatorial results of in a completely different way . concerning the problem of exchanging the limits we refer to for recent progress that goes beyond the bec .in this work we will show that a good deal can be learned by looking at the correlations ( more precisely the covariance ) , averaged over the channel outputs , of the map decoder .we comment below about the methods used , but let us say at the outset that our aim is to cover a fairly general class of binary input memoryless symmetric channels , including the binary symmetric and gaussian ones .one of our main results is that for sufficiently low noise ( ldpc codes ) the correlations between two code - bits decay exponentially fast as a function of the graph distance between the two code - bits , uniformly in the block length size .the sparsity of the underlying graph then implies that , if furthermore the decay rate beats the local expansion of the graph , the map gexit curve can be computed by de .another interpretation of this result is that the solutions provided by the replica / cavity methods of spin glass theory are exact .low - density generator - matrix codes ( ldgm ) codes have a very clear relationship to spin glass models on random graphs and it is useful to study them before we can attack the harder case of ldpc codes .besides , the present analysis could potentially be useful in other contexts where they are used ( e.g. rateless codes , source coding ) . for high noisewe prove the decay of correlations and that the map gexit curve can be computed by de .for that system , we can also show that the decay of correlations implies that the limits can be exchanged for the bp decoder , at least on the binary symmetric channel .the study of the behavior of correlations as a function of the distance between local degrees of freedom is one of the central aims of statistical physics . for lattice spin systems ( e.g. the ising model ) an important criterion that ensures correlation decay is dobrushin s criterion - which is of probabilistic nature - and its various improvements .the main other method - which is not necessarily of probabilistic nature - is based on suitable expansions in powers of `` the strength of interactions '' .there exists a host of such expansions collectively called `` cluster expansions '' , and the context of spin systems the first and simplest such expansion is the so - called `` polymer expansion '' .the main rule of thumb is that all these methods work if the degrees of freedom are weakly interacting , or if one can transform the original system into an effective one involving new weakly interacting degrees of freedom .it turns out that sophisticated forms of the cluster expansions can be carried out , for ldgm codes in a high noise regime , and for ldpc codes in a low noise regime , for a fairly general class of channels including the binary symmetric channel ( bsc ) as well as the binary input additive white gaussian noise channel ( biawgnc ) .as we will explain later it is necessary to use quite sophisticated cluster expansions for at least two reasons .concerning ldgm codes dobrushin s criterion and the polymer expansion require bounded channel outputs ( and thus do not covers the case of the biawgnc ) .concerning ldpc codes one has to transform the system to a dual one that involves `` negative gibbs weights '' and can not even be treated by probabilistic methods .the rest of the paper is organized as follows . in section [ section-2 ]we formulate the models and give a unified view of the main results both for ldgm and ldpc codes .the main strategy of the proofs is also explained there .sections [ section-3 ] and [ section-4 ] contain the proofs of correlation decay and its consequences for the gexit curves .the problem of exchanging the limits of iteration number and block length size is addressed in section [ section-5 ] .we conclude by pointing out open problems and further connections to the recent literature .the appendix reviews in a streamlined form the two cluster expansions that are used in sections [ section-3 ] and [ section-4 ] .summaries of the present results have been reported for the special case of the biawgnc , .we consider binary - input memoryless output - symmetric channels defined by a transition p.d.f with inputs and outputs belonging to .since we use techniques from statistical mechanics it is convenient to immediately map the usual input alphabet to .symmetry of the channel means that .the intensity of the noise is called . it will be convenient to trade off the channel outputs for the half - loglikelihood \ ] ] it is well known that on a symmetric channel one can assume without loss of generality that the all - one codeword ( i.e the usual all - zero codeword ) is transmitted and therefore the channel outputs are i.i.d with distribution . for clarity , we assume that the noise parameter varies in an interval  ] 3 .( low noise condition ) there exists small enough such that for we have = 0 ] , = \epsilon^{\frac{s}{2}}(1-\epsilon)^{1-\frac{s}{2 } } + ( 1-\epsilon)^{\frac{s}{2}}\epsilon^{1-\frac{s}{2}} ] , = e^{-s\epsilon^{-1}(1-\frac{s}{2})} ] which can spoil the convergence as . more physically , what happens is that even in the high noise regime there always exist with positive probability large portions of the graph that are at low noise ( or `` low temperature '' ) .we use a very convenient cluster expansion of dreifus - klein - perez that overcomes this problem by organizing the expansion over self - avoiding random walks on the graph . since the walks are self - avoiding the moment problem does not occur and we can treat unbounded loglikelihoods . for ldpc codesthe situation is more subtle because of the hard parity - check constraints that give an inherently low temperature flavor to the problem . from a purely code theoretical point of viewit is known that ldpc codes are the dual of ldgm codes .this algebraic duality can be exploited to transform the low noise communication model with ldpc codes to a dual model which , although not a genuine high noise communication model with ldgm codes , still retains this flavor .in fact this dual model involves `` negative gibbs weights '' .for this reason the cluster expansion of does not work anymore and we use resort to another one first devised by berretti .the two cluster expansions have to be adapted to our setting and are therefore reviewed in a somewhat streamlined form in appendix [ appendix - a ] .the proof of theorem [ theorem - corr ] will make it clear that for ldgm codes on channels with bounded loglikelihoods ( e.g the bsc ) at high noise , the average correlation  ] . as explained in section [ section-2 ]we need a technique that does not use the positivity of the gibbs weights . in appendixb we give a streamlined derivation of an adaptation of berretti s expansion . where and here and are two independent copies of the information bits ( these are also known as real replicas ) and . to explain what are and we will refer to -nodes ( check nodes in the tanner graph representing the ldpc code ) and -nodes ( variable nodes in the tanner graph representing the ldpc code ) .given a subset of nodes of the graph let be the subset of neighboring nodes . in the sum over is carried over clusters of -nodes such that `` is connected via hyperedges '' : this means that a ) for some connected subset of -nodes ; b ) is connected if any pair of -nodes can be joined by a path all of whose variable nodes lie in ; c ) contains both and . in the sum is a set of -nodes ( all distinct ) .we say that `` is compatible with '' if : ( i ) , ( ii ) and , ( iii ) there is a walk connecting and such that all its variable nodes are in . finally , the figure fig .[ fig : berrettifigure ] gives an example for all the sets appearing above .[ fig : berrettifigure ]  -nodes ) denoted by circles and check nodes ( -nodes denoted by squares ) . in this examplethe set is the set of _ dark _ check nodes .it is easy to verify that this choice of satisfies all our conditions .firstly , let be a set of variable nodes ( these are denoted by _dark _ circles in the figure ) .it is easy to check that the set of neighbours of is given by the _ dark _ check nodes which is .hence .secondly , any two variable nodes in are connected by a path all of whose variable nodes lie in , and thirdly , contains both and .one choice for .it is easy to check that is compatible with .the walk connects and and all its variable nodes lie in .another choice for would be the set . in the definition of _ light _variable nodes , , are not present because they have a non - empty intersection with . , scaledwidth=90.0%,scaledwidth=70.0% ]  we are now ready to prove the theorem on decay of correlations .  because of it suffices to prove that +  ] as we can always chose small enough to make small enough and conclude the proof .  0.25 cm   in the case of ldpc codes the functional giving the map - gexit function in is  note that the only formal difference with the ldgm case is in the normalization factor ; but of course now the gibbs average pertains to the ldpc measure .the bp - gexit curve is given by the same functionals with replaced by the average on the computational tree .as in section [ section-3 ] we introduce the neigborhood of node , radius an even integer .by the same arguments than in section [ section-3 ] we have again \ ] ] where is the gibbs bracket associated to the graph .it is important to note that for a tree the set of leaves are variable nodes and have `` natural boundary conditions '' as given by the channel outputs .the statistical mechanical sums on a tree yield the de formula where both limits exist and the are i.i.d random variables with distribution obtained from the iterative system of de equations with the initial condition . as before , these equations are an iterative version of the replica fixed point equation .the first few steps are the same as in the proof for ldgm .first , we expand the logarithm in and use nishimori identities to obtain a series expansion like ( the prefactor is now absent ) .second , we notice that since the resulting series expansion is uniformly absolutely convergent it is enough to show that = \lim_{d\to+\infty}{\mathbb{e}}_d[(\tanh\lambda^{(d)})^{2p}]\ ] ] thirdly , as before , one argues that this follows from = 0\ ] ] and because of it is enough to show this for replaced by .unfortunately one can not proceed as simply as in the ldgm case : is a consequence of the next two auxiliary lemmas stated below .let be the bracket defined on the subgraph with for .this in fact is formaly equivalent to fixing boundary conditions on the leaves of the tree .the first lemma says that the bit estimate can be computed locally .[ first - lemma ] under the same conditions than in corollary [ corollary-1 ] , = 0\ ] ]  the second lemma says that at low enough noise free and boundary conditions are equivalent  [ second - lemma ] under the same conditions than in corollary [ corollary-1 ] , = 0\ ] ]  we prove the first lemma. it will then be clear that the proof of the second one is essentialy the same except that the original full graph is replaced by , and thus it will be spared .  in ( and )the root node has which turns out to be technically cumbersome because we really work in a low noise regime .for this reason we use to deduce this implies and averaging over the noise and using cauchy - schwarz , & \leq  2\mathbb{e}[e^{4\vert l\vert}]^{1/2}\mathbb{e}_{l^n}[\vert\langle x_i\rangle - \langle x_i\rangle_{n_d(i)}^\infty\vert^2]^{1/2 } \cr & \leq  2\sqrt{2}\,\mathbb{e}[e^{8\vert l\vert}]^{1/2}\,\mathbb{e}_{l^n}[\vert\langle x_i\rangle - \langle x_i\rangle_{n_d(i)}^\infty\vert]^{1/2}\end{aligned}\ ] ] let us now prove  = 0\ ] ] we order the variable nodes at the boundary and consider the corresponding vector of loglikelihoods with components . if the first components of this vector are , the -th component is , and the other ones are i.i.d distributed as ( in other words they are natural ) we write . from the fundamental theorem of calculus , it is not difficult to see that using for any and we get let be the dual bracket ( with the first components of  and the -th component equal to ) . because of we have note that the denominator in the integral is important to make the integral convergent for .moreover at and is harmless as long as for .the next step is to use the cluster expansion in order to estimate \end{aligned}\ ] ] by following similar steps than in the proof of theorem [ theorem - corr ] one obtains an upper bound similar to except that the likelihoods of the end points are weighted differently and therefore there are two factors of ( see ) replaced by < \infty \qquad{\rm and}\qquad\mathbb{e}\biggl[\int_{l}^{+\infty } dl^\prime\ , \frac{2^{2s}e^{-4sl^\prime } + e^{-8sl^\prime}}{(\sinh 2 l^\prime)^{-2s}}\biggr]<\infty\ ] ] finally we can average over the code ensemble conditional on the event that is a tree . since the clusters that connect and , have size we obtain the result as long as is small enough , for small enough .in the ldgm case we prove the exchange of limits for the bsc channel .as will become clear one needs the decay of correlations ( or covariance ) of the gibbs measure on the computational tree for .hence the likelihoods are not independent r.v : the proof of theorem [ theorem - corr ] still goes through in the case of the bsc .the only difference is that in lemma [ lem : corrbound ] we can take such that and for all .[ lemma - bp ] consider communication with a fixed general ldgm code with blocklength size and bounded degrees , , over the bsc( ) .we can find , a small enough numerical constant such that for we have , for any given realization of the channel outputs , where is the root of the computational tree , and arbitrary node , a numerical constant and depending only on , , .moreover increases like as .basically , this result is contained in where it is obtained by dobrushin s criterion .note that it is valid for fixed noise realizations and not only on average .the unbounded case would require to take averages but then , on the computational tree one has to control moments $ ] and this requires more work .the following proof is a simple application of this lemma .we take for the number of iterations of the bp decoder . on the computational tree consider the subtree of root and depth .this subtree is a smaller computational tree and .let the leaves with and order them in an arbitrary way .consider the gibbs measure where for the first checks of we set in . proceeding as in section [ section-3 ] we get for the bsc , .from lemma [ lemma - bp ] for small enough ( but independent of ) in this equation is uniformly bounded with respect to and ( and the noise realizations of course ) .recall the gexit function of the bp decoder since for , , one can easily show for example one could proceed by expanding the in powers of and estimate the series term by term .now since is uniformly bounded with respect to implies for fixed now we take the limit , a similar result with replacing is derived in the same way .in this paper we have shown that cluster expansion techniques of statistical mechanics are a valuable tool for the theory of error correcting codes on graphs .we have not investigated the regimes of high noise for ldpc codes and low noise for ldgm codes . in the case of ldpc codes andhigh noise we are able to prove decay of correlations for ensembles that contain a sufficient fraction of degree one variable nodes . indeed one can eliminate the degree one nodes and convert the problem to a new graphical model containing a mixture of hard parity check constraints and soft ldgm type weigths . if the density of soft weights is high enough the analysis of the present paper can be extended ( see for a summary ) .combining theses ideas with duality one may also treat special ensembles of ldgm codes for low noise .this approach however is not entirely satisfactory and it is not clear how to directly go about with cluster expansions in these regimes .we hope that the ideas and techniques investigated in the present work could have other applications in coding theory and more broadly random graphical models .let us mention that various forms of correlation decay have been investigated recently for the random -sat problem at low constraint density , by different methods .this has allowed the authors to prove that the replica symmetric solution is exact at low constraint density . in authors derive a new type of expansion called `` loop expansion '' in an attempt to compute corrections to bp equations .the link to traditioanl cluster expansions is unclear to us , and also it would be interesting to develop rigorous methods to control the loop expansions .finaly , we would also like to point out the work where a new derivation of the gilbert - varshamov bound is presented using the mayer expansion for a hard - sphere systems .the work of shrinivas kudekar has been supported by a grant of the swiss national foundation 200020 - 113412 .in this appendix we explain the derivation of the two cluster expansions that we use . in the statistical mechanics literature these have been derived for spin systems with pair interactions on regular graphs .it turns out that they can be adapted to our setting .we try to give a self - contained by still reasonably short derivation here .   herewe adapt the cluster expansion of dreifus - klein - perez in . in the processwe prove lemma [ lem : corrbound ] stated in section [ section-3 ]. it will be very convenient to use the following compact notation in particular the code - bits become , and the correlation of lemma [ lem : corrbound ] becomes it is first necessary to rewrite the gibbs measure in a form such that the exponent is positive where is the appropriately modified partition function .we introduce the replicated measure , which is the product of two copies , thus we now have two replicas of the information - bits and . the gibbs bracket for the replicated measure is denoted by .it is easy to see that recall that for some fixed number , and set it will be important to keep in mind later that .we have where , .take a term with given in the last sum .we say that `` connects and '' if and only if there exist a self - avoiding walk for the definition of these walks . ] with initial variable node , final variable node and such that all check nodes of are in that connects and .since is fixed our definition is valid ] .the crucial point is that : if a set does not connect and , then it gives a vanishing contribution to the sum .we defer the proof of this fact to the end of this section . for the momentlet us show that it implies the bound in lemma [ lem : corrbound ] .the positivity of implies in the second inequality we used .now resumming over we obtain the second inequality follows by inserting extra terms for , and the second by reconstituting in the numerator .now , the last line is equal to hence the bound .  it remains to explain why, if does not connect and , the -term does not contribute to .let be the set of variable nodes connected to the check nodes .we define a partition into three sets of variable nodes . is the set of all variable nodes such that there exist a self - avoiding walk connecting some to , and such that all ckeck nodes of are in . is similarly defined with and instead of .finally . by construction .the point is that if does not connect and , then .indeed , otherwise there would be a with a walk and a walk both with all check nodes in , but this would mean that connects and through the walk .we also define three sets of check nodes , and . again the three sets are disjoint when does not connect and : indeed if there exists then belongs to both and which we just argued is impossible .this situation is depicted on figure ( 3 ) .is depicted by the dark squares .the set is depicted by the light squares . and contain both three nodes ; there does not exist a self - avoiding walk that connects these two sets with all its check nodes in .the sets of variable nodes , and are disjoint as well as the sets of check nodes , and : these sets are enclosed in the dotted areas.,scaledwidth=80.0%,scaledwidth=70.0% ]  now we examine a term of for a that does not connect and . expanding the product , using linearity of the bracket and symmetry under exchange of replicas , it is equal to the difference where  because of the disjointness of the sets and ( the areas enclosed in dotted lines , see figure ( 3 ) ) one can , in and , factor the sums in a product of three terms ( in fact there is a fourth trivial term which is a power of coming from the bits outside the dotted areas ) . then by symmetry recognizes that .thus and this proves that does not contribute to when it does not connect and .here we adapt the berretti cluster expansion to our setting . for more detailswe refer to , .consider the _ replicated _ partition function here are two replicas of the information bits and .we have where corresponds to the replicated system .we denote , . then we have where is defined in . expanding the productwe get , where denotes the set of all variable nodes of the original tanner graph for the ldpc code and is any subset of distinct variable nodes .suppose is such that _ one can not create a walk _( i.e. on the original tannger graph of the ldpc code , a set of alternating variable and check nodes ) connecting any check node in , to any check node in , and which has all its variable nodes contained entirely in .then we can partition into three mutually disjoint sets of variable nodes , such that , and .note also that are mutually disjoint otherwise we can create a walk between and .thus we can write this implies that vanishes .this is seen by using the antisymmetry of ( or ) and the symmetry of , under the exchange .thus only those which contain a walk with all its variable nodes in and which intersects both and contributes to the sum in .for any given ( contributing to the sum ) we construct the set of variable nodes as follows . is the union of all maximal connected clusters of distinct variable nodes in , such that each of those connected clusters intersects .let .clearly , there exists such a set because we know that the walk which connects and is a subset of .let be a set of check nodes .it is not difficult to see that satisfies all the requirements of the set in the sum .indeed , consider . by construction ; any two variable nodes in are connected by a walk with all its variable nodes in ; contains both and .also note that is compatible with as is required in the sum .indeed , by construction ; and ; there exists a walk between and with all its variable nodes in .  with thiswe can write now we resum over the sets such that .these consist of compatible with and the rest which does not intersect .so the last bracket is equal to and we recognize berretti s expansion . figure ( 4 ) shows a sample set and which give a non - vanishing contribution .   .the walk connects to and hence this has a non - vanishing contribution . , is union of the two maximal connected clusters and which has intersection with .

Abstract: the subject of this paper is transmission over a general class of binary - input memoryless symmetric channels using error correcting codes based on sparse graphs , namely low - density generator - matrix and low - density parity - check codes . the optimal ( or ideal ) decoder based on the posterior measure over the code bits , and its relationship to the sub - optimal belief propagation decoder , are investigated . we consider the correlation ( or covariance ) between two codebits , averaged over the noise realizations , as a function of the graph distance , for the optimal decoder . our main result is that this correlation decays exponentially fast for fixed general low - density generator - matrix codes and high enough noise parameter , and also for fixed general low - density parity - check codes and low enough noise parameter . this has many consequences . appropriate performance curves - called gexit functions - of the belief propagation and optimal decoders match in high / low noise regimes . this means that in high / low noise regimes the performance curves of the optimal decoder can be computed by density evolution . another interpretation is that the replica predictions of spin - glass theory are exact . our methods are rather general and use cluster expansions first developed in the context of mathematical statistical mechanics .


always prefer the _ simplest _ explanation for your observation , " says occam s razor . in learning and information theory , simplicityis often quantified in terms of description length , giving immediate rise to the minimum description length ( mdl ) principle .thus mdl can be seen as a strategy against overfitting .an alternative way to think of mdl is bayesian .the explanations for the observations ( the _ models _ ) are endowed with a prior .then the model having maximum a posteriori ( map ) probability is also a two - part mdl estimate , where the correspondence between probabilities and description lengths is simply by a negative logarithm .  how does two - part mdl perform for prediction ? some very accurate answers to this question have been already given .if the data is generated by an independently identically distributed ( i.i.d . )process , then the mdl estimates are consistent . in this case , an important quantity to consider is the _ index of resolvability _ , which depends on the complexity of the data generating process .this quantity is a tight bound on the regret in terms of coding ( i.e. the excess code length ) .moreover , the index of resolvability also bounds the predictive regret , namely the rate of convergence of the predictive distribution to the true one .these results apply to both discrete and continuously parameterized model classes , where in the latter case the mdl estimator must be discretized with an appropriate precision .  under the relaxed assumption that the data generating process obeys a central limit theorem and some additional conditions, rissanen proves an asymptotic bound on the regret of mdl codes . here , he also removes the coding redundancy arising if two - part codes are defined in the straightforward way . the resulting bound is very similar to that in for bayes mixture codes and i.i.d.processes , where the i.i.d . assumption may also be relaxed .other similar and related results can be found in .  in this work ,we develop new methods in order to arrive at very general consistency theorems for mdl on _ countable model classes_. our setup is _ online sequence prediction _ ,that is , the symbols of an infinite sequence are revealed successively by the environment , where our task is to predict the next symbol in each time step .consistency is established by proving _finite cumulative bounds _ on the differences of the predictive to the true distribution .differences will be measured in terms of the relative entropy , the quadratic distance , and the hellinger distance .most of our results are based on the only assumption that the data generating process is _ contained in the models class_. ( the discussion of how strong this assumption is , will be postponed to the last section . )our results imply regret bounds with _ arbitrary _ loss functions .moreover , they can be directly applied to important general setups such as pattern classification , regression , and universal induction .as many scientific models ( e.g. in physics or biology ) are smooth , much statistical work is focussed on continuous model classes . on the other hand ,the largest relevant classes from a computational point of view are at most countable .in particular , the field of algorithmic information theory ( also known as kolmogorov complexity , e.g.  ) studies the class of _ all lower - semicomputable semimeasures_. then there is a one - to - one correspondence of models and programs on a fixed universal turing machine .( since programs need not halt on each input , models are semimeasures instead of measures , see e.g.  for details ). this model class can be considered the largest one which can be in the limit processed under standard computational restrictions .we will develop all our results for semimeasures , so that they can be applied in this context , which we refer to as _universal _ sequence prediction .  in the universal setup ,the bayes mixture is also termed solomonoff - levin prior and has been intensely studied first by solomonoff .its predictive properties are excellent .precisely one can bound the cumulative loss by the complexity of the data generating process .this is the reference performance we compare mdl to .it turns out that the predictive properties of mdl can be exponentially worse , even in the case that the model class contains only bernoulli distributions .another related quantity in the universal setup is _ one - part mdl _ , which has been studied in . we will briefly encounter it in section [ subsecuniversal ] .the paper is structured as follows .section [ secpn ] establishes basic definitions . in section [ secmdl ], we introduce the mdl estimator and show how it can be used for sequence prediction in at least three ways .sections [ secdynamic ] and [ secstatic ] are devoted to convergence theorems . in sections[ sechybrid ] and [ secstabilization ] , we study the stabilization properties of the mdl estimator .section [ secapp ] presents more general loss bounds as well as three important applications : pattern classification , regression , and universal induction .finally , section [ secdc ] contains the conclusions .we build on the notation of and .let the alphabet be a finite set of symbols .we consider the spaces and of finite strings and infinite sequences over .the initial part of a sequence up to a time or is denoted by or , respectively .the empty string is denoted by .a _ semimeasure _ is a function  ] ; we call the _-prediction_. clearly , this is not necessarily a probability on for general . for a semimeasure in particular ,the -prediction is a semimeasure on .we define the _ expectation _ with respect to the true probability : let and be a function , then [ eq : expectation ] f= f(x)=_x^n(x)f(x ) .more general , the expectation may be defined as an integral over infinite sequences .but since we wo nt need it , we can keep things simple .the following is a central result about prediction with bayes mixtures in a form independent of algorithmic information theory .[ theorem : solomonoff ] for any class of ( semi)measures containing the true distribution , which is a measure , we have [ eq : solomonoff ] _ t=1^_a ( ( a|x)-(a|x ) ) ^2 w_^-1 .this was found by solomonoff ( ) for universal sequence prediction . a proof is also given in ( only for binary alphabet ) or ( arbitrary alphabet ) .it is surprisingly simple once lemma [ lemma : entropyineq ] is known .a few lines analogous to ( [ eq : theoremrhoconvergesims0 ] ) and ( [ eq : theoremrhoconvergesims00 ] ) exploiting the dominance of are sufficient .one should be aware that the condition is essential in general , for both bayes and mdl predictions .on the other hand , one can show that if there is an element in which is sufficiently close to in an appropriate sense , then still good predictive properties hold .note that although can be interpreted as a prior on the model class , we do not assume any probabilistic structure for ( e.g. a sampling mechanism ) .the theorem rather states that the cumulative loss is bounded by a quantity depending on the complexity of the true distribution .the same kind of assertion will be proven for mdl predictions later .the bound ( [ eq : solomonoff ] ) implies that the -predictions converge to the -predictions almost surely ( i.e. with -probability one ) .this is not hard to see , since with the abbreviation and for each , we have ( tn : s_t ) & = & ( _ tn\{s_t } ) + [ eqihswp1 ] & & _tn(s_t ) _ t = n^s_tactually , ( [ eq : solomonoff ] ) yields an even stronger assertion , since it characterizes the _ speed of convergence _ by the quantity on the right hand side .precisely , the expected number of times in which deviates by more than from is finite and bounded by , and the probability that the number of -deviations exceeds is smaller than .( however , we _ can not _conclude a convergence rate of from ( [ eq : solomonoff ] ) , since the quadratic differences generally do not decrease monotonically . )  since we will encounter this type of convergence ( [ eq : solomonoff ] ) frequently in the following , we call it _ convergence in mean sum ( i.m.s ) _ : [ eq : ims ] c>0 : _ t=1^_a ( ( a|x)-(a|x ) ) ^2 < .then theorem [ theorem : solomonoff ] states that the predictions converge to the predictions i.m.s . , or converges to i.m.s . " for short . by ( [ eqihswp1 ] ) , convergence i.m.s. implies almost sure convergence ( with respect to the true distribution ) . note that in contrast , convergence in the mean , i.e.\toinfty{t } 0 ] . for the dynamic mdl predictor , this holds by [ eq : rhoin01 ] ^x(x)^xa(x)^xa(xa ) .static mdl is omnipresent in machine learning and applications , see also section [ secapp ] .in fact , many common prediction algorithms can be abstractly understood as static mdl , or rather as approximations .namely , if a prediction task is accomplished by building a _ model _ such as a neural network with a suitable regularization to prevent overfitting " , this is just searching an mdl estimator within a certain class of distributions .after that , only this model is used for prediction .dynamic mdl is applied more rarely due to its larger computational effort .for example , the similarity metric proposed in can be interpreted as ( a deterministic variant of ) dynamic mdl .we will need to convert our mdl predictors to _ measures _ by means of _normalization_. if  ] , let [ eq : delta ] ( _ 1,_2)=(_t=1^_a ( _ 1(a|x)-_2(a|x))^2)^. then the triangle inequality holds for , since is ( proportional to ) an euclidian distance ( 2-norm ). moreover , by theorem [ theorem : rhoconvergesims ] and .we also have by multiplying in theorem [ thrhonorm] with another .note , since both  ] . breaking ties in the above expression in an arbitrary deterministic way ,the resulting prediction is _ deterministic _ for given and loss function .if is the true distribution as usual , then let be the -expected loss of the -predictor .then , by l^= = _ t=1^n ( x)l_t^(x ) we denote the cumulative -expected loss of the -predictor . with being the variants of the mdl predictor , we will bound the quantity , i.e. the cumulative _ regret _ , by an expression depending on and .we admit arbitrary non - stationary loss functions which may depend on the history .our analysis considers the worst possible choice of loss functions and consists of three steps .first the cumulative regret bound is reduced to an instantaneous regret bound ( lemma [ lemma : instant2cum ] ) .then the instantaneous bound is reduced to a bound in terms of special functions of and ( lemma [ lemma : instant2special ] ) .finally , the bound for the special functions is given ( lemma [ lemma : special ] ) .[ lemma : instant2cum ] assume that some -predictor satisfies the instantaneous regret bound , where is the hellinger distance of the instantaneous predictive probabilities ( [ eq : hellinger ] ) .then the cumulative -regret is bounded in the same way : = l^-l^2h(,)+2 .this and the following lemma hold with arbitrary constants , the choices and are the smallest ones for which lemma [ lemma : special ] is true . note that if the hellinger distance is replaced by the relative entropy , then may be replaced by .thus , normalized dynamic mdl and bayes mixture admit smaller bounds , compare . however , this is not true for the other mdl variants , as we have no relative entropy bound there .the key property is the _ super - additivity _ of the bound .a function is said to be super - additive if f(x_1+x_2,y_1+y_2)f(x_1,y_1)+f(x_2,y_2 ) .the function satisfies this condition .we now use an inductive argument .assume , where the summation starts at and the superscript indicates that the first symbol of the sequence was .let the same hold for the first symbol .writing and using , we obtain + & & 2 + & & 2 + & & 2h+2 . here , the first inequality is the induction hypothesis together with the instantaneous bound , the second bound is cauchy - schwarz s inequality , and the last estimate is the super - additivity .[ lemma : instant2special ] assume that some -predictor satisfies for all  ] , we have [ eq : instantbound ] 2h+2 .first we show that we may assume , i.e. we do not incur loss for correct predictions . to this end ,consider the modified loss function and assume w.l.o.g  ] .we consider the worst case by maximizing , i.e. choosing as large as possible .for this , we obtain and = ^1[-]=^1 ^ 1[2h+2]2h+22h+2 , showing ( [ eq : instantbound ] ) provided that .the other cases are shown similarly .[ lemma : special ] the bound holds for all  ] as defined in lemma [ lemma : instant2special ] .  the technical and not very interesting proof of this lemma is omitted .the careful reader may check the assertion numerically or graphically , as it is just the boundedness of some function on the unit square .we remark that the bound does _ not _ hold if the hellinger distance is replaced by the quadratic distance , not even with larger constants .  for arbitrary non - stationary loss function which is bounded in ] , where is the characteristic function of an interval .let , then the quadratic distance is , whereas the relative entropy is constant . ]( note that our use of the squared error is completely different from the standard use in regression ) .so we will use the hellinger distance instead , defined similarly to ( [ eq : hellinger ] ) by [ eqhellingercont ] h(f , f ) = ( - ) ^2dx f , f:[0 , ) .accordingly , is the cumulative hellinger distance of two predictive distributions and .similarly as in ( [ eqhellingerkl ] ) and ( [ eqhellingerabs ] ) , the hellinger distance is bounded by the ( continuous ) relative entropy and absolute distance .this shows in particular that the integral ( [ eqhellingercont ] ) exists .we now consider a countable class of models that are functions from to _ uniformly bounded probability densities _ on .that is , there is some such that [ eqc ] 0_i(x|u)c _ -^_i(x|u)dx=1 _ all i1 , u , x. for all , , and .the mdl estimator is then defined as the element which maximizes the _ density _ , .the uniform boundedness condition asserts that the mdl estimator exists .it may be relaxed , provided that the mdl estimator remains well - defined , such as for a family of gaussian densities which tend to the point measure .  with these definitions ,the proofs of the theorems for static and dynamic mdl can be adapted .since the triangle inequality holds for the hellinger distance , we obtain the following .let be a countable model class according to ( [ eqc ] ) , containing the true distribution .then for any sequence of inputs , we have , , and .  we may apply this for example to model classes with gaussian noise , concluding that the mean and the variances converge to the true values , see for an example .it is not immediately clear how to obtain an analogue of corollary [ cor : mdlstabilizes ] for continuous densities .since the assertions on static and dynamic mdl have been proven generally for _ semimeasures _ , we may apply them to the universal setup . here is the countable set of all lower semicomputable (= enumerable ) semimeasures on .so contains stochastic models in general , and in particular all models for computable deterministic sequences .there is a one - to - one correspondence of to the class of all programs on some fixed universal monotone turing machine , see e.g. . we will assume programs to be _ binary _ , in contrast to outputs , which are strings .this relation defines in particular the complexities and weights of each by [ eq : canweights ] w ( ) = , w_=2^w ( ) .we call these weights the _ canonical weights_. they satisfy for all and .an enumerable semimeasure which dominates all other enumerable semimeasures is called universal .the bayes mixture defined in ( [ eq : xi ] ) has this property .one can show that is equal within a multiplicative constant to solomonoff s prior ( * ? ? ?* eq . ( 7 ) ) , which is the a priori probability that ( some extension of ) a string is generated provided that the input of consists of fair coin flips .that is ( x)(x)= _ 2 ^ -(p ) _ all x^*. here , we use the notations fg : fg+o(1 ) , & & fg : fggf , + fg : fgo(1 ) , & & fg : fggf .the mdl definitions in section [ secmdl ] directly transfer to this setup .all bounds on the cumulative square loss ( subsumed in corollary [ cor : murhoall ] ) therefore apply to } ] and }$ ]. then and for all is immediate .it is straightforward that since .moreover , for any string , define the _ monotone complexity _ as the length of the shortest program such that s output starts with .the following assertion holds .we have .we must show that given a string and a recursive measure ( which in particular may be the mdl descriptor ) it is possible to specify a program of length at most that outputs a string starting with , where constant is independent of and .consider all strings ( ) of length arranged in lexicographical order .each has measure .let be the cumulated measures : and .let be the index of , i.e.  .then , the interval has measure and therefore contains exactly one -bit number .we describe with the number , this is known as _ arithmetic encoding _ ( see e.g.  ) .the coding is injective since and are disjoint for .  in order to decode , we may descend the -ary tree of all possible strings , first considering strings of length one , then of length two , etc .for each possible string , we can determine its binary code by approximating sufficiently accurately .eventually we will find , then we print the current . at this stage , might be only a prefix of , since an extension of might have a measure very close to and thus map to the same code .therefore we continue the procedure until all codes starting with are proper extensions of ( which may never be the case , then the algorithm runs forever ) . in each step , the appropriate additional symbol is written on the output tape .the resulting output will be or some extension of .this algorithm can be specified in a constant number of bits .the description of needs another bits .finally , has length .thus , the overall description has length as required .it is also possible to prove the proposition indirectly using ( * ? ? ?* thm.4.5.4 ) .this implies that for all and all recursive measures .then , also holds .so together with the above observations , we have [ eq : complexityrelations ] ( x ) ( x ) ( x ) ( x )  ( x ) . on the other hand , there is a deep result in algorithmic information theory which states that an exact coding theorem does _ not _ hold on continuous sample space ,  .therefore , at least one of the above must be proper .[ problemgacs ] which of the two inequalities and is proper ( or are both ) ?the proof in is very subtle , and the phenomenon is still not completely understood .there is some hope that by answering problem [ problemgacs ] , one arrives at a better understanding of the continuous coding theorem and even at a simpler proof for its failure .in this last section , we recapitulate the main achievements of this work and discuss their philosophical and practical consequences . in the first place , we have shown that if two - part mdl is used for predicting a stochastic sequence , then the predictive probabilities converge to the true ones in mean sum , provided that the distribution generating the sequence is contained in the model class .the two most important implications are almost sure convergence and loss bounds for arbitrary loss functions .the guaranteed convergence is slow in general : all bounds depend linearly on , the inverse of the prior weight of the true distribution . for large model classes ,this number must be regarded too huge to be relevant for practical applications .examples show that this bound is sharp .this is in contrast to the exponentially smaller corresponding bound for the bayes mixture .the latter predictor however is often computationally more expensive to approximate in practice .we believe that this principally indicates that with mdl , some care has to be taken when choosing the model class and the prior .conditions which are sufficient for fast convergence have been given for instance in .it remains a major challenge to generalize these results in order to obtain fast convergence under assumptions that are as weak as possible .in particular for universal induction , this question is interesting and possibly difficult .even when considering only computable bernoulli distributions endowed with a universal prior , fast convergence possibly holds for many environments , but maybe not for all .we also need to distinguish how the large error cumulates .either the instantaneous error remains significant for a long time , which is critical , or the instantaneous error drops just too slowly to be summable , e.g. as , which is tolerable .we have seen instances for both cases ; compare the discussion after example [ ex : bernoulli ] . in this light , the cumulative error might not be the right quantity to assess convergence speed .the main results have been shown under the only assumption that the data generating process is contained in the model class .this condition is essential in general , as shows that in its absence mdl can fail dramatically . in the universal setup, the assumption merely requires that the data is generated in some ( probabilistically ) computable way .this is a very weak condition .laplace , zuse and successors argue that nature operates in a computable way , and consequently _ all _ thinkable data satisfies the assumption . on the other hand ,predicting with a universal model is computationally very expensive .in particular it is provably infeasible if the thesis of computable nature holds . despite these practical problems ,the theory of universal prediction is valuable since it explores the limits of computational induction .m. hutter .sequence prediction based on monotone complexity . in _ proc .16th annual conference on learning theory ( colt-2003 ) _ , lecture notes in artificial intelligence , pages 506521 , berlin , 2003 . springer .

Abstract: minimum description length ( mdl ) is an important principle for induction and prediction , with strong relations to optimal bayesian learning . this paper deals with learning non-i.i.d.processes by means of two - part mdl , where the underlying model class is countable . we consider the online learning framework , i.e. observations come in one by one , and the predictor is allowed to update his state of mind after each time step . we identify two ways of predicting by mdl for this setup , namely a _ static _ and a _ dynamic _ one . ( a third variant , hybrid mdl , will turn out inferior . ) we will prove that under the only assumption that the data is generated by a distribution contained in the model class , the mdl predictions converge to the true values almost surely . this is accomplished by proving finite bounds on the quadratic , the hellinger , and the kullback - leibler loss of the mdl learner , which are however exponentially worse than for bayesian prediction . we demonstrate that these bounds are sharp , even for model classes containing only bernoulli distributions . we show how these bounds imply regret bounds for arbitrary loss functions . our results apply to a wide range of setups , namely sequence prediction , pattern classification , regression , and universal induction in the sense of algorithmic information theory among others .  minimum description length , sequence prediction , consistency , discrete model class , universal induction , stabilization , algorithmic information theory , loss bounds , classification , regression .


markov state models ( msms ) have become an integral part of the toolbox used to analyze the output of molecular dynamics ( md ) simulations of complex systems such as proteins and other large biomolecules  .they were developed in response to the need to process ever longer md timeseries data , made either of one long trajectory or very many shorter ones , generated e.g by special - purpose high - performance computers  , high - performance gpus  , or massively parallel simulations  .the basic idea of msms is to represent the original dynamics as memoryless jumps between predefined states in the configuration space of the molecular system . under this markovian assumption, md timeseries data can then be processed via inference techniques such as maximum likelihood estimation to calculate the rate matrix between these states .this matrix defines a markov jump process ( mjp ) , which in turns permits the calculation of interesting kinetic quantities of the system on time scales that may be larger than those reached in the md simulations : indeed msms also permit to recombine short simulations run in parallel to extract long time information about the system .  a recurrent issue in the context of msms is how to pick the states on which to map the original dynamics see refs . for some recent works in this direction .indeed this mapping amounts to a drastic coarse - graining of the dynamics , and the jumps between poorly chosen coarse states will not be markovian in general .this invalidates the basic assumption of msms and affect their reliability and accuracy .fortunately , md systems typically display metastability and this offers a way around this difficulty . in metastable systemsthere exists regions that play the role of hubs : after visiting one such hub , the system returns often to it before making a transition to another .this guarantees that transitions between these hubs is indeed approximately markovian , and metastability has therefore been invoked as they key property to justify msms and assess their accuracy ( see e.g. [ ] and [ ] for modern perspectives on the topic that summarizes this viewpoint ) . what remains mostly open, however , is how to identify these hubs in practice .  in the present paper we aim at addressing this question in the context of milestoning - based msms combine the core set method originally introduced in ref .[ ] with the milestoning of the trajectories developed in refs .unlike standard msms that are based on a full partition of the configuration space of the system into blocks that are used as states  , milestoning - based msms uses non - adjacent core sets as states ( the milestones ) , and assign the md timeseries to the index of the last such milestone it visited .this maps the original dynamics onto a symbolic dynamics on these indices that is then used as input to build the msm by maximum likelihood estimation of its transition matrix . in metastable systems ,the proper milestones to use should be the hubs mentioned before .here we propose to identify these hubs among a set of trial milestones via optimization of a metastability index that measures how good the hubs are .the method can be justified within the framework of the potential theoretic approach to metastability developed by bovier and collaborators  .it has the advantage that it can be used even in situations where the dynamics on the trial milestones is non - markovian . in this sense, it alleviates a difficulty with the standard approach used to build msms via clustering of trial states  : this clustering is typically done using spectral analysis of the rate matrix of the chain built on these trial states , which may lead to artifacts since the dynamics on these trial states is non - markovian in general .the method we propose avoids this difficulty altogether .in addition it avoids the need to introduce a time - lag to process the data , which may be difficult to adjust .as we will see , our method also offers a way to partition the state space of the system by identifying regions made of configurations most likely to reach a given hub , and to identify the members of the transition state ensemble as those trial milestones that have a non - negligible probability to reach more than one target milestone next .the remainder of this paper is organized as follows . in sec .[ sec : algo ] we start by presenting the algorithmic aspects of the method we propose , including how to define the trial milestones ( sec .[ sec : trialvstarget ] ) and the metastability index ( sec .[ sec : metaindex ] ) , how to identify the target milestones that optimize this metastability index ( sec .[ sub : how - to - find ] ) , and how to build the msm on these target milestones ( sec .[ sec : msm ] ) .we also introduce a variety of diagnostic tests that can be used _ a posteriori _ to analyze the output of the msm and use the trial milestones to get additional information about the system s dynamics ( sec .[ sec : diagno ] ) . a theoretical justification of this algorithmis then given in sec .[ sec : theo ] , first in the context of markov jump processes , which is relevant e.g. if one assumes that the dynamics on the trial milestones is itself markovian ( sec .[ sec : ctmc ] ) , then in the context of systems whose configurational space is continuous , like those encountered in md simulations , where we can not expect the dynamics on the trial milestones to be markovian ( sec . [sec : csp ] ) .we also test the method on a simple one - dimensional example with a multiscale energy landscape ( sec .[ sec : illustrativeex ] ) . in sec .[ sec : glyalagly ] we then apply the method to analyze the dynamics of a gly - ala - gly peptide , and in sec .[ sec : beta3s ] we use it to analyze the folding pathways of the beta3s mini - protein .concluding remarks are given in sec .[ sec : conclu ] .some technical derivations are relegated to an appendix .in this section , we outline the algorithm we propose to identify the milestones ( or core sets ) over which to build an msm . in a nutshell , this is done by picking among a set of trial milestones a subset of target milestones which minimizes a metastability index this index is defined so that small values are indicative of markovianity .the target milestones are taken as states in the msm , while the trial milestones are used to complement the predictions of this msm and e.g. partition the system s configurational space or identify transition state regions in it .   milestoning procedure .( a ) : piece of a long trajectory crossing a set of three circular milestones , , , and .( b ) : the trajectory shown in ( a ) is mapped onto the index of the last milestone it hit , thereby defining the piecewise constant function . ]denote by a trajectory containing the instantaneous position of the atoms in a molecular system .we assume that we have generated one or several such trajectories and our goal is to build a msm that captures their main features . to this end, we introduce a set of  trial milestones , which we will denote by , , , and label them by their index , i.e.  identifies  .these milestones are disjoint sets in the system s configuration space that can be defined e.g. by requiring that some of the dihedral angles of the molecules take values between specific bounds , etc . how to actually choose the trial milestones  will be illustrated below on specific examples . in the spirit of milestoning, we then map each trajectory onto the index of the last trial milestone it hit , see fig .[ fig : milestoning ] for an illustration . this way we obtain a piecewise constant index function value at time is the index of the last milestone hit by  .note in this procedure we discount recrossings : we only update the index function when a new milestone is hit .we stress that at this stage we do not assume that the dynamics of the index function  on the trial milestones is markov in general it will not be .what we would like to do next is extract from the set of trial milestones a subset of target milestones such that if we map the trajectory  onto this subset of target milestones , the corresponding index function will be approximately markovian these target milestones are shown in red in the cartoon shown in fig . [fig : metastabilityscheme ] . in the sequel, we will denote by the set of indices identifying the target milestones , i.e. these are .we will also refer to trial milestones that are not target ones as non - target milestones .how should the target milestones be chosen ?intuitively , they should be such that they are hubs among the trial milestones towards which the trajectory is attracted but between which it seldom makes transitions , as illustrated in fig . [fig : metastabilityscheme ] . to make this idea concrete ,let us first introduce the probability that , if the trajectory hits milestone , then subsequently it will hit with before hitting again ( discounting recrossings : recall that only hits of different milestones are counted in other words , to hit again , the trajectory must have hit at least one other milestone in between ) . to estimate , out of each of the piecewise constant index function , we first extract the sequence of successive values that this function takes for example , for the timeseries illustrated in fig .[ fig : milestoning ] this sequence starts with , , , , etc .we then cut this sequence into the pieces which start from and contain all the indices visited after before appears again .for example , if the sequence is made of 3 indices , , and reads we cut it into three pieces ( ) finally we count the number of pieces in which appears at least once ( in the example above since appears in the last two pieces but not in the first ) , we add up these numbers coming from every piece of at our disposal , and we set as estimator for .note that the quality of this estimator depends on the lengths of the pieces of timeseries that we have at our disposal , and how to assess the statistical accuracy of  is nontrivial . as usual , we can not expect the estimator to be accurate if these pieces are too short to observe all the relevant events in the dynamics of the system : in the context of  this requires that these pieces be long enough that trajectories starting at a non - target milestone have time to reach a target one .indeed , as we will see below , this condition is sufficient to guarantee that we will correctly identify target from non - target milestones among the trial ones .we will now use the matrix with entries to quantify how good a set of target milestones will be to build an msm . specifically , given a candidate identifying the target milestones , we estimate the quality of these milestones via their metastability index defined as the smaller , the better the set of target milestones identified by .this claim will be justified in sec .[ sec : theo ] by connecting to a quantity originally introduced by bovier , but let us briefly explain here why it is true .the numerator in , identifies the target milestone for which the probability is the highest that , after hitting , the trajectory will hit some other target milestone before hitting again . in this sense , is the worst target milestone in the set since we would like that transitions between these target milestones be unlikely , and the smaller the better .correspondingly , the denominator in , identifies the non - target milestone which is such that the trajectory has the lowest probability to hit a target milestone before hitting again . in this sense, is the worst non - target milestone since we would like that transitions from non - target to target milestones be likely , and the larger the better . the metastability index in  accounts for both the desiderata that be small and  be large , and in this sense it measures the quality of the set of target milestones identified by .schematic representation of a subset of target milestones ( shown in red ) immersed in a set of trial milestones .a good subset of target milestone is such that transitions from non - target to target milestones are likely , while transitions between target milestones are not .this can be quantified via the metastability index defined in . ]   since good sets of target milestones are those whose metastability index is small , we can systematically search for such good sets by minimizing . in principle , this can be done by considering increasing values , , etc . of the cardinal of , and for each compute for every choice of so as to identify the one with minimum . any choice which is small leads to a good set of target milestones .note however that can be small for different values of , and so several of them should be considered this effect will be illustrated in the examples below .of course , if the number of trial milestones is large , this direct search strategy will quickly become inefficient as increases since the number of sets to consider for each is to avoid this difficulty , we must adopt more efficient optimization strategies , for example using monte carlo schemes .we have used such schemes in the examples below .however , we found that the following searching strategy was typically the most efficient .given , we can identify the index of the milestone that the trajectory is most likely to hit after hitting  as the one that maximizes over all , i.e. the function can be used to define ` trajectories ' in index space : given , we update it to , then to , etc .correspondingly , given a set we can update this set by updating every entry in it , and only keeping the ones that remain different ( for example we could have in which case only one of these entries is kept in the update of ) . because this updating identifies set of indices of milestones that are likely to be hit , these sets should have small metastability index , and this is indeed what we observed in practice .specifically , we took random sets of indices with random values of and updated them as described above while monitoring the metastability index of these updated sets .we observed that this metastability index typically diminishes before starting to oscillates in a periodic fashion ( this is because the update has no fixed point , by construction ) . when this happened we stopped the update , kept the updated set with smallest , and restarted the procedure with a different random set of indices .after a few such iterations , we typically got a few different sets ( with different ) with small .once we have identified a good set of target milestones specified by the index set , we can build an msm using these milestones as states . how to do so was explained in refs . , so let us be brief here and refer the reader to the original papers for details .for the sake of clarity , in the sequel it will be convenient to distinguish between target and non - target milestones : we will do so by using greek letters , , ... to refer to the indices in the index set , and , , ... to refer to the target milestones , , ...  similarly to what was done before , we can map the trajectory onto the index of the last target milestone it hit .this defines a piecewise constant function taking values in . because we are now using target milestones instead of trial ones , unlike , should be approximately markov .this means that the evolution of this function can be completely specified by a rate matrix with entries : to leading order in , gives the probability that jumps from the value to the value in the interval ( i.e. that hits the target milestone in that time interval if the last target milestone it hit before time was ) .in particular , if denotes the probability distribution that takes the value at time  , then satisfies the master equation similarly , we can write down equations for the distribution of first passage time from target milestone to target milestone , its mean , etc . in terms of see e.g. refs . .the rate matrix entries can be estimated from the timeseries by the method of maximum likelihood .if we denote by the total time the last target milestone hit by is and by the number of times the target milestone was hit directly after along this timeseries , the maximum likelihood estimator for is this estimator is unbiased in the sense that , if is indeed markov with rate matrix entries and the length of the timeseries tends to infinity , then in this limit . if the length of the timeseries is finite , the statistical errors on can be estimated by bayesian sampling , see ref . [ ] for detail .another source of errors are those due to residual non - markovian effects in .the smaller , the smaller these non - markovian effects are , as discussed in sec .[ sec : theo ] .in practice they can also be estimated via markovianity tests , as illustrated on examples in secs .[ sec : glyalagly ] and [ sec : beta3s ] .notice also that if satisfies detailed balance , then we should have that as the length of the timeseries goes to infinity .when this length is finite , however , in general , and to enforce detailed balance of the msm , it is better to use the following symmetrized estimator for the rate matrix entries : with this choice , the equilibrium distribution of the msm , i.e. the distribution towards which the solution to  with replaced by its estimator converges as , is where is the total length of the timeseries used to estimate .the distribution gives the proportion of time during which was the last milestone hit by and as the length of the timeseries increases , , converges to the true equilibrium distribution of milestone  explicit expressions for this distribution in the context of a system whose dynamics is governed by a markov jump process or a langevin equation will be given in secs .[ sec : ctmc ] and [ sec : csp ] , respectively .we can use to estimate the free energy of milestone  where denotes the reciprocal of the thermodynamic temperature of the system .   even though the trial milestones merely serve as intermediary to construct the actual msm ,these milestones can still be used to analyze the md data and partition the state space in ways that highlights important features of its dynamics .first we can organize the trial milestones onto a network : if denotes the number of times milestone was hit directly after , we put an edge between node and node with weight where is the total length of the timeseries for .note that , consistent with the detailed balance condition , we have symmetrized the weight , i.e. the network is undirected , .  next we can compute the free energy of the trial milestones .if denotes the total time that the last milestone hit was ( so that , the total length of the timeseries ) , we can define the probability distribution the distribution gives the proportion of time during which was the last milestone hit by ( see secs . [sec : ctmc ] and [ sec : csp ] for its expression in the limit as when the system s dynamics is governed by a markov jump process or a langevin equation , respectively ) , and we can estimate the free energy of the trial milestone  via : the free energy defines a landscape on the network of trial milestones . one may expect that the nodes associated with the target milestones will be close to the local minima of ; however , we stress that this need not be the case , since the dynamics on the full network ( rather than the one restricted to the target milestones alone ) can be quite complicated ( in particular , not markovian ) .to partition the state space , it is better to introduce the committor functions of the trial milestone  with respect to the target milestone , . by definition , gives probability that , after hitting , the trajectory will hit the target milestone before hitting any other target milestones , and it can be estimated from the timeseries as where ( so that ) .the committor functions can be used to do a ( soft ) partitioning of the network into basins of nodes that are more likely to be attracted next to one target milestone rather than any other : for example , those nodes  such that is close to 1 are associated with milestones out of which the trajectory is very likely to hit next .a hard partitioning can also be performed by assigning to .the committor probability whose estimator is given in  can also be used to identify the transition state ensemble ( tse ) , i.e. the trial milestones that lie in between the states in the target set . if the target set contains only two states , , it follows that all trial milestones satisfy and the tse is such that . for target sets with more than 2 statesthe criterium may become less effective as the tse as it can in general connect multiple states . to get around this difficulty, we can introduce a tse index based on the entropy of the committor probability .recalling that for any milestone ( i.e. is a probability distribution in ) , we propose to use the normalized entropy ( sometimes also referred to as efficiency ) of as tse index : were the sum is carried out over the non - zero entries of and is the number of such entries ( more generally , we could restrict the sum to the entries of that are above some small threshold ) .the tse index is 1 if the non zero values of are all identical , and it is 0 if only one value of is different than zero .therefore , the closer is to 1 , the higher the chance that state be a member of the transition state ensemble .once the states in the tse have been identified , we can go back to their committor values to determine between which target states they lay .  to characterize the physical origin of the metastability of the target milestones, it is also useful to decompose their free energy into an energetic component and an entropic one .this can be done as follows .given the system s potential energy , a mean energy can be assigned to each of the target milestones via where the total time the last target milestone hit by is and so that .we can then estimate the entropy of the target milestone via where is the free energy estimated in  and is boltzmann constant .target milestones with comparable have similar statistical weights , and the lower the more thermodynamically stable they are ( meaning the timeseries tends to return to them more often , comparatively ) : by comparing their values of and we can then determine whether their stability is of energetic or entropic origin , respectively .the usefulness of the diagnostic tools introduced above will be illustrated in the examples treated in secs .[ sec : glyalagly ] and [ sec : beta3s ] .let us now justify the use of the metastability index defined in  to identify target milestones over which the dynamics can be mapped in an approximately markovian way .we begin by discussing the simpler case when the dynamics of on the trial milestones is itself markovian as we mentioned in sec .[ sec : algo ] , we do not make this assumption in general , but it is a convenient starting point for our theoretical explanation .the general situation when is not markovian will be discussed in sec .[ sec : csp ] .if is markov , its dynamics is specified by a rate matrix whose entries we will denote by to distinguish them from the rate matrix entries on the target milestones . assuming detailed balance , satisfies where is the number of trial milestones and is their equilibrium probability density the estimator for was given in .together with the assumption of ergodicity ( i.e. that time averages along converge towards ensemble averages over ) , implies that all the eigenvalues of are real , with one being 0 and all the other negative .we will denote these eigenvalues by , and order them so that latexmath:[12 & 12#1212_12%12[1][0] * * ,  ( ) * * ,  ( ) link:\doibase  10.1063/1.2714538 [ * * ,  ( ) ] * * ,  ( ) link:\doibase  10.1063/1.3590108 [ * * ,  ( ) ] _ _ , vol .( , ) _ _ , vol .( , ) * * ,  ( ) * * ,  ( ) * * ,  ( ) link:\doibase 10.1021/jp411822r [ * * , ( ) ] link:\doibase  10.1021/ct500353 m [ * * ,  ( ) ] ( ) ,  * * ,  ( ) link:\doibase 10.1063/1.1738640 [ * * , ( ) ] link:\doibase 10.1063/1.2996509 [ * * ,  ( ) ] link:\doibase 10.1063/1.3129843 [ * * , ( ) ] link:\doibase 10.1021/jp060039b [ * * , ( ) ] * * ,  ( ) * * ,  ( ) * * ,  ( ) http://scitation.aip.org/content/aip/journal/jcp/143/4/10.1063/1.4926940 [ * * ,  ( ) ] * * ,  ( ) * * ,  ( ) * * ,  ( ) * * ,  ( ) link:\doibase 10.1137/070699500 [ * * , ( ) ] * * ,  ( ) * * , ( ) * * ,  ( ) * * ,  ( ) * * ,  ( ) * * ,  ( ) `` ''  * * ,  ( ) * * ,  ( ) * * , ( ) * * ,  ( ) * * ,  ( ) * * ,  ( ) link:\doibase doi 10.1021/jp711864r [ * * ,  ( ) ] link:\doibase 10.1002/prot.21565 [ * * ,  ( ) ] * * ,  ( )

Abstract: a method is proposed to identify target states that optimize a metastability index amongst a set of trial states and use these target states as milestones ( or core sets ) to build markov state models ( msms ) . if the optimized metastability index is small , this automatically guarantees the accuracy of the msm , in the sense that the transitions between the target milestones is indeed approximately markovian . the method is simple to implement and use , it does not require that the dynamics on the trial milestones be markovian , and it also offers the possibility to partition the system s state - space by assigning every trial milestone to the target milestones it is most likely to visit next and to identify transition state regions . here the method is tested on the gly - ala - gly peptide , where it shown to correctly identify the expected metastable states in the dihedral angle space of the molecule without _ a priori _ information about these states . it is also applied to analyze the folding landscape of the beta3s mini - protein , where it is shown to identify the folded basin as a connecting hub between an helix - rich region , which is entropically stabilized , and a beta - rich region , which is energetically stabilized and acts as a kinetic trap .


the rise of economic inequality has become , in recent times , cause for serious concern and the object of frequent debates .inequality is increasing in several countries and also appears to be accompanied by a diminution of social mobility .this combination is especially harmful , because it frustrates the attempts of many individuals to move up the income ladder and improve their situation .empirical surveys confirm the existence of a negative correlation between mobility and inequality .it seems therefore very important to highlight the fundamental causes of this correlation .our aim in this paper is to explore these themes through mathematical methods , by providing a suitable definition of mobility and by comparing the possible scenarios compatible with different policies and conditions . in this connection, we introduce some indicators apt to measure various aspects of mobility within a family of kinetic - type models for the description of economic exchanges in a closed society .we investigate the relationship between economic inequality and mobility by analyzing income distributions emerging from different fiscal and welfare systems .the output of a large number of simulations shows clear evidence of a negative correlation between mobility and equality , in agreement with the qualitative features of real world situations. the models also allow to study the simultaneous dependence of the gini index and the mobility parameters on taxation rates and welfare means - testing , and to track variations in the mobility of different income classes .in particular , it turns out ( the simulations suggest ) that there are , with reference to variables measuring welfare and taxation , lines on which both the gini index and the mobility parameter are constant .our approach fits in with a complex system perspective , in the sense that we find the emergence of collective macro - level patterns out of a whole of interactions occurring at the micro level .the need of considering , when dealing with socio - economic questions , a multiplicity of individual interactions rather than a representative agent is highlighted and advocated e.g. in .for this purpose , agent based computational models have been typically developed and applied within the economics community , . on the other side , during the last two decades a number of physicists have started to tackle economic and financial questions by using tools of statistical physics , see e.g. and the recent review .the reason for that naturally arises from certain analogies existing between markets or populations of individuals who exchange money and particle systems or gases in which molecules collide exchanging energy . in particular , as illustrated below , our methods and models allow , thanks to their characteristic features , the definition and investigation of a mobility concept .  the remainder of the paper is organized as follows .section sets up a framework for the description of monetary exchanges in a closed society . in section a family of models is constructed within this framework . in the fourth section the concept of social mobilityis shortly discussed and quantitative indicators to measure it are introduced . hence , the correlation between the economic inequality and mobility is explored .finally , a summary is contained in the last section .moving from a stylized description of the complex of monetary exchanges which take place in a closed market society , we outline in this section a discrete - type framework suitable for its modelling .this framework was proposed and discussed in our previous works , but we give here a slightly different introduction .the problem at issue concerns the formation of the income distribution curve and the economic inequality in a population of constant size .if the variable denotes the individual income and the density of the population having income , one could say that the total population and the total income are given by and respectively . however , it is quite natural and also convenient in view of a numerical treatment of the problem to divide the population into a finite number of classes . toward this goal, we consider the numbers with so large as to represent a sort of supremum of the possible income of any individual in the given society .for simplicity we assume the density of the population whose income belongs to the interval to be constant and denote it by for .the fraction of individuals having income in is then and its income is we refer to the class of individuals with income in as to the -th class and to its individuals as to -individuals .an immediate consequence of and is that , where denotes the average income of individuals in the -th class .we assume that pairwise monetary exchanges take place between single individuals .a `` microscopic '' description of this can be as follows : when an -individual pays to a -individual an amount of money ( with for all ) , the -th class gets slightly poorer , while the -th class gets slightly richer .this induces a retrocession of a portion of the population of the -th to the -th class and an advance of a portion of the -th population to the -th class .easy calculations show that , if the total population and the total income have to remain unchanged as happens in the conservative case dealt with here , then the variations of population in the -th and in the -th class are given by how often monetary exchanges can occur can be postulated through the definition of suitable terms for , with each accounting for the encounter frequency rate of individuals of the -th and in the -th class and expressing as well the probability that in an encounter between an -individual and a -individual , the one who pays is the -individual .the possibility that none of the two pays has to be taken into account as well .hence , for any and in , the inequalities and have to hold true . apart from that , the values of the should be determined based on phenomenological observations .a specific choice for the values of the will be done in the third section of this note .  beside direct money exchanges, we also assume the existence of a process of taxation and redistribution . if we imagine that the money received form the -individual is subject to taxation with a tax rate equal to , supposing ( at least as a first case study ) that the -individual pays the due tax , we observe that the entire operation is equivalent to one in which the -individual pays a quantity , the -individual earns a quantity , and the quantity goes to the government .we then imagine , here again postulating an irreproachable situation , that the government redistributes the tax revenue ( which is just for the single mentioned interaction ) to the entire population through the provision of welfare services : education , health assistance and so on .at this step , we observe that in fact the government intervention can be ignored , since the mechanism can be equivalently described as follows : in correspondence to the mentioned _ direct _ interaction between an -individual and a -individual , individuals of all classes receive an amount of money corresponding to a returned tax portion .in other words , the taxation and redistribution effect is equivalent to the effect of an _ indirect _ interaction between the paying -individual and all others .- th class . indeed ,if an individual of the class would receive some money , the possibility would arise for him to advance to a higher class , which is impossible .] of course , the indirect interactions too contribute to the possibility of class change of individuals .  in order to formalize the overall mechanism described so far we introduce the _ direct transition probabilities _  ] for , where each }^i(x) ]is accomplished combining the variation terms and the probabilities .the variation in time of the number of individuals belonging to the income classes is described by the following balance differential equations : }^i(x ) x_h x_k \qquad \hbox{for } \i \in \{1 , ... ,n\ } , \label{evolution eq of p}\ ] ] which , in view of , can be simplified to take the form : }^i(x ) { \big ) } \ , x_h x_k - x_i \sum_{k=1}^n x_k \qquad \hbox{for } \i \in \{1 , ... , n\}. \label{evolution eq}\ ] ] they are nonlinear ordinary differential equations .in order to construct a specific model or model family , we start by fixing specific values for the . we take them as with the exception of the terms for , for , for , for and for .this choice is suggested by the consideration that poorer individuals typically earn and pay smaller amounts than richer ones and do so less frequently .it guarantees a degree of heterogeneity among people of different classes , also in connection with their interactions with others .recalling now the formulae , we assume that the only possibly nonzero elements among the are : we stress that the expression for in holds true for and ; the second addendum of the expression for is effectively present only provided and , while its third addendum is present only provided and ; the expression for holds true for and .  as for the }^i ] as }^i(x ) =  u_{[hk]}^i(x ) + v_{[hk]}^i(x ) , \label{t+u}\ ] ] where}^i(p)}) ] , and the terms involving the index [ respectively , are effectively present only provided [ respectively , . in other words : the term into parentheses on the r.h.s . of ( [ v_[hk]^i(p ) ] ) is present for ; the term into parentheses on the r.h.s . of ( [ v_[hk]^i(p ) ] ) is present for . ]}^i(x ) =  p_{h , k } \ , 2s \ , \tau_k \, \frac{\sum_{j=1}^{n-1 } w_j x_{j}}{\sum_{j=1}^{n } w_j x_{j } } \ ,  { \bigg ( }  \frac{\delta_{h , i+1}}{r_h - r_{i-1 } } \ , - \ , \frac{\delta_{h , i}}{r_h - r_{i-2 } } { \bigg ) } , \label{v_{[hk]}^i(p)}\ ] ] with denoting the _ kronecker delta _ , represents the variation corresponding to the retrocession from a class to the preceding one , due to the payment of some tax .we notice here that the effective amount of money representing taxes , which is paid in correspondence to a payment of and is then redistributed is instead of .this is in agreement with the fact that , due to the bound on the admissible income , individuals of the -th class do not benefit from the redistribution ( see note ) .finally , to fix ideas , we take , and for , where and respectively denote the minimum and maximum tax rate .still , the value of , and have to be fixed .hence , the equations describe a family of models rather than a single model .finding analytical solutions for these equations is in practice hopeless .but , as will be shown in the next section , several relevant facts can be understood through simulations . before passing to numerical results, we would like to mention the following two properties of .for both of them the proof of the corresponding result established in and referring in fact to a slightly different model can be easily adapted .  *well posedness of the cauchy problem*. for fixed values of the parameters , and , in correspondence to any initial condition , for which for all and , a unique solution of exists , which is defined for all , satisfies and also   * conservation of the total income *. for fixed values of the parameters , and , the scalar function , expressing the total ( and mean ) income , is a first integral for the system .  from now on ,we report on the outputs of a large number of simulations . to run them , we fixed .the first fact one observes is the following one .* uniqueness , for any fixed value of , of the asymptotic stationary distribution*. for fixed values of the parameters , and , for any fixed value  ] , representing complete equality and maximal inequality .the tax revenue is the total amount of tax collected in the unit time and redistributed as welfare provisions : with denoting the fraction of individuals in the -th class at equilibrium .this paper focuses upon an important socio - economic issue : social mobility .social mobility gives a measure of the chances individuals have to improve their economic status through education and work .it is known that a high level of inequality in a society is especially harmful when it is complemented by a diminution of social mobility , see e.g. .social mobility has been historically an important component of the `` american dream '' ( and of its emulations ) , thus making the high inequality level present in the us more tolerable . today, several observers point out a diminution of social mobility , as seen for instance from the growing difficulties that the sons and daughters of the poor and middle classes have to access the best universities and the best paid jobs . as mentioned in the introduction ,quantitative empirical surveys show that mobility is negatively correlated with inequality , i.e. more unequal societies also tend to have less mobility .studying the correlation between economic inequality and mobility is of particular interest because statistical evaluations of the economic mobility of a society are more difficult than measurements of the income distribution .this is due to the fact that they require to follow the evolution of the individuals income for at least one or two generations .correspondingly , in micro - to - macro theoretical models of economic exchanges based on kinetic equations , the income distribution depends only on the asymptotic equilibrium solution of the equations , while mobility estimates also involve the detailed structure of the transition probabilities of the model .  in our modelsthere are no generations and it is impossible to compare the income of one individual with that of his or her parents .individuals are supposed to live and work indefinitely , or at least for the time it takes for the system to reach its dynamical equilibrium . during this time the income distribution evolves from some initial conditions to a stationary state , in correspondenceto which the number of individuals belonging to a certain class remains constant in time : at equilibrium , the total rate of individuals leaving that class is equal in absolute value to the total rate of individuals arriving from other classes .this means that in the equations , whose right hand side can now be written in a simplified form in view of , one has .equivalently , the equilibrium populations are solutions of the nonlinear equations }^i(x ) { \big ) } \ , x_h x_k  - x_i = 0 \qquad \hbox{for } \i \in \{1 , ... , n\},\ ] ] with and }^i(x) ] and the four values .,width=264,height=193 ]in this paper microscopic models of economic exchanges , taxation and redistribution , described by kinetic equations are considered .some features of the emerging income distribution curves are discussed . in particular , we compare models with fiscal systems characterized by a different spread between the minimum and the maximum tax rate and models in which the welfare provision is weighted differently for different income classes .the main contribution of the paper is represented , in our opinion , by the introduction of a measure of social mobility .this is possible due to the structure of the equations , which involve several transition probabilities for the passage from an income class to others .together with the gini index , this measure serves as a tool to investigate the correlation between economic inequality and social mobility .the correlation turns out to be clearly negative : a lower mobility corresponds to a higher inequality .this is in agreement with what is observed in real world situations .while this negative correlation has been the object of several studies and debates in the socio - economic arena , to our knowledge it has not been treated within the brand of mathematical and physical approaches to complex systems .gallegati m. , _ reconstructing economics : agent based models and complexity _ , conference paper , berlin , paradigm lost : rethinking economics and politics .available at http://ineteconomics.org/conference/berlin/reconstructing-economics-agent-based-models-and-complexity , ( 2012 )

Abstract: statistical evaluations of the economic mobility of a society are more difficult than measurements of the income distribution , because they require to follow the evolution of the individuals income for at least one or two generations . in micro - to - macro theoretical models of economic exchanges based on kinetic equations , the income distribution depends only on the asymptotic equilibrium solutions , while mobility estimates also involve the detailed structure of the transition probabilities of the model , and are thus an important tool for assessing its validity . empirical data show a remarkably general negative correlation between economic inequality and mobility , whose explanation is still unclear . it is therefore particularly interesting to study this correlation in analytical models . in previous work we investigated the behavior of the gini inequality index in kinetic models in dependence on several parameters which define the binary interactions and the taxation and redistribution processes : saving propensity , taxation rates gap , tax evasion rate , welfare means - testing etc . here , we check the correlation of mobility with inequality by analyzing the mobility dependence from the same parameters . according to several numerical solutions , the correlation is confirmed to be negative .   _ keywords _ : kinetic models , taxation and redistribution , economic inequality , social mobility   + _ faculty of science and technology , free university of bozen - bolzano , italy , marialetizia.bertotti.it + faculty of science and technology , free university of bozen - bolzano , italy , giovanni.modanese.it + _