an important aim of cosmology is to determine a relationship between the cosmic density contrast and the peculiar - velocity field under the assumption that these fields have evolved under the action of gravity . in the eulerian theory of gravitational instability , retaining only the linear growing - mode solution , this relationship is simply given by and denote ( comoving ) eulerian and lagrangian coordinates respectively . ] where is the usual linear growth factor , such that in an einstein - de sitter universe . beyond linear theory , in the weakly non - linear regime , ( [ linear ] ) has been used to good effect as an estimate of the density contrast field from observed peculiar - velocities ( e.g. bertschinger & dekel 1989 ; dekel et al .1990 ; bertschinger et al .on the other hand , the potent method and other quasi - linear reconstruction procedures involve the zeldovich approximation ( za)(nusser et al .1991 ; nusser & dekel 1992 ) .the za does not however , strictly speaking , provide an exact algebraic relation between the fields and .it can however yield ( as shown in nusser 1991 ) a relation between the fields that is self - consistent within the order of the approximation , i.e. within . in the za , from the lagrangian integral of the continuity equationwe have  ^{-1}\;,\label{zeldovich}\ ] ]  and the fields are mapped back to eulerian space via the za map for the orbits of the particles .initial conditions are specified at an arbitrary early time . also , it is customary to make the assumption of smoothness at early times , , by virtue of the amplitude of the fluctuations in the microwave background . from ( [ zarecons ] )we obtain the linear relation ( [ linear ] ) by truncating the expansion of the determinant ( [ zeldovich ] ) to the lowest order in the perturbation field and replacing lagrangian by eulerian coordinates ( ) .these schemes have been generalized further ( gramann 1993a , 1993b ; for their eulerian analogue , see bernardeau 1992 ) by starting out with a parametrization of the particle orbits of the type :    in the case of a flat universe , the leading second - order term in perturbation theory ( bouchet et al .1992 ) is indeed .therefore , the coordinate map ( [ moutarde ] ) is in principle well - motivated ( for a numerical comparison of the different models , see e.g. dekel 1994 ) .however , as shown in gramann ( 1993b ) , orbits of the type ( [ moutarde ] ) do not yield a self - consistent relation for the cosmic density and velocity fields ; the density fields obtained via ( a ) the continuity equation and ( b ) the euler equation differ within the order of the approximation .the reason for this is that , as we shall see , ( [ moutarde ] ) is only an approximate second - order solution of the lagrange - newton system .thus , we wish to call the attention of the reader to the fact that the reconstruction models derived from ( [ moutarde ] ) and , more generally , from an ansatz including higher orders in ,    ( for arbitrary n ) , do not follow a rigorous line of analysis ( in the case of an arbitrary ) for the following reasons :  in these models , the parametrization of the orbits is given ad hoc and it is not derived as a solution of the lagrangian evolution equations for the flow field . in particular , the absence of lower - order growing modes , that are present in the general perturbation theory solution , implies that the derivation of through the continuity equation neglects couplings of these modes with leading - order terms .the density contrast obtained from the continuity equation in perturbation theory and satisfying euler s equation are mutually incompatible .there is no reason to justify that is a good perturbative parameter ( in terms of the convergence of the solutions ) and that a meaningful generalization of the family of solutions where the za belongs can be realized as a polynomial in .it is easy to see that ( ii ) follows as a consequence of ( i ) .regarding ( i ) and ( iii ) , it has been shown that the correct coordinate map between eulerian and lagrangian coordinates in lagrangian perturbation theory is obtained by calculating the perturbative solution of the lagrange - newton system for the trajectories .these evolution equations are obtained by transforming the euler - newton system to lagrangian coordinates and eliminating all eulerian fields by using exact lagrangian integrals for the eulerian acceleration and density ( buchert & gtz 1987 ; buchert 1989 ) . within this scheme ,the za is recovered as a subclass of solutions ( buchert 1992 ) , and hence , one can obtain a self - consistent solution for the over - density field in terms of the velocities as in nusser ( 1991 ) .furthermore , the second- and third - order solutions obtained in the lagrangian framework ( buchert & ehlers 1993 , buchert 1994 ; and in a slightly different approach bouchet et al .1992 , 1995 ; catelan 1995 ) are unique and well - defined at all times from a fiducial time , where initial data are specified , , an arbitrary but sufficiently early epoch , not necessarily . ] until shell - crossing , provided we impose periodic boundary conditions and fix some global gauge conditions ( ehlers & buchert 1997 ) as opposed to the orbits described by polynomials in , which are expected to have poor convergence ( for a discussion in the context of the variational approach to the solutions see susperregi & binney 1994 ) .although the leading terms of the longitudinal parts of the perturbation solutions seem to confirm this polynomial approach ( e.g. bouchet et al .1992 , for the second - order solutions ) , this is not a mathematically consistent motivation for using as an expansion parameter by going to higher orders in perturbation theory . on the contrary ,the structure of the lagrangian perturbation scheme is such that , starting at third order , the presence of interaction terms among perturbations of different orders are an obstacle to convergence , and in fact for all terms are interaction terms ( see buchert 1994 ) .therefore , for an ansatz of the form ( [ d ] ) can not reproduce gravitational dynamics , as stated in ( iii ) .this is an especially critical point to bear in mind in the context of reconstruction models in which higher - order couplings of flow field gradients show up in the density - velocity relation .  in this paper, we approach the reconstruction problem from a purely lagrangian perspective using the solutions of the lagrange - newton system obtained by buchert & ehlers ( 1993 ) . the goal of the paper is to obtain a self - consistent solution for the over - density field that is valid to second order , thus extending the results of nusser 1991 in the context of the za .our results are purely lagrangian , but in obtaining them we rely on the fact that there is a one - to - one mapping between lagrangian and eulerian coordinates and hence , the reconstruction formulae presented are only valid up to orbit - crossing time . the basic formalism is laid out in sect . 2 , and in sect .3 we discuss its application to two different classes of initial conditions . in sect .4 we discuss the self - consistency of solutions in perturbation theory models . sect .5 compares the model presented with previous reconstruction procedures and sect .6 sums up .in what follows , we shall mainly discuss the physically interesting case where the peculiar - velocity field is parallel to the peculiar - acceleration , up to decaying modes , a property which results from the eulerian linear theory for irrotational fields , and is also valid for the general irrotational lagrangian linear solution ( the growing - mode solution supports the tendency to becoming parallel , see buchert ( 1992 ) ) .we will further recall that both fields are practically parallel up to orbit - crossing time , also in the second - order schemes we will consider .the results presented however are not restricted to solutions where this property holds but , as we will see below , apply to a wider class of solutions .  inwhat follows we shall study subclasses of the second - order lagrangian perturbation solutions given by buchert & ehlers ( 1993 ) which are only restricted by the requirement that there exists an arbitrary functional relationship among peculiar - velocity and -acceleration at the initial time .the subclasses are singled out by special assumptions for this functional relationship .the second - order parametrization of the orbits of the particles in terms of their lagrangian coordinates and time for the subclasses of solutions which we consider later can be written as follows :    where and are dimensionless functions of time , calculated from solving the lagrange - newton system for deviations from the homogeneous solution to second order .moutarde et al .( 1991 ) investigated the orbits ( [ eulerian ] ) ( rather than ( [ d ] ) ) , for leading orders of and and for a model of a particular initial potential . in the next section, we will examine two particular solutions for and for different , but generic , initial conditions .the expansion factor is normalized to unity at , , ( and equal to the linear growth factor in an einstein - de sitter universe considered here ) .the perturbation potentials and are determined by the initial data such that , in general , they have to satisfy the ( non - local ) boundary conditions :      where denotes the initial peculiar - velocity potential   , and ( following the summation convention )      are the first and second principal invariants of the peculiar - velocity tensor gradient . for periodic boundary conditions and suitable gauge conditions ,however , which we shall assume hereafter , we can simplify ( [ s1])([s2 ] ) ( see ehlers & buchert ( 1997 ) for a rigourous proof ) :      therefore , ( [ eulerian ] ) gives us the trajectory of any given particle by solving ( [ pbc1])([pbc2 ] ) for an initial potential .at the time , both and vanish , so lagrangian and eulerian coordinates coincide ,    thus , the expression for the density contrast at any given time and for any trajectory field reads :  = [ \,1+\delta(t_0,\b x)\,]\;j^{-1}\ ; , \label{density1}\ ] ]  where the determinant is    in order to calculate as a function of eulerian coordinates , we need to invert the coordinate map ( [ eulerian ] ) so that lagrangian coordinates are expressed in terms of the eulerian coordinates and thereby derive the following :  =[\,1+\ueber{\delta}{o}\,]\;j^e , \label{density2}\ ] ]  where is the initial density contrast and the determinant is    eq .( [ density2 ] ) gives the eulerian values of the field at the eulerian positions corresponding to a given trajectory labelled by . in principle, we can interpret this as a field satisfying the eulerian equations of motion .nonetheless , it is important to bear in mind that the functional dependence of on is only well - defined while the map between both coordinate spaces remains one - to - one .the peculiar - velocity is given by    and , similarly , the peculiar - acceleration ,    the jacobian of the transformation is given explicitly in appendix b. this general expression involves a number of cross - terms of and components .however , we can ignore these by assuming the peculiar - velocity field to be parallel to the peculiar - acceleration , such that    this assumption can be justified on physical grounds : after some time , the peculiar - velocity field tends to be parallel to the peculiar - acceleration , for the second - order irrotational solution ( [ eulerian ] ) , due to the existence of decaying and growing solutions in the weakly non - linear regime .we will address this issue in detail in sect . 5 , andshow that a weaker condition on the gradients of the fields leads to the same result .as in the first - order model ( where ( [ parallelism ] ) holds exactly ) the growing part supports parallelism , as it is shown in buchert & ehlers ( 1993 ) for a wide class of irrotational second - order solutions .indeed , the difference between and for ( [ eulerian ] ) is given by decaying modes only , as we shall see in the next section .therefore , we are entitled to express by some function proportional to , so that the inversion of the map ( [ eulerian ] ) satisfies the implicit equation :    where is given explicitly in appendix a for two different models .  in the first - order model , by virtue of ( [ parallelism ] ) , the same relation ( [ lagrangian2 ] ) holds with the appropriate ( appendix a.1 ) . with condition ( [ parallelism ] )implicit in ( [ lagrangian2 ] ) , the determinant reduces to :    here , ( divergence ) , ( dispersion of diagonal components ) and ( determinant ) denote the three principal scalar invariants of the peculiar - velocity gradient with respect to comoving eulerian coordinates as given in appendix b. therefore , substituting ( [ jacobian ] ) into ( [ density2 ] ) , we derive the relation for the density and velocity fields :   where .this result is unique and valid up to orbit - crossing time , i.e. , while the coordinate inversion is well - defined . in principle , it enables us to calculate the cosmic density field from observations of the peculiar - velocity field in the weakly non - linear regime at a fixed time , provided we set .it is interesting to emphasize that , due to the parallelism condition ( [ parallelism ] ) , ( [ result ] ) applies at both first- and second - order ( the appropriate for each case is given in appendix a ) .the simplicity of ( [ result ] ) is remarkable .it shows that the first- and second - order models contain not only quadratic terms in ( as pointed out in the literature , e.g. gramann 1993a , b ) but also cubic terms , arranged in the form of the principal invariants of the deformation tensor : , and .a more general result , albeit a complicated one , can also be derived for the case where we drop the condition ( [ parallelism ] ) .the corresponding solution , taking into account all cross - terms , is obtained using the full solution for given in appendix b. nonetheless , considering is not directly measurable from observations , the application of the general solution without the assumption of parallelism would be impractical . in these circumstances ( [ result ] )provides a useful approximation .in this section , we will examine two particular solutions of the second - order orbits ( [ eulerian ] ) that are of interest in the reconstruction procedure described in sect .these are characterized by the nature of the initial conditions at . at this time , only one field , s , is specified .these solutions are derived in buchert & ehlers ( 1993 ) .model i : _  in this case , the velocity and acceleration fields are set to be identically parallel at the initial time . this relation is satisfied in zeldovich s ansatz , and it is a natural condition to set in a more general scheme , given the remarkable accuracy of the `` zeldovich approximation '' .the initial density perturbation is thus expected to be proportional to the divergence of the peculiar - velocity .the solution ( [ eulerian ] ) then is specified by the functions  ,\ ] ]  .\ ] ]  therefore , the peculiar - velocity reads   and the peculiar - acceleration ,   thus the departure from the parallelism condition ( [ parallelism ] ) is    hence , this difference is a negligible effect due to the presence of the decaying mode . at ,the rhs of ( [ diff - i ] ) is identically zero and , at later times , the decaying mode vanishes sufficiently fast and the overall coefficient therefore tends to a constant .this constant second - order term could be absorbed in a coordinate transformation . in this case , in the reconstruction formula ( [ result ] ) is given by ( see appendix a.2 for details )    this is a monotonic function of , and to the leading order it reads    which is consistent with the results presented in moutarde et al .( 1991 ) .model ii : _  although model i provides an appropriate initial value setting from the physical point of view , the initial density contrast in this model is strictly non - vanishing .however , for reconstruction models we have to neglect this , since we do not want to specify initial information . precisely speaking , this implies an error which can be removed by using model ii : given a vanishing initial peculiar - acceleration field , the initial density contrast is zero and its growth is induced by the initial velocity perturbations only . in this case , and in the class of solutions for the orbits ( [ eulerian ] ) are given by  ,\ ] ]   .\ ] ]  therefore , from ( [ u ] ) the peculiar - velocity is     and the peculiar - acceleration ,     like in the previous example we can calculate the departure from parallelism which is given in terms of the decaying modes   such that , it is once more manifest that at late times parallelism is supported by the growing modes . at first sightit might seem that the presence of the first - order potential in ( [ diff - ii ] ) is a shortcoming for model ii in comparison to model i , where departure from parallelism is a second - order effect , given by ( [ diff - i ] ) .however , although the initial fields are non - parallel as indicated by ( [ diff - ii ] ) , the orders of magnitude involved in the rhs are very small from the outset and these decaying modes soon decrease further to negligible values .parallelism of and _  the assumption of parallelism ( [ parallelism ] ) is justified by the presence of decaying modes in the difference in both models i and ii . on the other hand, it is apparent that the exact expression for requires knowledge of the derivatives of the peculiar - acceleration field . using the parallelism condition to simplifythe solution implies neglecting the contribution of the eulerian derivatives of the vector field . in this sectionwe produce an analytical argument to justify this approximation in the weakly nonlinear regime .  a measure of parallelism of and is given by the quantity (an alternative measure has been proposed recently by bagla & padmanabhan ( 1996 ) ) .we expect this quantity to be small in the regime where and are close to being parallel .as is given explicitly in appendix a , we have      and therefore ,    for the particular case of model i , for instance , this leads to    this quantity vanishes for all models where and are parallel .however this is not so in general , and the integrals of ( [ s1])([s2 ] ) can yield to non - parallel ( and non - negligible ) gradients and . furthermore, the presence of the coefficient in ( [ wedge ] ) indicates that an initial departure from parallelism of the fields and will grow in time . at this pointit is important to stress that our simplification to derive ( [ result ] ) does not strictly speaking rely on the parallelism of the fields and but on that of the spatial gradients and .the differences of these fields are intimately connected , but the time dependence involved is different as is clear by direct differentiation .parallelism of the gradients is a weaker condition to impose than that of the fields themselves .it is easy to show that \ ] ] ^{-1},\ ] ]  and \ ] ] ^{-1},\ ] ]  therefore , we have that for model i :    and for model ii :   where  ^{-1}.\ ] ]  in the case of model i , parallelism is exact in the initial conditions , therefore there is no dependence of the derivatives on the potential . in all other models ,the difference comes as a linear combination of the double derivatives , , up to the coefficient that plays a role when we approach orbit - crossing . for realistic fields , i.e. , those that evolve from negligible amplitudes at , the gradients of the perturbative potentials , are sufficiently smooth and slowly varying , and therefore they are well under the order of the approximation in the weakly nonlinear regime .we stress , however , as indicated above that in the strongly nonlinear regime ( or at the epoch of shell - crossing , respectively ) , this argument breaks down in the same manner as perturbative solutions at any order do .in our reconstruction procedure we have proposed a simplification of the exact solution ( [ density2 ] ) under the assumption of parallelism of the fields and .thus we are able to obtain an expression for the density in terms of the derivatives of the velocity only .we have seen that this assumption is sufficiently accurate , the difference between both fields is a negligible effect .however , it is not necessary to adopt parallelism , and in this section we will outline a possible scheme to construct iteratively a `` self - consistent '' solution for from the velocity field , so that the reconstruction method with the jacobian ( [ full - jacobian ] ) can be employed .  at this point, we would like to call the attention of the reader to the problem of self - consistency of approximations as pointed out by doroshkevich et al .( 1973 ) ( compare the discussion by buchert ( 1989 ) ) .we enunciate the problem by stating that perturbative solutions of the equations of motion are mutually consistent only within the truncation order .this can be illustrated with the `` zeldovich approximation '' .the density contrast as given by the poisson equation is    and the density contrast as given by mass conservation    introducing the dimensionless error   difference for the `` zeldovich approximation '' gives    which is within the order of the approximation . in the second - order modelwe expect an analogous situation .as the orbits we have used are obtained by solving the lagrange - newton system perturbatively , eulerian quantities constructed from these will automatically satisfy all equations to second - order .however , the density contrast constructed from the velocities through ( [ result ] ) is associated to a newtonian potential causing an acceleration that will in general differ from the acceleration given by the second - order orbits .this difference vanishes only for exact solutions .let us denote by a ( 1 ) superscript the quantities derived from the particle orbits .hence ,   and therefore ,   which in general differs from . from it is possible in principle to solve for , from which is similarly constructed , and we iterate henceforth via the prescription     that we use to integrate .a consistent solution is then found in the limit    note that after each iteration , the condition ( arising from the lagrange equations in the exact case ) is preserved at all times . finding a self - consistent solution to the dynamics by successive iterations is tantamount to minimization of the difference of the particle orbits    that we estimate at each iteration through the differential equation    this is however a non - trivial numerical problem that is beyond the scope of this article and we plan to pursue this possibility in a future work .we have calculated a unique and well - defined relation ( [ result ] ) for cosmic density and velocity fields that is a self - consistent solution in the second - order model .given the second - order ansatz for the particle orbits ( [ eulerian ] ) , the lagrange - newton system determines the functions of time and for generic initial conditions and it constrains the perturbation potentials , in terms of the peculiar - velocity potential at .consequently , the orbits are determined at all times until shell - crossing , and the density is obtained from the continuity equation by evaluating the jacobian of the transformation .  for simplicity , we have discussed the formulae on the assumption that the velocity and acceleration fields are parallel . in sect .3 , it is shown that this assumption is a sufficient condition but by no means necessary , and in general it is sufficient to adopt the condition of parallelism of the gradients of the fields , which is a weaker requirement .the self - consistency of the solution for the density field follows from the fact that the orbits employed are second - order solutions in perturbation theory and not any given ansatz .our result ( [ result ] ) generalizes previous results in the literature in the following respect .nusser ( 1991 ) obtained two relations for the density constrast in the za ,    and    where a scaled peculiar velocity is used ,    and is derived from the continuity equation and from euler s equation .the difference between ( [ nusseri ] ) and ( [ nusserii ] ) is second order and is therefore within the order of the approximation :    thus , in the za , only the first term on the rhs of ( [ nusseri ] ) is significant and consistent with conservation of momentum . in the second - order result ( [ result ] ) , we are able to retain two more terms and this relation has a similar form as ( [ nusseri ] ) , where is replaced by .this form of the over - density field satisfies automatically euler s equation to second - order , since the orbits are obtained solving the lagrange - newton system to this order , and therefore the and terms are a legitimate part of the solution and not higher - order ` error ' terms as in ( [ nusseri ] ) .one does not obtain self - consistent solutions in models where an arbitrary perturbative parameter , such as , is singled out , which results in a hybrid eulerian - lagrangian scheme .the chief problem inherent to such approaches is that the time dependence is set ab initio on the orbits and it is not determined by the equations of motion as is done in the solutions we have used .this type of ansatz can be employed to good effect in a least - square fit for the dynamics of the particles as a two - boundary condition problem ( see for example susperregi & binney 1994 ) , but in perturbation theory they are inconsistent and of little predictive power due to their lack of convergence as discussed in the introduction . in such approaches , both the continuity equation and the equation of motion must be truncated to the given perturbation order and , consequently , the corresponding solutions and in general differ . therefore , from the analysis of the previous section , we can also conclude that and do not only differ by a higher order in the approximation , but failure to solve for the orbits perturbatively is conducive to discrepancies in the solutions _ at the same order_. given a parametrization of the orbits such as ( [ moutarde ] ) , the second - order solution for the continuity density is given by ( gramann 1993b )    as shown by gramann ( 1993b ) the second order term for differs by a factor of from that of ( [ gramann ] ) .this inconsistency at second order arises as a consequence of using second - order orbits that do not correspond to the actual second - order perturbation theory solutions .in contrast to the solution ( [ result ] ) , ( [ gramann ] ) only accounts for quadratic couplings in . considering that at first order we already have the relation ( [ result ] ) , it is apparent that ( [ gramann ] ) is not an accurate generalization of the lagrangian linear theory .in the scheme discussed here , initial conditions are set at , and hence we can try to establish a connection with the reconstruction model ( [ gramann ] ) by specifying these at . however , the normalization chosen for impedes to set identically equal to zero .a choice of an arbitrarily small requires keeping only the growing solutions in the expression for , entirely neglecting the presence of decaying modes .we have to further restrict ourselves to the leading term ( which is a particular solution of the second - order differential equation ) and neglect the homogeneous second - order solution ( which is growing ) . the leading order in , at both first and second order ( for model i ) ,    and thus , using definition ( [ velocity ] ) in ( [ result ] ) , we get     with the normalization chosen for , ( [ growing - modes ] ) is in reasonable agreement with ( [ gramann ] ) at early times . at late timeshowever , the presence of the term in ( [ growing - modes ] ) diminishes the departures from linearity brought about by the quadratic term whereas this effect is not accounted for in ( [ gramann ] ) .it must be emphasized that ( [ result ] ) is strictly only valid so long as the inversion is well - defined .indeed , the expression given for provides the eulerian values of the density contrast and is given as a function of eulerian positions up to orbit - crossing time .at later times the inversion map becomes singular and this result does not hold .  by integrating the poisson equation in eulerian coordinates using is concluded ( e.g. gramann 1993a , eq .( 2.20 ) ) that the presence of the quadratic term in the density - velocity relation induces a departure from parallelism between peculiar - velocity and -acceleration vectors .the derivation of this relation under the assumption of parallelism proves this conclusion is not correct .certainly , one would be able to obtain agreement with solutions of poisson s equation if a self - consistent density contrast were known .it is manifest however that this is far from being the case in these models .differences in the solutions within the order of the approximation , as it is the case with and in gramann s analysis , imply that any conclusion drawn from poisson s equation applied to the reconstructed density is not justified as long as approximations are involved .we finally note that an expansion of the jacobian to a given order ( _ after _ the solutions are obtained ) mirrors formally the mathematical fact that , in principle , we can not believe terms of higher order than the perturbative order of the trajectories .however , the powerful properties of a lagrangian approach mainly rely on the existence of an exact integral for the density for _ any _ given trajectory field , which allows using these approximations until shell - crossing . at this epochthe contributions from second and third invariants is of the same order as that from the first invariant , a fact which underlies the success of the za .whether this extrapolation until shell - crossing is justified at all is another issue .the successes of lagrangian perturbation methods as summarized by bouchet et al .( 1995 ) and the success of following trajectories until shell - crossing and even beyond ( melott et al .1995 ) strongly argue in favour of this extrapolation .in this paper we have presented a relation between cosmic density and velocity fields following a purely lagrangian derivation ( eq.([result ] ) ) that is self - consistent in the second - order model .the advantage of working in the lagrangian picture is that the over - density field obtained from the continuity equation is self - consistent and automatically satisfies conservation of momentum . for simplicity, one can adopt the condition ( [ parallelism ] ) of parallelism of the fields and , satisfied at first order and by a large class of irrotational solutions in second - order lagrangian perturbation theory , and the result ( [ result ] ) holds accurately at both orders . however , as pointed out in sect .3 , we note that ( [ parallelism ] ) is not a sine qua non condition but a weaker condition of parallelism of the gradients of the fields would yield the same result .it is shown that this condition is well satisfied for generic initial conditions . in each case , the function is given explicitly in appendix b.1 and appendix b.2 respectively .its range of validity is limited by the epoch when the inversion map becomes singular , i.e. up to orbit - crossing time , while its principal range of validity should strictly be estimated by , where $ ] is the displacement field with respect to the hubble - flow .this condition is probably very conservative in view of the success of these approximations if followed up to shell - crossing .  by specifying the initial data at and keeping only the leading terms in the solutions , we are able to recover the leading terms of previous reconstruction models as shown in sect .furthermore , the time dependence of the orbits in our model is established by solving the lagrange - newton system at each order .this leads to a unique and consistent result for the eulerian density constrast in terms of the peculiar - velocities .we finally wish to show some skepticism about the usefulness of the reconstruction formulae given .we want them to be applicable to present day observational data. however , we know confidently that shell - crossing has occurred , i.e. the interesting regime is no longer covered .instead one would have to investigate a relationship between _ smoothed _ cosmic over - density and peculiar - velocity fields , where the smoothing window scale has to ensure the absence of vorticity and multi - stream systems in the average flow .given the present work we should ask whether there is a formally proper way to incorporate smoothing into a consistent lagrangian reconstruction formalism . with regard to this point , we think that solutions of the lagrange - newton system may possibly provide reasonable tools for reconstructing average fields , since averages in eulerian space of any vector function are straightforward to calculate : .however , we still encounter the fundamental problem of averaging over multi - stream flows , which may not be properly handled in the framework of the lagrange - newton system .rather , such a description has to be based on an approximation of the vlasov - poisson system .this major improvement of the methods lies well beyond the scope of present reconstruction techniques and requires the construction of vlasov - poisson type approximations .we thank james binney , avishai dekel , jrgen ehlers , adi nusser and david weinberg for valuable comments and suggestions .tb is supported by the `` sonderforschungsbereich 375 - 95 fr astro - teilchenphysik '' der deutschen forschungsgemeinschaft .mps is grateful to the max - planck - institut fr astrophysik at garching for its hospitality .the first - order particle orbits are    where the perturbation potential is related to the initial gravitational potential through the relation    the inversion of ( [ firstorder ] ) is then :    and therefore is given by :    thus , we obtain :      from this we derive the well - known result for the zeldovich approximation used in the sachs - wolfe effect that    since     eqs .( [ u])([w ] ) in terms of the perturbation potentials read  = \left[\begin{array}{cc }    \alpha(t ) & \beta(t ) \\\eta(t ) & \zeta(t )    \end{array}\right ] \left[\begin{array}{c }                \nabla_x s^{(1)}(\b x)\\                \nabla_xs^{(2)}(\b x )              \end{array}\right],\ ] ]  where          we write the perturbation potentials in terms of and by inverting ( [ uwmatrix ] ) :  = \left[\begin{array}{cc }    \alpha'(t ) & \beta'(t )\\    \eta'(t ) & \zeta'(t )    \end{array}\right ] \left[\begin{array}{c }                \b u(t,\b h)\\                \b w(t,\b h )              \end{array}\right],\ ] ]  where          using ( [ smatrix ] ) , we invert ( [ eulerian ] ) in the form   , \label{inverted}\ ] ]  where      therefore , given the condition of parallelism([parallelism ] ) , we have that ( [ inverted ] ) reduces to ( [ lagrangian2 ] ) with  \ ] ]     following the same procedure as in appendix a.2 , we write  = \left[\begin{array}{cc }    \alpha(t ) & \beta(t ) \\\eta(t ) & \zeta(t )    \end{array}\right ] \left[\begin{array}{c }                \nabla_x s^{(1)}(\b x)\\                \nabla_x s^{(2)}(\b x )              \end{array}\right],\ ] ]  where        therefore , inversion of ( [ uwmatrix2 ] ) leads to :  =\gamma ( t)\ , \left[\begin{array}{cc }    \alpha'(t ) & \beta'(t ) \\\eta'(t ) & \zeta'(t )    \end{array}\right ] \left[\begin{array}{c }                \b u(t,\b h)\\                \b w(t,\b h )              \end{array}\right],\ ] ]  where          jacobian is given by :

Abstract: a first- and second - order relation between cosmic density and peculiar - velocity fields is presented . the calculation is purely lagrangian and it is derived using the second - order solutions of the lagrange - newton system obtained by buchert & ehlers . the procedure is applied to two particular solutions given generic initial conditions . in this approach , the continuity equation yields a relation between the over - density and peculiar - velocity fields that automatically satisfies euler s equation because the orbits are derived from the lagrange - newton system . this scheme generalizes some results obtained by nusser et al . ( 1991 ) in the context of the zeldovich approximation . as opposed to several other reconstruction schemes , in this approach it is not necessary to truncate the expansion of the jacobian given by the continuity equation in order to calculate a first- or second - order expression for the density field . in these previous schemes , the density contrast given by ( a ) the continuity equation and ( b ) euler s equation are mutually incompatible . this inconsistency arises as a consequence of an improper handling of lagrangian and eulerian coordinates in the analysis . here , we take into account the fact that an exact calculation of the density is feasible in the lagrangian picture and therefore an accurate and consistent description is obtained .


reversible markov chains are central to a number of fields .they underlie problems in applied probability like card - shuffling and queueing networks and pervade computational statistics through the many variants of markov chain monte carlo ; in physics , they are natural stochastic models for time - reversible dynamics . however , the notion of reversibility in stochastic proscesses with memory is not as widely discussed , and statistical problems like testing the order of a reversible process remain a challenge .we define a conjugate prior for higher - order , reversible markov chains , which extends a prior for reversible markov chains by diaconis and rolles  .we begin by defining reversibility in a more general setting and motivating the significance of higher - order processes . in section [ sec2 ] ,we present two graphical representations for an order- , reversible markov chain , which are used in section [ sec3 ] to derive the conjugate prior via a random walk with reinforcement .we dedicate section [ sec4 ] to variable - order markov chains , a family of models that avoids the curse of dimensionality associated with higher - order markov chains , proving essential in certain applications .finally in section [ sec5 ] , we discuss properties of the prior pertaining to bayesian analysis . in examples ,we test the extent of memory of a lumped markov chain and discretized molecular dynamics trajectories , and compare the posterior inferences of different models .[ defreversibility ] a stochastic process , with distribution is called _ reversible _ , if for any ,   it is not difficult to show that reversibility implies stationarity ; if stationarity is given , the above condition need only be checked for .now suppose is an order- , irreducible markov chain taking values in a finite set .we will also apply the term _ reversible _ to this process when the stationary chain satisfies the reversibility condition .[ reversibilitycondition ] let be the stationary law of the order- markov chain .if , then the markov chain is reversible .it is not difficult to check that the hypothesis together with stationarity imply for any . for any : where we have used the markov property , stationarity and the hypothesis .  as a first remark ,note that , can be represented as a first - order markov chain , taking values in the space of sequences .however , the reversibility of does not imply the reversibility of its first - order representation ; therefore , the analysis of higher - order reversible markov chains requires novel techniques . in the following sections , we often use the first - order representation , referring to it nonetheless as an order- markov chain and using the notion of reversibility associated with the markov chain .secondly , we recall that kolmogorov s criterion is another necessary and sufficient condition for the reversibility of a markov chain , which only depends on the conditional transition probabilities .its equivalence to definition [ defreversibility ] in the higher - order case is proven in the .kolmogorov s criterion requires that the probability of traversing any cycle in either direction is the same .accordingly , a reversible markov chain can be interpreted as a process with no net circulation in space .  .in a circuit process started at in , we transition on some circuit that contains with probability proportional to its weight . ]reversibility is preserved under certain transformations .for example , let , be a stationary , reversible markov chain and consider a finitely valued function , .it is easy to check that this process is stationary and reversible , even though it may not be a markov chain of any finite order .functions or projections of reversible markov chains appear under different guises in physics and other fields , and in many cases the effects of memory subside with time , motivating the use of finite order models .the problems of determining the order and estimating the parameters of markov models have been studied extensively ; here , we address these problems with the constraint of reversibility .for any sequence , let be its inverse , the subsequence obtained by deleting its last element and the one obtained by deleting its first element .we call with an _ admissible path _ if for all . the concatenation of these sequences without repeated overlaps is denoted .  the first representation we will consider is the circuit process of macqueen .let a _ circuit _be a periodic function on , and consider a class of positively weighted circuits ( for an example , see figure [ circuit process picture ] ) .  a _ circuit process _ of order is a markov chain of the same order , where the transition probability from to any with is given by where is the weight of circuit , and the function counts the number of times that the circuit traverses a sequence in one period . in other words , in each step we move along some circuit in containing the current state with probability proportional to its weight .the process only visits states that appear in the circuits , for which transition probabilities are well defined .  an irreducible order- markov chain with stationary law is parametrized by for all .one can check that in a circuit process , this is just .macqueen showed that any order- markov chain can be represented as a circuit process on a finite set , which is not unique .this is true in particular when the chain is reversible .  .in a reversible random walk , the two highlighted edges have the same weight . ]we introduce a second graphical representation that is canonical , unlike the circuit process .consider a _de bruijn graph _ on the vertices , which has a directed edge from to if and only if .that is , every path on the graph is an admissible path . for an example , see figure [ triangle ] .assign a weight to each edge , and let be the summed weights of edges departing from .furthermore , require that   the _ reversible random walk of order _ is a random walk on such a graph , with transition probabilities   [ representation ] an irreducible , reversible random walk of order represents a reversible markov chain of the same order .every irreducible , reversible order- markov chain is equivalent to a unique reversible random walk of order .let be the stationary distribution of the random walk . to prove the first statement, we will first verify that for all .let be the transition probability from to in the random walk , and recall that iff , then then , the stationary law in the random walk of a path is just which implies that .therefore , the -valued , order- markov chain represented by the random walk satisfies the reversibility condition in proposition [ reversibilitycondition ] .proving the second statement is now straightforward .let , be the first - order representation of an irreducible , order- markov chain , with transition probabilities . by the perron frobenius theorem , has a unique stationary distribution .assign edge weights to the de bruijn graph on , setting .since the order- markov chain is reversible , it follows directly from proposition [ reversibilitycondition ] that the edge weights satisfy conditions ( [ eq1])([eq3 ] ) .an edge - reinforced random walk ( errw ) is a random walk on an finite , undirected graph , where every edge - weight is increased by 1 each time it is crossed . since diaconis and coppersmith defined this process , we have learned that it is partially exchangeable and , by de finetti s theorem for markov chains , a mixture of markov chains .the mixing measure , which lives on the space of reversible markov chains , was more recently characterized in the literature .diaconis and rolles showed that this distribution is a conjugate prior for the reversible markov chain , much as the beta distribution , arising from a poly urn scheme , is a conjugate prior for sequences of i.i.d .binary random variables .  here, we construct a conjugate prior for higher - order reversible markov chains via a reinforced random walk in , making use of de finetti s theorem for markov chains .this process is markedly different from an errw in due to the structure of a reversible markov chain with memory , although it is designed to be partially exchangeable .let be any sequence on and a sequence shorter than .define the function , which counts the number of times that appears in , and , which counts the number of times that appears in followed by at least one state .fix , a stationary measure for an irreducible , reversible , order- markov chain . also fix .let be a palindromic sequence that starts with and ends with .choose a positive constant , such that for all , .now , given a sequence , starting with , and any sequence , define the functions when represents the path of a stochastic process in up to time ( formally , ) , we will use the notation and .[ reinforcement ] the _ reinforced random walk of order _ is a stochastic process , on with distribution .the initial state is with probability 1 . for any admissible path , the conditional transition probability whenever is admissible and zero otherwise .the law also depends on and .these parameters are constant in the following discussion , so they are omitted from the notation for conciseness . when , this process is equivalent to an errw . in this case , the palindrome is unnecessary because the terms involving in and can be modeled with a different . for ,this is not the case , and is essential for partial exchangeability ( see proposition [ pe ] ) .[ circuit_interpretation ] this process admits an interpretation as a reinforcement scheme of the circuit process .consider a circuit process of order with stationary probability for all .in addition , consider three weighted sequences : the palindrome , a sequence that represents the path of the reinforced process from the initial state up to the current state , and the reversed path .these are depicted in figure [ reinforcement_picture ] along with their weights , and , respectively .as in the circuit process , we move along any circuit or sequence that contains the current state with probability proportional to its weight . the reinforcement is accomplished by elongating the paths and .[ errw_interpretation ] the process is also a reinforcement scheme of a modified reversible random walk of order .consider a weighted de bruijn graph , where for every admissible , .then , for every in the palindrome , subtract from .the reinforcement scheme will consist of a random walk on the resulting graph , where after every transition we increase both and by .accordingly , if is a palindrome , the weight is increased by .  [ pe ]the reinforced random walk of order is partially exchangeable in the sense of diaconis and freedman .we must show that the probability of any admissible path is a function of the initial state and the transition counts between every pair of states . for any pair in with ,let be the total number of transitions , and .we will show the stronger statement that and are sufficient statistics for the reinforced random walk .   reinforced random walk . ]let us first establish some properties that are conserved in the process .for every , the initial weights and are equal .this is direct from the definition in equation ( [ w ] ) because : defines a reversible markov chain of order ; the functions and are zero for both and ; and is a palindrome , so if it contains , it also contains , and .this property is maintained after every transition , because the weights may both be increased by if is or , or both remain constant otherwise .  for every in , the initial weights .this is direct from equation ( [ w ] ) because : is reversible , both and are zero for and , and the sequence is a palindrome , so for every transition starting at there will be another starting from . the last fact is not necessarily true for , because unless itself is a palindrome , will contain a transition starting from it , but no transition starting from .so , in the beginning , .when a transition occurs from to , the weights and become equal , while , provided is not a palindrome . hence, this singularity is preserved by the last state visited by the process .the probability is a ratio of two products . in the numerator, we find a factor of the form for every admissible transition , while in the denominator , we find a corresponding weight .it is easy to check that the numerator is only a function of .every transition or adds a new factor of , which is always greater than the previous one by .if is a palindrome , then every new factor of is increased by .so , the numerator can be computed from the initial weights and .we have left to show that the denominator is only dependent on and .note that the transition counts from or are a function of and , because every event is a transition from , while every event is followed by a transition from , unless this is the final state , which is determined by .after every transition from or , we add a factor of or to the denominator . at any time , these weights differ by ( if is not a palindrome ) , but the factor added is always the smaller of the two . between two transitions , each of these weights is reinforced by , so consecutive factors differ by that amount . if is a palindrome , there is no distinction between and , and consecutive factors differ by  .[ positive - recurrence ] suppose that in the reinforced random walk , we visit and in infinitely often a.s ., and let be the time we visit either state .the process is a mixture of markov chains .furthermore , if is the ratio of the number of visits to and by , converges a.s . to a finite limit .we claim that if is partially exchangeable , so is .it is sufficient to show that the probability of a sequence is invariant upon block transpositions , which generate the group of permutations that preserve transition counts ( , proposition 27 ) .the probability of a path in is the sum of the probabilities of all paths in that map to it .denote this set of paths .after a transposition of -blocks or -blocks , the probability of the path in is equal to the sum of the probabilities of a different set of paths in .however , it is easy to see that this transpostion of -blocks or -blocks defines a bijection from to , and the probability of each path and its transposition is the same , because is partially exchangeable. therefore , is partially exchangeable .furthermore , we assume that and are recurrent , so by de finetti s theorem for markov chains is a mixture of markov chains with a unique measure on the space of 2 by 2 transition matrices .note that both states are recurrent with probability 1 , so the subset of transition matrices where one of the states is transient has -measure zero .this implies that -a.s .the transition matrix is irreducible , and since the state space is finite , both states are positive - recurrent .therefore , converges a.s . to a finite limit.=-1the reinforced random walk of order traverses every edge with infinitely often , almost surely .[ recurrence ]  as is finite , we must visit at least one state in infinitely often , so without loss of generality , let this state be .let be the time we visit , and be .for with admissible and , let be the event that .also , let . by lvys extension of the borel cantelli lemma ( lemma [ levy ] ) , therefore , to show that the transition is observed infinitely often with probability 1 , it is sufficient to show that a.s .the conditional probability is just .let be the event that we observe fewer than times between and .on , we can lower - bound using the minimum possible value of , which is its initial value , and the maximum possible value of , which is .thus , now , consider the event .on this set , for any , we will be in for all but finitely many , which implies , by the previous inequality .but , by lemma [ positive - recurrence ] we have , so noting we conclude that -a.s . , and happens infinitely often . since defines an irreducible markov chain , the proposition follows by induction .propositions [ recurrence ] and [ pe ] are sufficient to show by de finetti s theorem for markov chains that the reinforced random walk of order is a mixture of markov chains on , or where is the distribution of a markov chain started at and parametrized by the matrix , is the space of stochastic matrices and is a unique measure on the borel subsets of this space .let be the set of matrices that represent irreducible , reversible markov chains of order .the reinforced random walk of order is a mixture of reversible markov chains of the same order , or .this is a special case of proposition [ concentration ] .the number of parameters of a markov chain grows as with the order , , which renders higher - order models impractical in many statistical applications . in this section, we investigate a family of models with finite memory length which do not suffer from this curse of dimensionality .[ mixed - order definition ] a _ variable - order markov chain _ is a markov chain of order with the constraint that for every _ history _ in the set , if two states both end in , the transition probabilities and are equal for every .in essence , this is a discrete process which upon reaching a sequence loses memory of what preceded it .when is empty , we recover a general markov chain of order .variable - order markov chains have proven useful in applications where there is long memory only in certain directions .the literature on the subject can be traced to rissanen and weinberger , who developed tree - based algorithms for estimating the set of histories efficiently in the context of compression .bhlmann and wyner proved several consistency results on these algorithms , and the former later addressed the problem of model selection .for an evaluation of different algorithms in applications , see .it is worth noting that macqueen mentioned variable - order markov chains in an unpublished abstract .however , there is a marked difference between his definition and bhlmann and wyner s , which relates to the closure properties of .macqueen requires that if is in , then so are all the sequences that begin with  .intuitively , this means that the process can not recover memory once it is lost .bhlmann and wyner do not impose this constraint .however , this is guaranteed when the process is reversible .[ regaining_memory ] let , be an irreducible , reversible , variable - order markov chain with histories .if , then is also a history ; additionally , any sequence that has as a prefix is also in .let be the stationary law of the chain . if , then for any pair , where and the length of sum to , is independent of , or this implies using the fact that is invariant upon time reversal and rearranging factors , we obtain the left - hand side is equal to , which by the previous identity is independent of . asthis is true for any , must be a history in . to prove the second part of the statement , suppose is a prefix of . since is in , and ends in , then by definition . using the first result ,we conclude that .we will define a reinforcement scheme , which like the one in the previous section is recurrent , partially exchangeable and , by de finetti s theorem , a mixture of markov chains .but , in this case , the mixing measure is restricted to the variable - order , reversible markov chains with a fixed set of histories .as before , we begin with a stationary , reversible function , an initial state , and a palindromic sequence that starts with . let the function map any sequence to its shortest ending in .[ mixed - order reinforcement ] the _ variable - order _ , _ reinforced random walk _ is a stochastic process , on with measure .the initial state is with probability 1 . for any admissible path , the conditional transition probability whenever is admissible and zero otherwise .this process is a reinforced circuit process , just like the one defined in remark [ circuit_interpretation ] , with the difference that in computing the transition probabilities , instead of taking the current state to be the sequence , we let it be the shortest ending of in , or .the variable - order , reinforced random walk is partially exchangeable in the sense of diaconis and freedman .[ mixed - pe ]  this proof is deferred to the .one can show that this process is recurrent following the same argument of proposition [ recurrence ] . in the proof of proposition [ recurrence ] , we use a shortest history in place of , and lemma [ positive - recurrence ] still holds for and .recurrence and partial exchangeability imply for a unique measure characterized by the function , and the initial state , in addition to the parameters , and , which we keep fixed . in the, we show that is restricted to the reversible , variable - order markov chains with histories .[ concentration ] let be the set of transition matrices representing an irreducible , reversible , variable - order markov chain where every is a history . then , .in section [ sec3 ] , we defined a family of measures in the space of order- , reversible markov chains , and in section [ sec4 ] we extended it to variable - order , reversible markov chains . in the following , we will show that these distributions are conjugate priors for a markov chain of order  .we discuss properties of the prior relevant to bayesian analysis , such as a natural sampling algorithm and closed - form expressions for some important moments .consider a variable - order , reinforced random walk , , with distribution and take any admissible path .we define , to be the process with law   in words , is the continuation of a variable - order reinforced random walk after traversing some fixed path .we can rewrite the law which makes it evident that is partially exchangeable , because for a fixed  , the numerator only depends on the transition counts in , while the denominator is constant .it is also not hard to see that the process visits every state infinitely often with probability 1 .therefore , by de finetti s theorem for markov chains , it is a mixture of markov chains with a mixing measure that will be denoted .suppose we model a process , as a reversible , variable - order markov chain with histories , and we assign a prior to the transition probabilities , .given an observed path , , the posterior probability of is . in consequence, the family of measures is closed under sampling .consider the event . by bayes rule ,the posterior probability of this event given the observation is the prior probability of divided by the prior probability of . by equation ( [ post ] ) , this posterior is equal to .let be the posterior distribution of  given the observation , then for any and any , by de finetti s theorem for markov chains , the mixing measure is unique ; therefore , we must have .  in the next proposition ,we show that the variable - order , reinforced random walk may be used to simulate from the conjugate prior ( or  a similar argument , a posterior of the form ) .let be independent samples of the reinforced random walk with initial parameters and .for any sequence , consider the random variable , the weight defined in equation ( [ w ] ) for a sample path with distribution , normalized by the path s length .define the empirical estimate , , to be the mean of this random variable evaluated at the paths .also , let be the stationary law of an order- markov chain with transition probabilities .we have seen that has a one - to - one correspondence with .  for any bounded , real - valued function , [ posterior_sampling ]   the empirical estimate is the average of i.i.d .observations , so by the strong law of large numbers , w.p.1 , ,\ ] ] where the right - hand side is the expectation in a reinforced random walk with parameters . in the proof of proposition [ concentration ] , we showed that converges -a.s . taking the limit as , by dominated convergence , .\end{aligned}\ ] ] conditional on a variable measurable on its tail -field with distribution ,the reinforced random walk is a markov chain with law .we know converges -a.s . to , so equation ( [ simulation ] ) follows .several moments of have closed - form expressions .in particular , the mean likelihood of any path beginning in is just the probability of the path in the reinforced random walk by equation ( [ mixed - order mixture ] ) . from the proof of proposition [ mixed - pe ], one can deduce a closed - form expression for the law of the variable - order reinforced random walk as a function of the transition counts in a path ( see supplement ) . from a realization of the transition counts as a path, one can also compute the law by modeling a random walk with reinforcement .the expectation of cycle probabilities with a prior on may also be computed exactly .  for any cyclic path , not necessarily including , the expectation of with prior on has a closed - form expression , provided is greater than for all .find the shortest cycle with positive weight .then , for any transition matrix in the support of , we have taking the expectation with a measure on , we obtain by bayes theorem , the product of the likelihood and the prior is equal to the marginal prior probability of the path times the posterior of : where are the weights parametrizing the posterior of given the path . to solve the integral on the right - hand side ,let us rewrite it using bayes theorem and equation ( [ mixed - order mixture ] ) , where are the weights reduced by the cycle .these weights are positive because of the assumption for all , which could certainly be relaxed in some cases . applying equations ( [ mixed - order mixture ] ) and ( [ ratio ] ) once more, the last expression becomes which completes our derivation .the ability to compute these expectations exactly makes it possible to use bayes factors for model comparison .given some data and two probabilistic models , where each model has a prior measure and parameters , a bayes factor quantifies the relative odds between them .it is formally defined as , the ratio between the marginal probabilities of the data under each model .each marginal probability is sometimes referred to as the _ evidence _ for the corresponding model .diaconis and rolles apply bayes factors to compare a number of models on different data sets .they consider reversible markov chains , general markov chains , and i.i.d .models , assigning conjugate priors which facilitate computing the marginal probabilities in equation ( [ bayes_factor ] ) .the conjugate priors introduced here facilitate similar comparisons , where the family of models under consideration is expanded to include reversible markov chains that differ in their length of memory .for some data , one can define two variable - order reversible markov models , with different histories , and . in each case, we assign a conjugate prior , and , respectively , to the transition probability matrix . to make the prior uninformative in some sensewe could set to be uniform for all and let be the shortest palindrome starting with , for example .the constant is set to 1 .the bayes factor is then     we have seen that the expectations on the right - hand side can be computed exactly when is a path starting at or any cyclic path . in the following example, we apply this test to finite data sets simulated from a lumped markov chain .[ example1 ] a random walk was simulated on the 9-state graph shown in figure [ example ] , from which we omitted self - edges on every state , all weighted by 1 .the observation was lumped into the 3 _ macrostates _ separated by the dashed lines .this is meant to illustrate a natural experiment , where the difference between the states within each macrostate is obscured by the measurement . from the resulting sequence , we take the initial macrostate and every 7th macrostate thereafter to form a path of length 1000 in .we test 4 reversible markov models , that differ in the length of memory :  1 .  a first - order , reversible markov chain .2 .  a second - order , reversible markov chain .3 .  a variable - order model with maximum order 2 , where states 1 and 3 are histories . intuitively , only state 2 has `` memory . ''4 .  a variable - order model with maximum order 2 , where states 2 and 3 are histories .intuitively , only state 1 has `` memory . ''  for each model , we assign a prior to the transition matrix , where is the initial state in , for all and is the shortest palindrome starting with .we compared the 4 models using 50 independent realizations of the lumped markov chain and found that model 3 had the highest evidence in 72% of the cases , while model 2 was selected in all the remaining cases . in figure[ boxplot ] , we report a boxplot of the logarithm of the bayes factors comparing models 1 , 2 , and 4 against model 3 .this represents compelling evidence for model 3 .the result is not entirely surprising given that this model gives memory to state 2 , which is slowly mixing , as indicated in figure [ example ] .the fact that the most complex model ( model 2 ) is not necessarily selected showcases the automatic penalty for model complexity in bayes factors .   and .the periodic map on the right shows a partition of conformational space into 5 states .the colored markers indicate the free energy of bins centered at each point , which reveals the metastable nature of this molecule s dynamics . ]we conclude this section with two applications of bayesian analysis of reversible markov chains to molecular dynamics ( md ) .an md simulation approximates the time - reversible dynamics of a molecule in solvent .the trajectories produced by a simulation are discretized in space and time .the terminally blocked alanine dipeptide , shown in figure [ alanine ] , is a common test system for markov models of md .the conformational space of the molecule , which is represented in the figure in a two - dimensional projection , is partitioned into 5 states .the states are believed to be metastable due to the basins that characterize the free - energy function , also plotted in the figure .this metastability allows one to approximate the dynamics of the molecule , projected onto the partition , as a reversible markov chain. the approximation will be good when the discrete time interval at which a trajectory is sampled is larger than the timescale for equilibration within every state , but smaller than the timescale of transitions .few statistical validation methods are available for markov models of md .bacallado , chodera and pande used a bayesian hypothesis test to compare different partitions of conformational space . here, we apply bayes factors to test a first - order markov model on a fixed partition , by comparing it to second - order and variable - order models on the same partition .the data are the transition counts in a single md trajectory of 1767 steps sampled at an interval of 6 picoseconds , as recorded in table [ counts ] .the prior parameters and are the same as in the previous example .the results of the model comparison are summarized in the following table ..0d3.0d2.0d2.0ccccd2.0d2.0cc@ & & & & & + & & & & & + & & & & & & & & & & & & & + & 0 & 261 & 187 & 13 & 2 & 0 & 3 & 0 & 5 & 13 & 2 & 0 & 0 + & 1 & 188 & 144 & 13 & 11 & 0 & & 1&5 & 4 & 2 & 1 & 0 + & 2 & 12 & 4 & 9 & 15 & 0 & & 2&4 & 3 & 16 & 5 & 0 + & 3 & 5 & 1 & 0 & 1 & 0 & & 3&2 & 5 & 3 & 3 & 0 + & 4 & 1 & 0 & 0 & 0 & 0 & & 4&0 & 0 & 0 & 0 & 0 + [ 6pt ] 1 & 0 & 180 & 143 & 22 & 5 & 0 & 4 & 0 & 1 & 0 & 0 & 0 & 0 + & 1&141 & 125 & 5 & 5 & 0 & & 1&0 & 0 & 0 & 0 & 0 + & 2&4 & 3 & 10 & 4 & 0 & & 2&0 & 0 & 0 & 0 & 0 + & 3&4 & 1 & 10 & 3 & 0 & & 3&0 & 0 & 0 & 0 & 0 + & 4&0 & 0 & 0 & 0 & 0 & & 4&0 & 0 & 0 & 0 & 0 + [ 6pt ] 2 & 0 & 16 & 13 & 3 & 0 & 0 & + & 1&16 & 4 & 1 & 1 & 0 & + & 2&12 & 12 & 37 & 11 & 0 & + & 3&9 & 5 & 15 & 6 & 0 & + & 4&0 & 0 & 0 & 0 & 0 & +  [ cols="<,^",options="header " , ]   this represents compelling evidence for a model that gives memory to states 14 , 15 , 30 and 31 .it is interesting to contrast inferences based on this model to those based on a first - order markov model . to do this , we computed 1000 approximate posterior samples of the transition matrix in each case .this was done by simulating a reinforced random walk , which is a mixture of variable - order markov chains with the posterior distribution of as a mixing measure ( see proposition [ posterior_sampling ] ) .the reinforced random walk was simulated steps to obtain each sample ., in .all sample means and standard deviations are shown . ]  in figure [ histograms_stat ] , we histogram stationary probabilities of the transition matrices sampled from the posterior .in particular , we show plots for the stationary probabilities of states 14 , 15 , 30 and 31 . in the variable - order model, we define .the inferences of each model in this case are very similar .the largest eigenvalues of the transition matrix are also of interest because they are related to different modes of relaxation .each eigenvalue is associated with a timescale , which is useful in exploratory analysis . here , is the length in time of one step of the markov chain , or 500 picoseconds . in figure[ histograms eig tim ] , we histogram posterior samples of the three largest nonunit eigenvalues and their associated timescales . in this case , the inferences of each model are quite different , with the variable - order model predicting larger eigenvalues and timescales .we define a reinforcement scheme for the higher - order , reversible markov chain that extends the errw on an undirected graph .several properties of the errw , like recurrence and partial exchangeability , were shown to generalize to this process .other properties may also generalize but were not pursued here .in particular , we can mention the uniqueness results of johnson and rolles , and the fact that mixtures of measures in are weak - star dense in the space of all priors .the reinforced random walk leads to a conjugate prior that facilitates estimation and hypothesis testing of reversible processes in which the effects of memory decay after some time .certain statistical problems remain a challenge , such as inferring the transition matrix with a _ fixed _ stationary distribution . in applications, it will become important to evaluate the objectivity of the prior and to determine the optimal value of its parameters in this sense .  from a practical point of view , we only discussed bayesian updating for data sets composed of a single markov chain starting with probability 1 from the initial state used in the prior .numerical algorithms are needed to perform inference with data sets composed of multiple chains .a starting point could be the method developed by bacallado , chodera and pande to apply the prior of diaconis and rolles to first - order , reversible markov chains .in the following , we use the notation defined in the first paragraph of section [ sec2 ] .  let , be an irreducible order- markov chain with transition probabilities . then is reversible if and only if for any cyclic admissible path ,   the `` only if '' statement is straightforward . by the definition of the stationary distribution and reversibility to prove the `` if '' statement ,choose an arbitrary state ; then , for any , since the chain is irreducible , there is an admissible path with positive probability .define where is a positive constant .note that this expression does not depend on the sequence chosen .take a different sequence .let be a palindrome , then because the chain is irreducible , we can find a sequence with positive probability , and it is easy to see from equation ( [ kolmogorov ] ) that the palindrome has positive probability .we can construct another palindrome in the same way . multiplying equation ( [ stat_construction ] ) by factors of 1 , the first four terms equal 1 because the numerator and denominator are the probabilities of the same cycle forward and backward , which are equal by equation ( [ kolmogorov ] ) .now , we check that satisfies the reversibility conditions specified in the .first , we show that .take a path with positive probability , and the previously found palindrome , then applying the same method , from this , and equation ( [ stat_construction ] ) we deduce that for any admissible , . since the state space is finite , we can choose such that sums to 1 .we have shown that the weights satisfy the conditions of a reversible random walk with memory , so by proposition [ representation ] the process with transition probabilities represents a reversible , order- markov chain .proof of proposition [ mixed - pe ] the probability is a product of transition probabilities , to which the transition contributes a factor of we know that can not be longer than by proposition [ regaining_memory ] ; let be the set of histories of that are shorter than .if this set is nonempty , let us multiply equation ( [ factor1 ] ) by factors of 1 , to obtain the following factor for the transition : where is the ending of that is longer than by 1 .the added factor equals 1 because , if , then must end on , which by definition is a history shorter than , a contradiction .consider all the possible factors in the numerator of .take any that is _ minimal _ , meaning that it does not end in another history .for any , we will see a factor after every transition through .the conjugate factor will appear every time we go through , because :  * if , it is minimal by the closure properties of , so will be the numerator of the first factor in equation ( [ factor2 ] ) . * otherwise , the minimal history in the transition ending in will be longer than , and there will be an added factor in equation ( [ factor2 ] ) with in the numerator .conversely , note that the factor is only added to the numerator of equation ( [ factor2 ] ) when we go through for some minimal , because we required that , so and does not end in another history .as in the proof of proposition [ pe ] , we argue that every new factor or is increased by with respect to the previous one ( or by if is a palindrome ) . therefore , the numerator of is only a function of the transition counts and the initial state .  finally , consider all the factors in the denominator of .take any minimal history .we will see a factor , for every transition through .the conjugate factor will appear every time we go through , because :  * if is also minimal , then will be in the denominator of the first factor in equation ( [ factor2 ] ) .* otherwise , we know that is not a history , so the transition ending in must have a history at least as long as , which is longer than the history .so , will appear in the denominator of a factor added in equation ( [ factor2 ] ) .conversely , we only add factors of to the denominator of equation ( [ factor2 ] ) when we go through for a minimal , because we required which implies minimal .  as before , every new factor or will be increased by with respect to the previous one ( or by if is a palindrome ) .therefore , the denominator is a function of the transition counts and the initial state , and the process is partially exchangeable .proof of proposition [ concentration ] let be the transition counts from to in the first steps of a stochastic process on . also , define , which counts the visits to .remember is the set of irreducible transition matrices for variable - order , reversible markov chains where all are histories .define the event , that the set admissible converges to a transition probability matrix in .  from the recurrence of the variable - order ,reinforced random walk and equation ( [ mixed - order mixture ] ) , it is evident that the set of irreducible markov chains has measure 1 under .in this set , the variables converge almost surely to the transition probabilities , so for any irreducible , .furthermore , by lemma [ q ] , happens almost surely in the variable - order , reinforced random walk .putting this into equation ( [ mixed - order mixture ] ) , we have which implies the proposition .[ levy ] consider a sequence of events , in some filtration .let be the total number of events occurring among the first , and let be the sum of the first conditional probabilities .then , for almost every :  * if converges as , then has a finite limit . *if diverges , then .   .[ q ]  for any in , the variables and are functions of , therefore they converge almost surely , because the reinforced random walk is a mixture of irreducible markov chains for which the latter converge .the reinforcement scheme defined in definition [ mixed - order reinforcement ] imposes some constraints on the limits of and .note that , , and never differ by more than ; we also know that the reinforced random walk is positive recurrent ( it is a mixture of irreducible , finitely - valued markov chains ) , so almost surely \\[-8pt ]  & = & \lim_{n\to\infty } n^{-1}w'_n(u^*)>0 . \nonumber\end{aligned}\ ] ] denote this limit .it is also easy to see that if , then for all ,   now , let be the visit to and let be the event that we make a transition to at .define we know converges a.s . to .therefore , a.s . , and by lvy s extension of the borel cantelli lemma ( lemma [ levy ] ) , this means that converges -a.s .to a set of transition probabilities , , for a variable - order markov chain with histories . to show that this markov chain is reversible , note that is the stationary distribution , because where we used equation ( [ balance ] ) in the last two identities . by equation ( [ reversibility of limit ] ), satisfies the conditions for reversibility .therefore , .the author would like to thank persi diaconis and vijay pande for valuable suggestions , and lutz maibaum for providing molecular dynamics datasets .

Abstract: we define a conjugate prior for the reversible markov chain of order . the prior arises from a partially exchangeable reinforced random walk , in the same way that the beta distribution arises from the exchangeable poly urn . an extension to variable - order markov chains is also derived . we show the utility of this prior in testing the order and estimating the parameters of a reversible markov model .  .