the defects in condensed materials have a significant effect on their physical properties .while their existence may be caused by impurities and flawed preparation process , controlled manipulation of defects can enable tuning of the physical characteristics of the material .the ability of a medium to host defects is a side - effect of symmetry breaking on the microscopic scale . as such ,the defects are intimately related to the topological properties of degenerate microscopic degrees of freedom . in materials without broken translational symmetry , such as uniaxial and biaxial nematics , ferromagnets and superfluid helium , the defects can be roughly classified by their dimension into point defects , line defects and walls .their classification with the aid of homotopy groups is a well - established and efficient way of describing their interactions and the conservation rules they obey .however , the topological analysis does not account for the fine geometric details , which are under control of the free energy .  specifically for nematic braids networks of closed nematic disclination lines  recent development has shown that additional geometric constraints allow for a finer classification of nematic braids and simplifies the understanding of their rewiring .the constraint of a fixed disclination profile that holds for nematic braids is not universal and does apply necessarily to some constrained or chiral systems .but even without constraints , it should still be possible to convey the topological information through the geometric primitives . in this paper , we demonstrate a formalism that can describe the most general types of nematic disclinations in terms of the local director behavior and enumerate them using quaternions . we show that our interpretation naturally extends the existing formalism for disclination loops and reveal a connection between closed disclination loops and two - dimensional nematic textures .before introducing any new results , we shall briefly overview the existing theoretical background regarding nematic defect loops .a nematic liquid crystal consists of elongated molecules that are locally orientationally aligned . in a continuum approximation , the order is described by the director , a unit vector field , pointing in the direction of average molecular orientation .the head - to - tail symmetry of the molecules causes the sign of the director to be ambiguous .topologically , the director field is a map from the coordinate space to the real projective plane  the ground state manifold ( gsm ) .the fundamental group distinguishes disclination lines , where the director makes a half - turn around the disclination , from nondefect states , but makes no distinction between different disclinations .our specific interest is focused on disclination lines that are closed into a loop .disclination loops may be linked by other disclinations or carry a topological point charge , measured by the second homotopy group . closed loops therefore also carry an additional topological index .an established result shows , that a set of disclinations obeys a law in the form where is the topological point charge of the set of disclination loops , are linking numbers between loops and the indices and run over all the loops .odd indices correspond to disclinations that are threaded by other disclinations an odd number of times while even indices correspond to those that are threaded by and even number , which also applies to unthreaded loops .this theory describes all existing topological properties of disclination loops .however , the index is an abstract attribute assigned to each of the loops , so the application to the systems that are found in experiments , simulations and theoretical models requires a way to relate this index to observable attributes .if instead of a three - dimensional space , the molecules of the nematic are restricted to move and orient in a two - dimensional plane , the system exhibits point defects , which can have any half - integer winding number , the lowest ones being and  .these two - dimensional point defects represent a restricted set of cross sections the disclination lines in three dimensions can have .the topological equivalence of all disclination lines in the three - dimensional case is caused by the ability of the director to rotate through the third dimension .if the director is forced by the free energy minimization or geometric constrants to be perpendicular to the disclination tangent , which is excellently obeyed in well - researched nematic colloids , closed disclination loops can be described as ribbons that follow the rotation of the profile .ribbons can be assigned an invariant called the self - linking number  , which in the case of disclinations quantifies how many times the three - fold disclination profile rotates around the tangent when we traverse the loop . in a braid of many disclination loops , each self - linking number is of the form , where the numerator is odd if the loop is threaded by an odd number of other disclinations , and even if the loop is linked by nothing or by an even number of disclinations .the self - linking numbers of a set of disclination loops with a profile obey a rule that bears resemblance to the jnich s rule ( [ eq : janich ] ) .the integers are the linking numbers between loops and are self - linking numbers of individual loops .the comparison of equations ( [ eq : janich ] ) and ( [ eq : sl_law ] ) suggests a connection between the index and the self - linking number . in the following section , we confirm and extend this notion to all disclination loops .( a ) the director profile specified on a torus enveloping a general disclination loop .the small radius is parametrized by and the large radius is parametrized by , producing a seamless doubly periodic parametrization .( b ) the director on any small cross section lies on a great circle in the gsm .the origin of the parameter provides a reference in the plane of the circle and the sense of rotation in gsm with increasing orients the circle s normal so that it defines a clockwise motion according to the right - hand rule .( c ) each circle in the gsm matches a cross section in real space with a fully specified parametrization .( a - c ) the orientation of the director cross section and its associated circle changes along the disclination with parameter , which is described by a continuously changing rotation . ]all disclinations in a 3d nematic have in common that the director on a test loop around the disclination traces an irreducible path in the ground state manifold ( gsm ) .as the gsm is a sphere with additional condition of the antipodal points identified , the loop is topologically equivalent to a half of a great circle , which can be symmetrically extended to the full circle ( fig .[ fig : figure1 ] ) . asthe vicinity of the disclinations is subject to elastic free - energy restrictions , this is a quantitatively reasonable approximation .  a closed disclination loop can be wrapped in a torus , , generated by revolving a disclination - encircling loop parameterized by , around the disclination loop parameterized by . for each value of , the parameter traces a different great circle in the gsm ( fig .[ fig : figure1]a ) .each circle is fully specified by its normal and a reference director at a chosen parameter , ( fig .[ fig : figure1]b , c ) .the sense of circulation around the circle is important and prescribes the sign of its normal .for example , the profile corresponds to the normal of the circle being anti - parallel to the disclination tangent in the real space , as the director rotates in the opposite sense compared to the circulation of the enclosing loop ( hence the minus sign in ) .the circle is invariant to rotations for around its normal because each point in the real space maps to two antipodal points in the gsm .let the parametrization of the director field on the torus be .when changes , the circle rotates , so the director at all values of parameter transform with the same rotation , parameterized by , the is simply a reference profile at a freely chosen point on the loop .this sets the initial rotation to unity , .the rotations vary continuously with loop parameter and thus describe a path in the space of all rotations in three - dimensions .we will use the quaternion representation of the group .the unit quaternions , and obey the conventional structure relations , , and . a rotation around an arbitrary axis by an angle follows the usual spinor formula rotation by around any axis yields , which is the irreducible rotational motion that ends up in the same state where it started .note that if we took the representation , these irreducible paths would be considered equal to the identity transformation , and some of the topological information would be lost .the continuity condition for a closed loop is satisfied if the director is continuous for _ all _ , that is , if the circle we obtain after completing the loop at ( ) matches the circle at the beginning of the loop ( ) up to a rotation around the loop tangent , which we align with the -axis .because the group covers the space of rotations twice , the set of possible cumulative rotations expands to the allowed rotations form a cyclic group , enumerated by an integer , which can be shown to correspond to the index introduced by jnich ( [ eq : janich ] ) . as is simply the total rotation of the director at any fixed on a circuit around the disclination , it measures the fundamental group of the nematic encircled by the disclination . are nontrivial rotations by and mean the disclination is threaded by another disclination . and correspond to an unlinked loop with even and odd topological charge , respectively ( this topic is discussed thoroughly in ) .each rotation can be decomposed into the rotation of the local coordinate frame relative to the global coordinate frame , and the rotation of the director relative to the local coordinate frame . we set the local coordinate frame to keep its -axis aligned to the disclination loop tangent and vary continuously along the loop ( e.g. the frenet - serret frame ) .we also ensure our framing is topologically equivalent to a framing of a planar circular loop without torsion , so the local coordinate frame simply rotates by when we traverse the loop : .this extra rotational offset is present for all the loops , so we can factor it out .the multiplication of by simply shifts by so an alternative parameter can be introduced , rewriting the jnich s law ( [ eq : janich ] ) as we can see that the number of loops plays a role in the conservation law because a trivial disclination loop with holds a nonzero topological charge ( one example is a saturn ring defect ) . all the significant information is now encoded in , which is measured in a convenient disclination - aligned local coordinate frame .each geometric operation on the disclination profile can now be directly translated into the quaternion language .first , we reproduce the result for the disclination profile . in the relative frame along the curve ,the director lies in the plane and can only rotate around the -axis , as it has to stay perpendicular to the disclination tangent .rotation of the profile consists of both the passive rotation of the coordinate frame and the active rotation of the director . as the director lies in a plane , it can be specified by an angle , so the profile rotation by yields a rotation of the director , and subsequently the entire circle ( fig .[ fig : figure2]a , b ) :   in a loop with a self - linking number , the profile rotates by a total angle of , and the equation ( [ eq : r_spinor ] ) gives comparison to yields the index .the factor of elegantly removes the fractional quantisation of the self - linking number that stems from the three - fold symmetry of the profile .this result immediately transforms the expression ( [ eq : janichast ] ) to ( [ eq : sl_law ] ) and explains the correlation between the linking of disclinations and the self - linking number , discussed in ref . .  a similar calculation yields and for disclinations with a profile .   ( a ) notation used in equation ( [ eq : alpha ] ) . the profile is two - dimensional , so the director can be represented by a single angle that varies with parameter .( b ) rotation of the director around the axis induces a rigid rotation of the disclination profile .depicted is a rotation of the director for , which yields a of a full turn for the profile ( ) .( c ) the transition between and disclination profiles is realised by a rotation for around a perpendicular axis ( axis , marked in black ) .there are two senses of rotation the one depicted has the parity . ]there is more to disclinations than just rotations around the tangent .an obvious possibility is rotation around a perpendicular axis for , where is the parity that distinguishes left - handed and right - handed rotation for .the result of such a rotation is the transition between and defect profiles ( fig .[ fig : figure2]c ) .the axis of rotation in equation ( [ eq : r_spinor ] ) can point at any angle in the plane , , which leads to   the rotations can be composed sequentially in the order they are encountered with increasing parameter . a real disclination loop can have a profile that rotates around a gradually varying axis that is neither parallel nor perpendicular to the disclination tangent. however , the principal reasoning in our derivation is topological , so the exact geometric realisation of the rotations is not important . for example, the location of a transition between profiles can be pinpointed to the place where the circle s normal is exactly perpendicular to the disclination s tangent . whether the transition between and disclination profiles happens along a very short segment of the disclination , or takes the entire disclination length to complete , has no effect on the calculated index , as the director field in the second scenariocan be smoothly combed to the first one without changing the topology .consequently , rotations around the axis ( ) and rotations for around a perpendicular axis ( ) are enough to describe all director profile variations : for the rotations , the angle is the local rotation angle of the director , which relates to the _ physical _ rotation angle of the profile : for a profile and for a profile .  because of the presence of the unit quaternion in the perpendicular rotation , it does not commute with -axis rotations .{\sf r}'_\perp[\theta]= { \sf r}'_\perp[\theta]{\sf r}'_z[-\phi]\ ] ] the rotation of has the opposite effect than the rotation of profile .the product of all local rotations must satisfy the continuity condition ( [ eq : quatern_cycle ] ) .as a perpendicular rotation introduces a factor , they must always appear in pairs if the disclination turns from to it must turn back into .the offset in the axis orientations adds an extra amount of torsion , {\sf r}'_\perp[p_2,\theta_2]&=-p_1p_2\exp({\mathbb{k}}(\theta_1-\theta_2)).\end{aligned}\ ] ] if two sequential rotations about the same perpendicular axis are performed with the same parity ( twice by or twice by ) , they yield which effectively changes the index by and switches the topological charge between even and odd .opposite parities naturally cancel out .as an example , consider a disclination loop with a reference profile of at , that rotates by around its tangent , flips to a profile around an axis at an angle and with parity , then rotates by and flips back to around an axis at with parity .the total rotation of the profile along the entire loop becomes .\end{aligned}\ ] ] the exponent must be a multiple of to satisfy the continuity equation , which puts a restriction on the angles and .this generalises the third - integer quantization of the self - linking number .when this condition is met , the above expression can be compared to to extract the index .note that the self - linking rotations of and parts have opposite - signed contributions to the index . taking a different part of the loop asa reference cyclically permutes the order of the transformations .for example , if a part serves as a reference , the entire exponent changes sign .this does not affect the rotations ( ) , but exchanges the signs of for linked disclinations .the same effect is observed if the parametrization of any of the disclination loops is reversed , which also reverses the linking numbers in equation ( [ eq : janichast ] ) .this ambiguity stems from the requirement of the classification with homotopy groups to have a fixed reference a base point .the choice of reference is very important once more than one loop is present in the system .most of the disclination loops can be analysed directly by the described formalism . wherever the normal of the circle traced by the director on the unit sphere is not perpendicular to the disclination tangent, it can be simply regarded as being a or a profile and the director can be unambiguously combed to the perpendicular plane .the decomposition thus takes advantage of easily recognisable physical features of the disclination profiles and can be counted by hand if provided with a properly visualised director field . in singular cases , such as a twist disclination line , which is exactly half - way between and profiles , appropriate quaternion expressions can be quickly retrieved using a procedure similar to the one described above .the correspondence between subsurface disclination loops and geometric features of nematic textures on a sphere - like surface ( a ) .the patches of perpendicular director ( red for outgoing and blue for ingoing field ) are segmented by boundaries ( orange ) with the director parallel to the surface .boundary signature corresponds to ( b ) or ( d ) profiles of the subsurface disclination loops , while the grains places where the director is parallel to the boundary correspond to the twist profile , which is an intermediate state in transition between the other two elementary disclination profiles . ]  to present above statements from a different perspective , we must first shortly review the theory behind nematic point defects . the topological charge of a set of defects contained inside a surface is determined by the second homotopy group class of the nematic texture on said surface . a convenient way of calculating the topological charge is introduced in reference .it states that a surface texture can be split into patches ( 2-dimensional objects ) where the director is perpendicular to the enclosing surface , separated by boundaries ( 1-dimensional objects ) where the director is parallel to the surface , but perpendicular to the boundary itself , and grains ( isolated points on the boundaries ) where the director is also parallel to the boundary of which it is part .the topological charge of the texture can then be calculated by counting the contributions of boundaries and grains . the first term is a half of the euler characteristic of the enclosing surface and is related to the fact that a radial hedgehog has a topological charge of .  there is a striking similarity between the classification of disclination loops and the classification of nematic textures .both theories observe the behaviour of the director in a local frame , aligned to the surface or the curve in question .this splits the global contribution of the shape s topology from the local geometric contributions . for disclination loops ,the global part is the factor that describes the nonzero topological charge of a simple saturn ring defect , while for surface textures , it is the contribution of the euler characteristic that describes the nonzero topological charge of a radial hedgehog .both approaches also assume the texture or the director profile can be smoothly transformed , so that all the changes happen in narrow transition zones .the figure [ fig : figure3 ] shows that every surface texture can be alternatively interpreted as a homeotropic core with added disclination loops below the surface of interest at the places where the boundaries are , and with disclination profile transitions where the grains are .the condition that the grains come in pairs and change the topological charge by if they have the same parity , corresponds exactly to the same rule for the profile transitions .the topological charge of the texture can therefore also be calculated by adding the core defect ( the euler characteristic ) to the charges contributed by jnich s indices for all the virtual subsurface disclination loops below the boundaries , for boundaries and grains .one big difference between equations ( [ eq : texturefull ] ) and ( [ eq : texturemod ] ) is that the result is only given by modulo in the latter .this is a natural consequence of dealing with line defects instead of point defects ( see ref . ) . however , the narrow subset of disclinations that can act as subsurface representations of boundaries , are restricted by the surface of the homeotropic core , so there must be no linking and self - linking .the correspondence between the textures and disclination loops is not necessarily only theoretical .if the surface on which we are observing the texture lies just above the disclinations , the geometric features on the surface texture will reflect the underlying disclination geometry . from this , another strong conclusion can be derived .the defect rank , a geometric measure for classifications of point charges , introduced in ref . , is only valid if there are no grains on the surface . by the correspondence principle, we can state that if there are no disclination transitions , no linking and no self - linking in the system , the defect rank can be used and the usual intuition of assigning proper integer charges to both disclination loops and point defects is valid .as soon as the system becomes entangled or there are different disclination profiles present in the system , there are possible discrepancies in the full integer conservation of the topological charge and only the even and odd conservation law holds .the topological properties of nematics are an interesting topic , crucial for interpretation and design of experiments .although the basic , strictly topological formalism has been known for a long time , easy application of the theory to the real - world examples requires geometrization of the basic topological concepts . by considering local director behavior ,we can account for restrictions enforced by the energy minimisation and thus make a fine - grained classification using not only the homotopy theory but also additional geometric information .we have decomposed a general disclination loop into a sequence of local rotations that must add up continuously to a rotation that satisfies the director continuity in a closed disclination loop .quaternions are an analytically efficient way of enumerating the local behavior of the director profile .the use of quaternions further reinforces the close relationship between the uniaxial and biaxial nematics , as the unified handling of the entire cross section removes the differences between both phases .  as an extension of the theory of defects ,the introduced formalism describes a wider range of disclination networks .notable examples of systems that exhibit transitions between different director profiles are confined cholesteric phases , optically induced defects and transient defects that are created with quenching from the isotropic phase .studies of random blue phases and entangled structures in random or structured pores could also benefit from improved classification of disclination profile variations .the formalism we presented also shows a stunning connection with the texture decomposition , suggesting that the approaches of geometrisation for different kinds of defects can be unified , leading toward a consistent set of tools for enumeration of defects .this research was funded by slovenian research agency under contracts no .p1 - 0099 and no .j1 - 2335 , namaste center of excellence , and hierarchy fp7 network 215851 - 2 .s.  . was supported , in part , by nsf grant dmr05 - 47230 .both authors also acknowledge partial support by nsf grant phy11 - 25915 under the 2012 kitp miniprogram `` knotted fields '' .

Abstract: disclination lines in nematic liquid crystals can exist in different geometric conformations , characterised by their director profile . in certain confined , colloidal and even more prominently in chiral nematics , the director profile may vary along the disclination line . we construct a robust geometric decomposition of director profile variations in closed disclination loops based on a quaternion description and use it to apply topological classification to linked loops with arbitrary variation of the profile . the description bridges the gap between the known abstract classification scheme derived from homotopy theory and the observable local features of disclinations . we compare the resulting decomposition of disclination loop features to a similar decomposition of nematic textures on closed surfaces


during the past one - and - a - half decades , the transient fluctuation theorems , known as jarzynski equality and crooks relation have attracted wide interest .they constitute exact relations for the non - linear response of systems in thermal equilibrium to arbitrary perturbations . as suchthey apply to a wide variety of systems , including for example bio - molecules , colloids , nuclear spins , cold atoms , and clusters of atoms , to name just a few .these theorems have motivated numerous studies on the dynamics of small systems and also have revived and fertilized the field of nonequilibrium thermodynamics .reviews of the different theoretical aspects of transient fluctuation theorems applying to classical and quantum dynamics of open and closed systems can be found in .apart from theoretical studies , the fluctuation theorems have also inspired experimental studies and have become the basis of a method for determining free energy differences of different configurations of large molecules .the central object of the transient fluctuation theorems is the _ work _performed on a system by the variation of external parameters .work together with heat constitute basic notions on which any thermodynamic description rests .therefore , a convenient operational definition for work that is also applicable to quantum systems is of eminent importance in the thermodynamic description of biological and artificial devices on the nanoscale , the design of quantum engines , and quantum information processing .  in the present work , we will restrict ourselves to the discussion of closed quantum systems .the validity of the fluctuation theorems is then based on the reversibility of hamiltonian dynamics and on the assumption that the system is initially prepared in canonical equilibrium described by a gibbs state ; out of this state , external time - dependent forces entering the system hamiltonian drive the system away from equilibrium according to a prescribed protocol of finite duration .the work performed on a system in such a process is a statistical quantity , which can be characterized by a probability density function ( pdf ) .an associated backward process starts in thermal equilibrium at the initial inverse temperature and at the time - reversed final parameter values .the parameter values are then retraced according to the backward protocol where is the parity of the parameter under time - reversal , for example for an electrical and for a magnetic field .the crooks relation then connects the pdfs of the forward protocol and of the backward protocol in the following way : provided that the hamiltonian satisfies the instantaneous time reversal symmetry where denotes the time - reversal operator . in the crooks relation, denotes the difference in free energies of canonical equilibrium systems at the force values at the end and at the beginning of the protocol .the free energy is defined in terms of the respective canonical partition function the jarzynski equality follows as a direct consequence of the crooks relation .  for later use, we note that the crooks relation can be expressed in terms of the characteristic function of work , defined as the fourier transform of the work pdf , in the equivalent way , reading   other than in the classical case , direct experimental confirmations of the quantum transient fluctuation theorems are still missing .et al_.  suggested an experimental proof by means of a cold atom sitting in a parabolic trap whose position can be varied .the difficulty of this , but also of other experiments testing transient quantum fluctuation theorems , lies in the measurement of the work performed on the system in a single realization .this requires two measurements of the system energy in the beginning and at the end of the protocol . in the theoretical investigations of the transient quantum fluctuation theorems , these measurements are projective means that an energy eigenvalue with a corresponding eigenfunction will be measured in a state described by a density matrix with the probability and will leave the system immediately after the measurement in the state .one might suspect that a weaker , less invasive form of the energy measurements is simpler to realize in an experiment and still might leave the transient fluctuation theorems intact .this belief could be supported by the argument that any weaker , or less invasive measurement can be obtained as a projective measurement on a so - called ancilla which is typically an auxiliary two - level system that has to be brought into an entangled state with the actual system . under proper conditions ,the measurement on the ancilla gives the required information on the system , possibly with somewhat reduced accuracy , while the back - action of the ancilla on the system is weaker than a direct measurement would impose .  in two recent papers ,an experimental approach was suggested by means of which the characteristic function of work is encoded in the reduced density matrix of an ancilla .its realization requires a time - dependent system - ancilla coupling which follows the original force protocol amended by constant force periods of variable lengths . a tomography of the final ancilla state yields the characteristic function of work .in this way , projective energy measurements are avoided , yet , unlike the original setting , the forcing of the system is mediated by the coupling to the ancilla .the confirmation of the crooks relation and the jarzynski equality by means of this method was reported in .another experiment utilizing circuit cavity qed was suggested in ref . .it though remains an open question whether generalized measurements can replace the projective energy measurements and still conform with the fluctuation relations .the paper is organized as follows . in section [ s2 ]we shortly review the theory of generalized measurements ; for more extensive expositions we refer to the literature . in section [ s3 ]we derive the work distribution in the presence of generalized measurements and analyze in section [ s4 ] the restrictions on these measurements under which the crooks relation and the jarzynski equality hold .we illustrate our results by the example of a two level system whose hamiltonian is suddenly changed , and end with some conclusions in section [ s5 ] . for the sake of the reader sconvenience , some technical arguments are presented in appendices [ a1 ] , [ a2 ] , and [ a3 ] .a generalized measurement is formally characterized by a set of measurement operators satisfying the normalization condition where denotes the identity operator on the hilbert space of the considered system . in the present study ,energy measurements are of central importance . for that purpose ,we need measurement operators that can identify the eigenstates of a hamiltonian with spectral representation where , for the sake of notional simplicity , we restrict ourselves to non - degenerate energy - eigenvalues .the identification of an energy eigenstate by means of the measurement operator in general will not be perfect , rather the expression quantifies the likelihood of an erroneous assignment if a measurement is applied on the energy - eigenstate . for general energy measurements ,the post - measurement state is given by the density matrix where denotes the density matrix immediately before the measurement and the expectation value of the `` effect '' , gives the probability with which the eigenstate is observed . if a measurement is performed but its result is not registered , the post - measurement state is given by .we want to emphasize that an energy measurement operator is primarily indicative of an energy eigenstate ; the assignment of the energy eigenvalue occurs by the quantum number .therefore , the energy measurement operator need not depend on the actual energy eigenvalue .below we will make use of this possibility .  in the particular case when the conditional probability is sharp , , we call the measurement _ error - free_. any error - free measurement operator is characterized by a single , normalized post - measurement state in terms of which it can be written as see appendix [ a1 ] for details . in general , for different , the post - measurement states need not be mutually orthogonal or linearly independent .  a special case of an error - free measurement is a projective measurement , for which and hence , .projective measurements are the only measurements that are error - free and that reproduce the result of the first measurement in an immediate repetition of the same measurement .error - free measurements which are not necessarily repeatable were introduced by landau and lifshitz in the first russian edition of their quantum mechanics in 1948 ; for an english translation see . according to the classification by wiseman and milburn , error - free measurements are called sharp measurements .as already mentioned , to obtain the work which is performed on a system during a protocol , two energy measurements are required , one at the beginning , and the second one at the end of the protocol . since , in general , the respective hamiltonian operators and at the beginning and at the end will be different , their instantaneous sets of eigenfunctions and and eigenvalues and will in general also differ . therefore , different sets of measurement operators and will be needed to determine the work . the probability to register an energy eigenstate of the hamiltonian given by ( [ pn ] ) with describing the thermal equilibrium at the beginning of the protocol . the conditional probability to measure theeigenstate of upon the observation of the eigenstate then becomes where denotes the unitary time evolution for the period of the protocol , given by the solution of the schrdinger equation with the resulting joint probability to measure first and then , the pdf of work takes the form : the characteristic function then takes the form where with and . requiring that the crooks relation ( [ gg ] ) also holds in the presence of generalized measurements , we find as a condition on the measurement operators the equation to obtain the right hand side , we expressed the time evolution of the reversed protocol , in terms of that of the forward process by means of time reversal symmetry , i.e. , by , .the condition ( [ cc ] ) is necessary and sufficient for the crooks relation to be satisfied .it presents a central formal result of our paper .we ask now whether energy measurement operators exist that do not depend on details of the protocol but for which the crooks relation is satisfied .more precisely , we want to identify _ universal _ measurements operators , and , that satisfy the crooks relation for all possible protocols having the same pairs of initial and final energy eigenstates , independent of the particular values of the eigenenergies at the beginning and the end . as a consequence ,universal measurements operators must be independent of the temperature of the initial state and of the particular protocol .obviously , projective measurements are universal , but are there also others ?the answer to this question can be found by analyzing ( [ cc ] ) . as just mentioned , for universal measurement operators , ( [ cc ] ) must be satisfied for any protocol that connects the initial and the final hamiltonian within a finite time .therefore , it must also hold for the following particular protocol consisting of two sudden switches : one at the beginning and the second one at the end of the protocol .the first switch suddenly changes the hamiltonian from its initial form to . here , is a positive time scale and , as a hamiltonian , must be hermitean , but otherwise it is arbitrary .the second switch results in the final hamiltonian . for this protocol , the time - evolution from to is given by because the initial and final moments of the protocol at which the hamiltonian differs from do not contribute to .with being an arbitrary hamiltonian and an arbitrary time , the condition ( [ cc ] ) must be satisfied for any unitary operator in the case of universal measurements . as the simplest choice, one may take leading with to using the explicit expressions ( [ q ] ) and ( [ r ] ) , we find where \:,\label{cab}\end{aligned}\ ] ] with because the coefficients and are independent of , , , and for universal measurements , the terms must vanish individually , further implying that for and for .taking the sum of over all and , accordingly , the sum of over all one obtains for the conditional probabilities hence , both the first ( ) and the second ( ) energy measurements must be error - free . using the expression ( [ mef ] ) for error - free measurements ,we find as a necessary condition that the measurement operators assume the form where denotes the instantaneous eigenstate of and the respective , normalized post - measurement state .putting the explicit form of the measurement operators ( [ mnef ] ) into the necessary and sufficient condition ( [ cc ] ) for the validity of the crooks relation , one obtains = 0\:.\ ] ] for a universal measurement this equation must hold for all values of and and hence the square brackets must vanish for all values of the indices and , i.e. , as explained above , for a universal measurement , this condition must also be satisfied for all possible unitary maps .this requirement leads after some algebra , for details see appendix [ a2 ] , to the conclusion that the post - measurement states have to coincide with the eigenstates , for and , possibly up to irrelevant phase factors . in other words ,our main conclusion is that projective measurements are the only universal measurements that lead to the crooks relation .however , one may design the post - measurement states such that they satisfy the relation ( [ db ] ) for a given protocol . for this particular , non - projective error - free measurement , the crooks relation and consequentlyalso the jarzynski equality are satisfied .it is interesting to note that ( [ db ] ) is analogous to the generalized detailed balance condition for transitions in forward and backward force protocols when the respective initial states are microcanonical . in the present case of generalized measurements, this generalized detailed balance condition requires that the respective transition probabilities from the post - measurement states of the first to the target states of the second measurement are the same for the forward and the backward protocol , i.e. , this follows from the fact that the right hand side of ( [ db ] ) can equally be expressed in terms of the time - reversed dynamics as .   requiring only the validity of the jarzynski equality for universal generalized measurements ,one finds that the initial energy measurement must be error - free , with post - measurement states that form a complete orthonormal basis set , hence satisfying moreover , the second energy measurement must be determined by effects that have unit trace , . for a proof of these necessary and sufficient conditionssee appendix [ a3 ] .we illustrate our results by the example of a two level system which undergoes a sudden switch from to where and denote pauli spin matrices .the eigenvectors and eigenvalues of are given by and those of by where in the energy expressions the upper ( lower ) sign corresponds to the label 1 ( 2 ) , and denote the eigenstates of with the respective eigenvalues .the angle follows from . in the following examples we set .we first consider error - free measurements of the form with as defined by the eigenstates of the hamiltonians with and the orthogonal post - measurement states in figure [ f1 ] , we compare the actual probability distribution of the four possible outcomes of the work for the backward process with that of the the prediction resulting from the crooks relation ( [ cr ] ) based on the probability distribution for the forward process . as a quantitative measure we used the sum of the absolute differences of these , .this distance may take values between and , where indicates perfect agreement .the maximal distance reveals two completely disjoint distributions . in this example the initial and the final energy measurements are both error - free with orthogonal post - measurement states . since the effects of the second measurement have unit trace , , the jarzynski equality is satisfied .   and . for , , and the generalized detailed balance relation ( [ db ] ) is satisfied , and therefore the crooks relation holds , even though these measurements are not projective.,scaledwidth=60.0% ]  in the second example , illustrated in figure [ f2 ] , we consider the same protocol but now choose a linear combination of projective energy measurements given by with .these measurement operators give successful and wrong assignments with probabilities and , respectively . the deviation from the jarzynski equalitycan be quantified by the product \nonumber\\ & \quad+ p^{\mathrm{eq}}_2(\tau ) \left [ \sin^4(\alpha ) e^{\beta \epsilon }  + \cos^2(\alpha )  + \cos^2(\alpha ) \sin^2(\alpha ) e^{-\beta \epsilon } \right ] \ : , \label{ja}\end{aligned}\ ] ] where .a value of unity indicates the validity of the jarzynski equality and deviations quantify its failure . for the angles and , the measurement is projective and hence the jarzynski equality is satisfied . at the mixing angle , the measurement is also projective , but the assignment of energies is inverted .the deviations are more pronounced at low temperatures and diminish with increasing temperature .+   are displayed for the measurement operators ( [ m2 ] ) as a function of the mixing angle where and correspond to projective measurements with correct assignment , respectively . for , .deviations of the ratio from unity decrease with increasing temperature ; ( dash - dotted line ) , ( dashed line ) , and ( solid line ) ., scaledwidth=60.0% ]  in the second example the effects of the second measurements have unit trace , whereas the first energy measurements are not error - free , and hence the initial energy measurements cause the violation from the jarzynski equality .  in the third example we consider error - free measurements with complete post measurement states initially but with final energy measurements whose effects have traces which may differ from unity . choosing the second measurements as we find for the deviation from the jarzynski equality only at both effect traces , become unity and consequently the jarzynski equality holds only for this particular angle . the deviations from the jarzynski equality again are largest for low temperatures for which the population difference approaches unity , and goes to zero in the limit of infinite temperature .we found that generalized energy measurements at the beginning and the end of a force protocol lead to work pdfs which in general violate the transient fluctuation theorem of crooks .the jarzynski equality holds for universal energy measurements if the first one is error - free with a complete set of orthonormal post - measurement states , and the second one has effects with unit trace .the crooks relation remains valid for other than projective measurements only if both the initial and final energy measurements are error free with post - measurement states that are adapted to the actual protocol in such a way that the transition probabilities satisfy the generalized detailed balance relation ( [ db ] ) .whether such measurement operators can be experimentally implemented in an easier manner than projective energy measurements remains an important open question .we acknowledge support by the max planck society , the korea ministry of education , science and technology ( mest ) , gyeongsangbuk - do , pohang city , as well as by the basic science research program through the national research foundation of korea funded by the mest ( no .2012r1a1a2008028 ) .any energy measurement operator may be represented by means of energy eigenfunctions as and accordingly the adjoint operator as where the coefficients are complex numbers which , as a consequence of ( [ m1 ] ) , fulfill the normalization condition the conditional probability , defined in ( [ pmn ] ) , specifying the reliability of the measurement can be expressed in terms of the coefficients  for error - free measurements satisfying , this implies for , and hence where the normalization follows immediately from the normalization condition ( [ m1 ] ) .this completes the proof of the expression ( [ mef ] ) for error - free measurement operators .finally we prove that the post - measurement states of an error - free measurement provide a complete orthonormal basis - set if they allow a partition of the unity , i.e. if holds .multiplying both sides by from the left and from the right we obtain with the normalization we immediately obtain the missing property of orthogonality for .denoting we may write ( [ db ] ) as with the spectral representation of the unitary operator this can be further transformed into \nonumber \\ & \qquad= 0\ : .\label{c1}\end{aligned}\ ] ] because for universal measurements the time evolution operator may be considered as an arbitrary unitary operator ( see the discussion of the first paragraph of section [ s41 ] ) , the phases as well as the complete orthonormal basis set are arbitrary . therefore , the brackets under the sum of ( [ c1 ] ) must vanish individually . rearranging the termswe obtain hence , the ratios of matrix elements do neither depend on nor on . using again the notation and we may treat both ratios simultaneously .they can be rewritten with the help of ( [ pin ] ) and ( [ sin ] ) as obviously , the constant is given by a product of the form with solving for , multiplying with , and finally summing over all yields because the choice of the basis is arbitrary , we may take and hence obtain where the normalization of implies that is a mere phase factor , and hence can be put as .the jarzynski equality ( [ je ] ) may be expressed in terms of the characteristic function ( [ gw ] ) as in combination with ( [ g ] ) , ( [ q ] ) , ( [ r ] ) and the spectral representation of the initial hamiltonian this condition becomes where denotes the equilibrium probability of the energy state . the coefficients are defined by where determines the conditional probability to find in the initial and in the final measurement when the system was prepared in the eigenstate .  for universal measurements ,the coefficients are independent of the final energies .therefore , the condition ( [ cp ] ) implies that for all . using ( [ cmdefn ] )this leads to due to the non - negativity of the conditional probabilities and their independence of temperature it follows that this implies and consequently , that the first measurement must be error - free .using the form of an error - free measurement , the condition ( [ spmnk0 ] ) in combination with ( [ pmnk ] ) becomes by means of the spectral representation of the propagator we obtain because the phases are arbitrary in this double sum over and the diagonal terms must sum up to unity , while the non - diagonal elements must vanish individually .hence , we obtain in the second equation one can divide by the matrix element of because the basis is arbitrary .this leads to   only for an operator proportional to unity , the non - diagonal elements vanish in any basis .hence , we find if the post - measurement states were not orthogonal to each other one could find an orthonormal basis with respect to which the left hand side of ( [ psi1 ] ) would have non - diagonal elements in contrast to the right hand side .hence , the post - measurement states must represent a complete orthonormal basis with .finally , from ( [ ckk ] ) in combination with the completeness relation , we obtain this completes the proof that the jarzynski equality implies for universal measurements that ( i ) the initial measurements must be error - free , ( ii ) the corresponding post - measurement states must form an orthonormal complete basis , and ( iii ) the effects of the second measurements must have unit trace .the sufficiency of these coefficients can be easily seen by inspection .99 jarzynski c 1997 _ phys . rev .lett . _ * 78 * 2690 crooks ge 1999 _ physe _ * 60 * 2721 jarzynski c 2011 _ ann .cond . mat .* 2 * 329 esposito m , harbola u and mukamel s 2009 _ rev .phys . _ * 81 * 1665 campisi m , hnggi p and talkner p 2011 _ rev .phys . _ * 83 * 771 seifert u 2012 _ rep .phys . _ * 75 * 126001 douarche f , ciliberto s , petrosyan a and rabiosi i 2005 _ europhys . lett ._ * 70 * 593 bustamante c , liphardt j and ritort f 2005 _ phys . today _* 58*(7 ) 43 huber g , schmidt - kaler f , deffner s and lutz e 2008 _ phys .lett . _ * 101 * 070403 kurchan j , arxiv : cond - mat/0007360 tasaki h , arxiv : cond - mat/0009244 talkner p , lutz e and hnggi p 2007 _ physe _ * 75 * 050102 von neumann j 1996 _ mathematical foundations of quantum mechanics _ ( princeton university press , princeton ) nielsen ma and chuang il 2000 _ quantum computation and quantum information _( cambridge university press , cambridge ) dorner r , clark sr , heaney l , fazio r , goold j and vedral v 2013 _ phys .lett . _ * 110 *230601 mazzola l , de chiara g and paternostro m 2013 _ phys .lett . _ * 110 * 230602 batalho t , souza am , mazzola l , auccaise r , sarthour rs , oliveira is , goold j , de chiara g , paternostro m and serra rm , arxiv:1308.3241 campisi m , blattman r , kohler s , zueco d and hnggi p , arxiv:1307.2371 wiseman hm and milburn gj 2010 _ quantum measurement and control _( cambridge university press , cambridge ) landau ld and lifshitz em 1958 _ quantum mechanics _ , 7 ( pergamon press , new york ) andrieux d and gaspard p 2008 _ physlett . _ * 100 * 230404 talkner p , morillo m , yi j and hnggi p 2013 _ new j. phys . _ * 15 * 095001

Abstract: the transient quantum fluctuation theorems of crooks and jarzynski restrict and relate the statistics of work performed in forward and backward forcing protocols . so far , these theorems have been obtained under the assumption that the work is determined by projective energy measurements at the end and the beginning of each run of the protocols . we found that one can replace these projective measurements only by special error - free generalized energy measurements with pairs of tailored , protocol - dependent post - measurement states that satisfy detailed balance - like relations . for other generalized measurements , the crooks relation is typically not satisfied . for the validity of the jarzynski equality , it is sufficient that the first energy measurements are error - free and the post - measurement states form a complete orthonormal set of elements in the hilbert space of the considered system . additionally , the effects of the second energy measurements must have unit trace . we illustrate our results by the example of a two - level system for different generalized measurements .


understanding the mechanisms of learning and maintenance of effective behavior in stochastic environments with complex hierarchy of goals is one of key issues in the study of neural processing and for the development of bio - inspired artificial intelligence .however , these mechanisms are better understood if history of their development is taken into account .  fields of machine learning and adaptive behavior have been rapidly developing in the recent decades .one of the most popular approaches in these areas is reinforcement learning ( rl ) and its modifications . it can be effectively used for learning of autonomous agents in many domains .however , problem spaces with multiple goals require execution of alternative behavioral strategies that are not possible for rl algorithms , since the behavior of the agent is determined by the single value function .also , it is not always possible to define appropriate value function _ a priori_. to address this issue approaches for the generation of reinforcement values using evolutionary algorithms are currently under development .other algorithms are also unable to produce effective solution for the environments with hierarchy of goals .the various methods designed for the generation of behavioral sequences in most cases can not effectively work in situations with a large number of goals present in the environment and when an agent has to autonomously explore them .neuroevolutionary approaches have been successfully used for the synthesis of autonomous agent s controllers in low - formalized tasks .  to solve the problem of alternative behaviorsthe algorithm should implement non - markov decision process in opposite to reinforcement learning .in other words for selection of different actions at the same state of the environment an agent should have memory of previous actions .short - term memory ( stm ) is extensively covered in the area of recurrent neural networks research in terms of the signal reverberation and serial order recall .theorists suggested reverberation phenomenon as significant mechanism for stm . nevertheless neural principles underlying generation of the behavior as a non - linear integration of the sensory information and internal state of the network are not yet discovered . as wellthe question of the role stm plays in the adaptive behavior with alternative actions is still underexplored .  in this paperwe present a model of an agent situated in an environment with hierarchy of goals .the behavior of the agent is controlled by recurrent neural network .we simulated evolution of agents in the model world and studied emergence of short - term memory and its role in adaptive behavior .in the current study a state of an environment in which an autonomous agent `` lives '' is represented by binary vector :    at any discrete time step the agent may change a single bit of this vector to the opposite .thus , environmental structure is a -dimensional hypercube .every dimension of this hypercube might be interpreted as some feature of the real environment , for instance temperature , pressure , etc .competing goals of different complexity are prespecified in the environment as an ordered set of single bit changes of the environmental state vector :    where is a number of target bit of the state vector , is target value of the bit , is a complexity of the goal. goals of varying complexity are defined in the environment , together forming a branched hierarchical structure .we have generated structure of the environment by , firstly , composing hierarchy of full goals and then each goal was divided into sequence of subgoals .to determine the complexity of a particular environment we introduce occupancy based on a probabilistic approach :    where is the number of goals in an environment , is the number of unique bits of the state vector , changed during the process of achieving a goal ( single bit can be changed several times ) .the value , which is opposite to the occupancy , we shall call the difficulty of the environment :    reward is associated with each goal in the environment and directly proportional to the complexity of corresponding goal . after the agent reached goal , level of accruing reward associated with this goal is reset and then linearly recovers to the original value over the time .  environment can be either deterministic or non - deterministic . in the later changes of the state vectoroccur not only when the agent performs actions , but also stochastically according with a fixed probability .agent s behavior in the environment is controlled by artificial neural network ( ann ) of arbitrary topology , which develops through the evolutionary process .ann is divided into input , output and number of hidden layers .the network consists of mcculloch - pitts neurons with nonnegative logistic activation function .signal passes through the synapse only when output value of a presynaptic neuron is above a certain activation threshold ( was set to 0.5 in all simulations ) .thus , we introduce two possible states for each neuron : active ( when output is above a threshold ) and inactive ( otherwise ) .the current state vector of the environment is directly fed to the neurons of input layer of the network .hidden layers allow existence of recurrent connections with delayed transmission .pair of the most active neurons of output layer encodes agent s action at the current time step .each pair stands for a particular bit of the state vector and target value .thereby transfers of a bit from 0 to 1 and vice versa are encoded by different pairs of the output neurons .the agent could produce ineffective actions , when attempts to change a state of a bit to one in which it already is .population of agents evolves in the hypercube environment described in previous section .every individual is separately placed in the environment to evaluate fitness . for a given time the agent operates in the environment , reaching goals and accumulating reward .value of accumulated reward is inaccessible to the agent and total reward affects its reproductive success .  to evolve agents anns we used neuroevolutionary algorithm based on duplication of neurons .this algorithm is similar to well - known neat as it makes possible evolution of neural network topology but in more natural way by duplicating neurons with attached connections .specifically we use _neuron s duplication _ mutation when duplicated neuron inherits from parent the whole synaptic structure instead of _ add node _ mutation of neat .incoming connections of the descendant and parent neurons retain its previous weights , while weights of outgoing connections divided in half for both neurons .thus , two neurons in the aggregate perform the former function , but later in evolution the descendant neuron could diverge into a separate structure . to optimize evolution bydynamically reducing dimensionality of the search space we also introduced _ delete connection _ mutation , which performed in the same manner as _ add connection _ mutation .neurons that lost all their connections are being deleted from the network .we did not use any crossover algorithm , since preliminary results of comparison between two versions of evolutionary algorithm ( with crossover and without ) have not showed significantly difference .the structure of the agents networks in initial population consists of input and output layers and one additional interneuron , which will be an ancestor for all future interneurons .this initial interneuron is fully connected to input layer ( by incoming connections ) and to output layer , herewith there are no initial direct connections between input and output neurons .we used the following parameters of the artificial environment in all simulations : dimensionality of the hypercube environment  8 bits ; agent lifetime duration  250 time steps ; reward s recovery time  30 time steps ; probability of random change of the state vector s bit  0.0085 ( for each bit ) .evolutionary algorithm was running with parameters : population  250 agents ; period of evolution  5000 generations ; probability of the synaptic weight mutation  0.6 ( for each synapse ) ; variance of the synaptic weight mutation  0.08 ; probability of adding the synapse  0.1 ( for whole network ) ; probability of deleting the synapse  0.05 ( for whole network ) ; probability of neuron s duplication  0.007 ( for whole network ) .as the first step we studied how occupancy of the environment affects efficiency of evolved behavior in deterministic and stochastic environments .as expected the simulation results show the growth of an average cumulative reward with increase of the occupancy , reflecting that a task is easier when the environment is more densely populated with goals ( fig .[ fig : statevolution ] ) .agents evolved in non - deterministic environments behave significantly more successfully ( fig .[ fig : statevolution ] ) in terms of average accumulated reward and in most cases have richer repertoires of behavior .such phenomenon can be explained by the fact that during the `` life '' in an environment which could suddenly change the agent with a wider spectrum of behavioral policies should be more successful .thus stochasticity of the environment contributes to the selection of agents with more flexible and robust behavior , allowing them to reach greater number of goals from different initial position .however , with increasing probability of random changes in the environment one can observe a sharp decline in the efficiency of the evolution ( data not shown ) due to the destabilization of all policies caused by excessively frequent changes in the environment .we then analyzed evolution of behavioral sequences of the agents in the environment of intermediate goal occupancy .typical evolved behavior usually consists of two phases : preliminary phase of converging to the main behavioral cycle and then residing on it . in a particular behavioral runboth of these phases depend on the initial state of the environment , from which the agent has been started .we shall call the main behavioral cycle ( sequence of actions or attained goals ) a behavioral strategy.  results of simulations demonstrate that sharp increase in the average cumulative reward among the population usually coincides with emergence of a large number of new cycles and their dynamic competition .corresponding periods are the most interesting in the study of behavioral evolution . on the fig .[ fig : behevolution ] we give an example of such period .one can observe competition between different behavioral strategies during a very limited evolutionary period .herewith new strategies are appearing as an extension of previously evolved cycles .extension can be implemented not only as simple addition of a new part to an old one , but also as complex compilation of two strategies .eventually one of the strategies wins the competition and dominates in the behavior of the most of the population .then evolution occurs through increase in number of states , from which it could be implemented .the analysis of behavioral strategies emerging in the simulated evolution shows that the agents acquire ability to store short - term memory due to reverberation in the neural network by recurrent connections .the evidence for short - term memory comes from the fact that agents can implement policies based on alternative actions .agents usually can perform different actions from one state of the environment depending on the previous history of behavior .state transition diagram for the exemplar evolved behavior with three alternative actions is shown on the fig .[ fig : alternativebehavior ] .such phenomenon would not been possible in the case of reactivity work of the neural network .corresponding behavior is generated by a neural network consists of 30 neurons with only 15 interneurons and 611 synaptic connections .only a small number of neurons significantly change output and affect the decision making at states associated with alternative situations .this group of neurons is specialized on particular alternative actions .the majority of neurons remain at the same level of activity , while neurons which determine a behavior change their activity from zero to maximum level .based on the various behavioral policies the deepest short - term memory we found was at least 4 past states ( lower bound ) .such conclusion was made after considering situations in which alternative actions are performed with the same history of behavior and , thus , the lower bound can be determined as the first different action of the two behavioral sequences .the ability to use short - term memory makes possible implementation of much more complex behavior and accumulation of more reward during the agent s `` life '' .example of a behavioral strategy , based on the alternation of two cycles of actions , is shown on the fig .[ fig : behstrategy ] .since after the goal is reached it restores the reward for a fixed number of time steps , such a strategy allows goals that were reached on the first cycle to restore their value , while the agent passes the second cycle .behavior on the fig .[ fig : behstrategy ] can be implemented using short - term memory with depth of two states .  primitive alternative behavior in the modelcould also be supported by slow neurodynamical process ( fig .[ fig : slowprocess ] ) . in this casethe `` memory '' , required to perform alternative actions , could be up to 30 previous states .the mechanism underling this type of behavior is oscillatory dynamic of the neuron s output .while output of a neuron is above threshold , and therefore transmitted by outgoing connections , agent performs the first part of behavior . at some point output falls below the transmission threshold and the neuron no longer inhibits the other neuron , which activity is essential for the second branch of behavior .    in the records of neural network activity ( fig .[ fig : rasterplot ] ) during the entire period of autonomous agent s behavior one can identify both neurons , that are active while performing most of the actions , and neurons , that are active only in a very limited number of states of the environment ( sometimes even in a single state ) .in our study we have explored possibility of evolutionary emergence of short - term memory in recurrent neural networks controlling agents behavior in multi - goal environments . on the first stage we analyzed overall dynamics of a model . as expected , performance of agents was worse when the number of goals in environment was smaller . on the other hand it turned out surprisingly that in stochastic environments agents with more flexible and stable behavioral policies evolve and able to accumulate more reward in comparison to agents in the same environment but without noise .this phenomenon is quite unusual for machine learning algorithms , since in the case of non - deterministic environment the performance is usually worse .detailed investigation of evolutionary dynamics revealed that the evolution of agents consisted of two stages : rapid emergence / development of new behavioral strategies and their propagation among the population .formation of new strategies was usually manifested as complex integration of few already existed .  on the level of individual behavior we found that evolution develops strategies , which are based on alternative actions .that happens due to acquisition of ability to operate with short - term memory and , therefore , to select actions taking into account previous history of behavior .evolution discovered two different neuronal mechanisms for implementation of alternative actions : first one is based on integration of sensory information and internal signal , which is reverberating through recurrent connections ; and the second is based on slow neurodynamical oscillatory processes . in fact , there is no need to recruit synaptic plasticity during the agent s `` life '' to have an effective use of short - term memory .it is important to notice that emergence of the ability to operate with short - term memory in our model occurs without any artificial prerequisites in the structure of an evolutionary algorithm .the next stage of presented study is introduction of learning during agent s `` life '' , since it makes significant contribution into development of adaptive behavior .the main mechanism of learning algorithm is detection on the neuronal level of problems for the whole organism situated in environment .possible approach to the construction of such algorithm is the use of research in the theoretical neuroscience .singh , s. , lewis , r. , barto , a.g .: where do rewards come from ? in : taatgen ,n.a . , van rijn , h. ( eds . ) , proceedings of the 31st annual meeting of the cognitive science society , pp .austin , tx : cognitive science society ( 2009 )     floreano , d. , mondana , f. : automatic creation of an autonomous agent : genetic evolution of a neural - network driven robot . in : cliff , d. , husbands , p. , meyer , j.- a . , wilson , s.w .proceedings of the 3rd international conference on simulation of adaptive behavior , pp . 421430 ( 1994 )    hochreiter , s. , bengio , y. , frasconi , p. , schmidhuber , j. : gradient flow in recurrent nets the difficulty of learning long - term dependencies .a field guide to dynamical recurrent neural networks , ieee press , pp .237243 ( 2001 )

Abstract: animals behave adaptively in the environment with multiply competing goals . understanding of the mechanisms underlying such goal - directed behavior remains a challenge for neuroscience as well for adaptive system research . to address this problem we developed an evolutionary model of adaptive behavior in the multigoal stochastic environment . proposed neuroevolutionary algorithm is based on neuron s duplication as a basic mechanism of agent s recurrent neural network development . results of simulation demonstrate that in the course of evolution agents acquire the ability to store the short - term memory and , therefore , use it in behavioral strategies with alternative actions . we found that evolution discovered two mechanisms for short - term memory . the first mechanism is integration of sensory signals and ongoing internal neural activity , resulting in emergence of cell groups specialized on alternative actions . and the second mechanism is slow neurodynamical processes that makes possible to code the previous behavioral choice .   behavior , neuroevolution , alternative behavior , recurrent neural networks , short - term memory


pesin s theorem prescribes the equality of the kolmogorov - sinai ( ks ) entropy with the sum of the positive lyapunov exponents of a dynamical system , i.e. . the former is a rate of entropy growth that is a metric invariant of the dynamical system , while the latter is the total asymptotic expansion rate present in the chaotic dynamics .this relation provides a deep connection between equilibrium statistical mechanics and chaos . in the limiting case of vanishing lyapunov exponent , as in the onset of chaos in one - dimensional nonlinear maps ,pesin s theorem is still valid but there are important underlying circumstances that are not expressed by the trivial identity . at variance with the chaotic region , at the transition to chaos phase spaceis no longer visited in an ergodic way and trajectories within the attractor show self - similar temporal structures , they preserve memory of their previous locations and do not have the mixing property of chaotic trajectories baldovin1 , mayoral1 . for chaotic and periodic attractorsthe sensitivity to initial conditions converges to an exponential function for large iteration times giving rise to the familiar positive or negative lyapunov exponents associated with them . at the transition to chaos exponential separation or merging of trajectoriesno longer occurs , but notably the sensitivity to initial conditions does not converge to any single - valued function and , on the contrary , displays fluctuations that grow indefinitely with time . for initial positions on the attractor the sensitivity develops a universal self - similar temporal structure and its envelope grows with iteration time as a power law .for unimodal maps it has been shown that this rich borderline condition accepts a description via a spectrum of generalized lyapunov exponents that matches a spectrum of generalized entropy growth rates obtained from a scalar deformation of the ordinary entropy functional , the so - called tsallis entropy expression .the entropy rate also differs from the ks entropy in that it is local in iteration time .that is , the entropy rate at time is determined only by the positions of trajectories at this time while the ks entropy considers all previous sets of positions . herewe present evidence that this general scenario of unimodal maps at the onset of chaos is captured by a special type of complex network .very recently , the horizontal visibility ( hv ) algorithm that transforms time series into networks has offered a view of chaos and its genesis in low - dimensional maps from an unusual perspective favorable for the discovery of novel features and new understanding .we focus here on networks generated by unimodal maps at their period - doubling accumulation points and characterize the fluctuations in connectivity as the network size grows .we show that the expansion of connectivity fluctuations admits the definition of a graph - theoretical lyapunov exponent .furthermore , the entropic functional that quantifies the amount of information generated by the expansion rate of trajectories in the original map appears to translate , in the light of the scaling properties of the resulting network , into a generalized entropy that surprisingly coincides with the spectrum of generalized graph - theoretical lyapunov exponents .this suggests that pesin - like identities valid at the onset of chaos could be found in complex networks that possess certain scaling properties .the rest of the letter is as follows : we first recall the hv algorithm that converts a time series into a network and focus on the so - called feigenbaum graphs as the subfamily of hv graphs generated by iterated nonlinear one - dimensional maps .we then expose the universal scale - invariant structure of the feigenbaum graphs that arise at the period - doubling accumulation points .we subsequently show how to define a graph - theoretical lyapunov exponent in this context . in agreement with the known dynamics of unimodal maps at the transitionto chaos the graph - theoretical lyapunov exponent vanishes , and proceed to define generalized exponents that take into account the subexponential expansion of connectivity fluctuations .we finally show that the feigenbaum graph that represents the onset of chaos admits a spectrum of generalized graph - theoretical exponents that coincide with a spectrum of deformed entropies .the horizontal visibility ( hv ) algorithm is a general method to convert time series data into a graph and is concisely stated as follows : assign a node to each datum of the time series of  real data , and then connect any pair of nodes , if their associated data fulfill the criterion , for all such that .the capability of the method as well as other variants to transfer properties of different types of time series into their resultant graphs has been demonstrated in recent works . when the series under study are the trajectories within the attractors generated by unimodal or circle maps the application of the hv algorithm yield subfamilies of visibility graphs , named feigenbaum and quasiperiodic graphs respectively , that render the known low - dimensional routes to chaos in a new setting . for illustrative purposes , in fig .[ grafos de feigenbaum1 ] we show a hierarchy of feigenbaum graphs obtained along the period - doubling bifurcation cascade of unimodal maps . at the accumulation point of the cascadethe trajectories within its non - chaotic multifractal attractor become aperiodic , and in analogy with the original feigenbaum treatment feigenbaum , the associated feigenbaum graphs evidence scaling properties that can be exploited by an appropriate graph - theoretical renormalization group transformation . because the order of visits of positions of periodic attractors or of bands of chaotic attractors in unimodal maps are universal , a relevant consequence of the hv criterion is that the resulting feigenbaum graph at the onset of chaos is the same for every unimodal map .that is , it is independent of the shape and nonlinearity of the map .this permits us to concentrate our study on a specific unimodal map , that for simplicity will be the logistic map , and claim generality of the results .+    consider the logistic map , , , at the period - doubling accumulation point and generate a trajectory within the attractor . since the position of the maximum of the map belongs to the attractor we can choose as the initial condition .( the position of the maximum of the unimodal map belongs to all the superstable periodic attractors or supercycles and the accumulation point of these attractors , the attractor at the transition to chaos , contains this position ) .a sample of the corresponding time series is shown in the top panel of figure [ log_k ] , whereas in the left panel of the same figure we represent in logarithmic scales the positions of the rescaled variable .a similar rescaling was proposed in previous works to put in evidence the striped intertwined self - similar structure of the trajectory positions as they are visited sequentially , and reflects the multifractal structure of the attractor . in the right panel of the same figure we plot , in log - log scales , a representation of the relevant variable , where is the degree of node in the feigenbaum graph associated to the original time series ( that is , ) .notice that the distinctive band pattern of the attractor is recovered , although in a simplified manner where the fine structure is replaced by single lines of constant degree .the hv algorithm transforms the multifractal attractor into a discrete set of connectivities , whose evolution mimics the intertwined fluctuations of the map .it is convenient to write the node index in the form , where and , so that running over the index with fixed selects subsequences of data or nodes placed along lines with fixed slope . for illustrative purposes , the order of visits to either the associated data or nodes in some of these subsequencesare explicitly shown in both panels of figure [ log_k ] , as well as in the upper panel .notice that along these subsequences , increase as a power law ( i.e. subexponentially ) , and are associated to data with increasing larger values ( upper panel ) .this is reminiscent of the data subsequences highlighted in the left panel , whose values also show a monotonic power law decay .+ from the direct observation of figure [ log_k ] it is clear that all trajectory positions within each band in the left panel ( attractor ) become nodes with the same degree in the right panel ( network ) .it is also worth noting at this point that there exists a degeneracy in the positions of trajectories that are initiated off , but close to , the attractor positions . in particular ,if an ensemble of uniformly - distributed initial positions is placed in a small interval around their trajectories expand while remaining uniformly - distributed at later iteration times ( see discussion below ) and the hv algorithm assigns to all of them the same feigenbaum graph. this degeneracy accounts for the universal feature that only a single feigenbaum graph represents the transition to chaos for all unimodal maps. it also determines the way in which fluctuations in trajectory separations translate into their respective degree fluctuations : see figure [ expansion ] , where we plot in logarithmic scales the distance of initially nearby trajectories for specific values of a time subsequence , and compare the expansion of this distance with the increment of the degree of the associated feigenbaum graph along the same node subsequence , as highlighted in the right panel of figure [ log_k ] .the properties of trajectory separation is thereby inherited by the graph and can be quantified through the increasing sequences of values for . +   close to , where , measured at a precise time sequence ( this path is also highlighted in the left panel of figure [ log_k ] ) . along this path trajectories evidence a subexponential expansion characterized by a power law separation .note that both trajectories generate the same feigenbaum graph , whose degree increases along this concrete path in the same vein as the trajectory separation : the network inherits the properties of trajectory expansion in the hierarchy of degrees ( see right panel of figure [ log_k ] ) .a similar behavior holds for trajectory separation along other paths.,scaledwidth=45.0% ]  let us focus now in the feigenbaum graph .the relabeling generates a one - to - one tiling of the natural numbers . the aforementioned constant degree and fixed - slope lines in the right panel of figure [ log_k ] are , respectively , ,  , fixed , and ,  , fixed ( for example , the first path highlighted in red corresponds to ) .the scaling property associated with the former case is that all horizontal lines with constant degree , fixed , can be overlapped into the bottom line , , via consecutive translations each consisting of a shift of in and a shift of in , that is , , or alternatively , .this scaling property has a parallel for the trajectory within the feigenbaum attractor where the shift of in is replaced by a shift of , where is the absolute value of the feigenbaum constant . the scaling property associated with the latter caseis seen via the collapse of all fixed - slope lines of data into a single sequence of values aligned along the first subsequence , , when the numbers of nodes for each of the other subsequences are rescaled consecutively by , .we recall that the standard lyapunov exponent accounts for the degree of exponential separation of nearby trajectories , such that two trajectories whose initial separation is evolve with time distancing from each other as , , where .the sensitivity to initial conditions is and the general definition of the lyapunov exponent is fluctuations about are permanently put out when , but , as mentioned , at the period - doubling accumulation points of unimodal maps and trajectories display successive subexponential separation and convergence for all .two possible extensions of the standard lyapunov exponent have been used when while the amplitude of the fluctuations grows as a power law , or , equivalently , expansion and contraction rates of trajectories are logarithmic in time : mori _ et _ _ al _ suggested the expression this expression captures subexponential fluctuation rates , this quantity is not well defined if we want to measure the magnitude of rates _ per unit time _ ,in which case we need a quantity that grows linearly in time .this is important if we want to relate the growth of fluctuations to some kind of entropic rates that derive from an extensive entropy expression . in this respect a better - suited alternative for capturing subexponential fluctuationsis to deform the logarithm in equation lyapunov1 by an amount that recovers the linear growth present for .this is ( is restored in the limit ) where the extent of deformation is such that while the expansion is subexponential , grows linearly with  . notice that we have eliminated the limit in the definition of the generalized lyapunov exponents since fluctuations are present for all values of and the objective is to characterize them via a spectrum of generalized exponents .we define now a connectivity expansion rate for the feigenbaum graph under study .since the graph is connected by construction ( all nodes have degree ) , the uncertainty is only associated with a rescaled degree . to keep notation simple we make use of this variable and drop the subindex from now on .the formal network analog of the sensitivity to initial conditions has as a natural definition , as suggested from figures [ log_k ] and [ expansion ] , where we are implicitly assuming that the expansion is always compared with the minimal one provided by nodes at positions ( i.e. with ) .accordingly , the standard network lyapunov exponent is defined as value of oscillates with ( see figure [ log_k ] ) but its bounds grow slower than , as , and therefore in network context , in parallel to the ordinary lyapunov exponent at the onset of chaos .the logarithmic growth of the bounds of is readily seen by writing as first approach to study the subexponential fluctuations is to proceed _ la mori _ and define the following graph - theoretical generalized exponent visualize the network growth paths , where is fixed to a constant value and , the expression above for is written as constant for all is obtained only when , otherwise the same value is reached for all other when .in general , a spectrum of mori - like generalized exponents is obtained by considering all paths such that .as indicated , the preceding generalization of the lyapunov exponent is not time extensive and it is therefore not useful if we ultimately wish to relate the network link fluctuations to an entropy growth rate . with this purpose in mindwe deform the ordinary logarithm in into by an amount such that depends linearly in , and define the associated generalized graph - theoretical lyapunov exponent as  is the lapse time between an initial condition where is fixed ( see figure [ log_k ] ) , and position .one obtains  and .eq . ( 8) can be corroborated via use of together with the identifications of the other quantities in it ( note that implies for which and is trivially undefined ) . according to eq .( 9 ) a spectrum of exponents is spanned by running over the values of , in parallel with the spectrum of generalized exponents previously found at the transition to chaos in unimodal maps , where the value of the parameter , in the feigenbaum graph is to be compared with ( where is a feigenbaum constant ) in the unimodal map for trajectories originating at the most compact region of the multifractal attractor that are seen to expand at prescribed times when the least compact region of the multifractal is visited .to complete our arguments we summon up the persistency property of trajectory distributions of unimodal maps at the period - doubling onset of chaos . that is , for a small interval of length with uniformly - distributed initial conditions around the extremum of a unimodal map ( _ e.g. _ for the logistic map we use ) , all trajectories behave similarly , remain uniformly - distributed at later times ,follow the concerted pattern shown in the left panel of fig .( see for details ) and expand subexponentially at prescribed time paths ( see figure [ expansion ] ) .studies of entropy growth associated with an initial distribution of positions with iteration time of several chaotic maps have established that a linear growth occurs during an intermediate stage in the evolution of the entropy , after an initial transient dependent on the initial distribution and before an asymptotic approach to a constant equilibrium value . at the period - doubling transition to chaosit was found that ( i ) there is no initial transient if the initial distribution is uniform and defined around a small interval of an attractor position , and ( ii ) the distribution remains uniform for an extended period of time due to the subexponential dynamics .we denote this distribution by where is the number of cells that tile the initial interval , and is the total length of the interval that contain the trajectories at time .since iteration - time dependence transforms into node - value dependence we inquire about how scales along the set of nodes with links , , , , , that is , while is defined in the map its properties are reflected in the network .after iterations , the loss of information associated with the fact that the same network is generated by all of the initial conditions is represented in the network context by the expansion of the initial interval into after .therefore we find that at time the network scaling properties prescribe a larger number of cells to tile the new interval .after normalization , we find that the dependence of reads    that is , the uniform distributions for the consecutive node - connectivity pairs ( ) and ( ) scale with the same factors noticed above when discussing the right panel of figure 2 .namely , when these pairs can be made equal by a shift of in and a shift of in . by extension of the argument the same expression holds for all other values of . since ordinary entropy associated with grows logarithmically with the number of nodes , = \log w_{j}\sim \log n$ ]. however , the -deformed entropy = \log _ { q}w_{j}=\frac{1}{1-q}\left [ w_{j}^{1-q}-1% \right ] ,  \label{q - entropy1}\]]where the amount of deformation of the logarithm has the same value as before , grows linearly with , as can be rewritten as , \label{numberw2}\]]with and . therefore ,if we define the entropy growth rate \equiv \frac{1}{n - m}s_{q}\left [ \pi ( n)\right ] \label{entropyrate2a}\]]we obtain = \lambda _ { q } ,  \label{entropyrate3}\]]a pesin - like identity at the onset of chaos ( effectively one identity for each subsequence of node numbers given each by a value of ) .in conclusion , the transcription into a network of a special class of time series , the trajectories associated with the attractor at the period - doubling transition to chaos of unimodal maps , via the hv algorithm has proved to be a valuable enterprise as it has led to the uncovering of a new property related to the pesin identity in nonlinear dynamics .the hv method leads to a self - similar network with a structure illustrated by the related networks of periods , , shown in fig .[ grafos de feigenbaum1 ] . under the hv algorithmmany nearby trajectory positions lead to the same network node and degree , all positions within one band in the left panel of fig .[ log_k ] lead to the same line of constant degree in the right panel of the figure .only when trajectory positions cross a gap between bands in the left panel the corresponding node increases its degree by two new links .also trajectories off the attractor but close to it transform into the same network structure .the degrees of the nodes span all even numbers , , and we have studied how these fluctuate as the number of nodes increases .the fluctuations of the degree capture the core behavior of the fluctuations of the sensitivity to initial conditions at the transition to chaos and they are universal for all unimodal maps .the graph - theoretical analogue of the sensitivity was identified as while the amplitude of the variations of grows logarithmically with the number of nodes .these deterministic fluctuations are described by a discrete spectrum of generalized graph - theoretical lyapunov exponents that appear to relate to an equivalent spectrum of generalized entropy growths , yielding a set of pesin - like identities .the definitions of these quantities involve a deformation of the ordinary logarithmic function that ensures their linear growth with the number of nodes .therefore the entropy expression involved is extensive and of the tsallis type with a precisely defined index .a salient feature of the application of the hv algorithm to dynamical systems time series is direct access to the degree distribution and therefore to the entropy associated with it .therefore this seems to be a good tool to discover , or , inversely , to construct , network entropy properties as studied here .the capability of the methodology employed in this study to reveal the occurrence of structural elements of this nature in real networked systems that grow in time appears to be viable .  * _ acknowledgements_. * we acknowledge financial support by the mec and comunidad de madrid ( spain ) through project nos .fis2009 - 13690 and s2009esp-1691 ( bl and ll ) , support from conacyt & dgapa ( papiit in100311)-unam ( mexican agencies ) ( ar ) , and comments of anonymous referees .

Abstract: we analyze the properties of networks obtained from the trajectories of unimodal maps at the transition to chaos via the horizontal visibility ( hv ) algorithm . we find that the network degrees fluctuate at all scales with amplitude that increases as the size of the network grows , and can be described by a spectrum of graph - theoretical generalized lyapunov exponents . we further define an entropy growth rate that describes the amount of information created along paths in network space , and find that such entropy growth rate coincides with the spectrum of generalized graph - theoretical exponents , constituting a set of pesin - like identities for the network .