it is well known that , in general , the act of measuring a quantum system necessarily disturb the system itself . a clear exemplification of this fact is given by joint ( or simultaneous ) measurements , _i.e. _ measurements in which one tries to simultaneously acquire information about two noncommuting observables from a single system . in this case , the possibility to sharply measure ( _ i.e. _ , to obtain measurement results not affected by more noise than the one intrinsically present into the system ) one observable is forbidden if a sharp measure of the other one is performed .nevertheless , information about both observables can be retrieved , paying the price of an extra noise in the statistics of the measurement results .an optimal joint measurement is the one introducing the minimum extra noise compatible with the laws of quantum mechanics .the first investigations in this issue can be traced back to the seminal paper by arthurs and kelly . there ,position and momentum observables have been considered and the minimum extra noise was found to be four times larger than the intrinsic noise related to the heisenberg uncertainty relation .notice that two points are crucial in their analysis : i ) the equality between the jointly inferred mean values and the mean values obtained if the measurements were performed non simultaneously is imposed ; ii ) the dependence on an _ a priori _ knowledge of the initial ensemble is needed , in order to adjust the relative sharpness of the two measurements .the usual implementation of an arthurs and kelly measurement concerns the field of quantum optics , in which two incompatible quadratures of the field are considered .typically , an heterodyne detection scheme ( or equivalently a double homodyne detection ) implements the joint measurement , however also a different strategy can be devised , in the case of fields described by a coherent state .one can in fact first generate two ( imperfect ) copies of the original state , via an optimal cloning process , and then measure the two quadratures one for each clones .both the strategies perform an optimal joint measurement .this fact , together with the quite general relationship between cloning and measurement , might lead to think that optimal cloning can be associated to optimal joint measurements .however , this is not the case in general . as an example , consider a qubit system and two pauli observables , for which again an optimal joint measurement involves an extra noise four times larger then the intrinsic one .an experimental implementation of such a joint measurement is reported in ref. , using an entangled two - photon state as information carrier from the very beginning .then , it is easy to demonstrate that optimal universal cloning for qubits does not allows to perform an optimal joint measurements of the two pauli observables ( see below ) .this is due to the fact that cloning machines are in general designed such to optimize the fidelity between input and output states , not such to optimize the joint measurement of a specific couple of noncommuting observables .it is the latter viewpoint that we will instead adopt in this work .more precisely , we will actually consider a further step , which can be understood in the spirit of state dependent cloning . recall that the latter cloning strategy , unlike the universal case , is specifically designed to act only on a particular class out of the whole hilbert space of the system ( _ e.g. _ , equatorial qubits ) . following an analogous strategy, we will design our process ( from now on called _ cloning of observables _ ) in order to clone and possibly jointly measure neither all possible observables nor two observables only , but rather the class generated by any linear combination of two observables .more in details , a cloning machine for the given set of observables is a device in which a signal qubit interacts with a probe qubit via a given unitary with the aim of reproducing the statistics of each observable on _ both _ the qubits at the output , independently of the signal qubit .the optimality of our cloner will be assessed by investigating the optimality of the joint measurement ( of the two observables generating the class ) , but the additional requirement of cloning the whole class of observables poses a clear distinction between a joint measurement and a cloning of observables .furthermore , unlike a joint measurement , our protocol is not focused only on the measurement process , which can actually not be performed .indeed , our approach is motivated , besides by the exploitation of a new aspect of the fundamental issue of quantum cloning , also in view of quantum communication purposes , in a repeater - like configuration .more specifically , one can address the transmission of information encoded , rather than in a set of states , in the statistics of a set of observables , independently of the quantum state at the input . in the followingwe will consider set of observables generated by two pauli operators , whereas for more general classes of observables we refer to ref . .the paper is organized as follows . in sec .[ s : clom ] we describe the general concept of cloning machine for observables , introducing some basic properties .we point out in sec .[ s : ncom ] the impossibility of designing a perfect cloning machines for a class of two noncommuting observables , analogously to the impossibility of perfectly cloning two nonorthogonal states .then , an approximate cloning machine for this case will be introduced . in sec .[ s : jointclob ] we assess the optimality of such a cloning machine and compare it to a joint measurement process .we will see that , even if the cloning machine is designed to act on the whole class generated by two noncommuting observables , the same performances of a joint measurement can be attained , when one restricts the attention to the only two observables generating the class .[ s : esco ] closes the paper with some remarks .we consider a device in which a signal qubit ( say , qubit `` 1 '' ) is prepared in the ( unknown ) state and then interacts , via a given unitary , with a probe qubit ( `` 2 '' ) prepared in the known state . for a given class of qubit observables where is a subset of the real axis and  ] is the mean value of the observable at the input and \quad { \overline{\rm x}}_2 \equiv \hbox{tr}_{12}\left [ r\ :  { \mathbb{i}}\otimes { { \rm x}}\right]\end{aligned}\ ] ] are the mean values of the same observable on the two output qubits ( see fig .[ f : machine ] ) .the density matrix describes the ( generally entangled ) state of the two qubits after the interaction , whereas denotes the identity operator .  :a signal qubit prepared in the unknown state interacts , via a given unitary , with a probe qubit prepared in the known state .the class of observables is cloned if a measurement of any on either the two qubits at the output gives the same statistics as it was measured on the input signal qubit , _ independently _ on the initial qubit preparation .,scaledwidth=70.0% ]  the above definition identifies the cloning of an observable with the cloning of its mean value .this is justified by the fact that for any single - qubit observable the cloning of the mean value is equivalent to the cloning of the whole statistics .in fact , any  ] . then , since is a cloning machine , we have \nonumber \\ & = { { \rm tr}_{1,2}}[u\,(w\otimes { \mathbb{i}})(\varrho \otimes { { \varrho_p}})(w^\dag \otimes { \mathbb{i}})\,u^\dag\ : ( w\otimes w)({{\rm y}}(j ) \otimes { \mathbb{i}})(w^\dag\otimes w^\dag ) ] \nonumber \\ & = { { \rm tr}_{1,2}}[v\,(\varrho \otimes { { \varrho_p}})\,v^\dag\,({{\rm y}}(j ) \otimes { \mathbb{i}})]={\overline{\rm y}}_1(j)\;.\end{aligned}\ ] ] the same argument holds for   another result which will be used in the following is the parameterization of a two - qubit transformation , which corresponds to a matrix , obtained by separating its local and entangling parts . a generic two - qubit gate matrix may be factorized as follows : \,l_1\end{aligned}\ ] ] where and the s are the pauli s matrices . the local transformations and belongs to the group , whereas accounts for the entangling part of the transformation . in our context , decomposition ( [ su4 ] ) , together with unitary covariance of cloning machines , allows to ignore the local transformations , which corresponds to a different state preparation of signal and probe qubits at the input . on the other hand , as we will see in the following , the degree of freedom offered by the local transformations will be exploited to design suitable cloning machines for noncommuting observables .in order to introduce cloning machines for a class of noncommuting observables let us consider the specific class .if a cloning machine existed , then the mean values as well as the statistics of any observable belonging to would be cloned at its output . as a consequence, one would jointly measure any two non - commuting observables belonging to ( _ e.g. _ , on the output signal and on the output probe ) without any added noise , thus violating the bounds imposed by quantum mechanics . generalizing this argument to any two parameter class of noncommuting observables ( _ i.e. _ , to any class , with generic non - commuting observables ) , we then conclude that a cloning machine for a generic two parameter class of noncommuting observables does not exist .this results resembles the no cloning theorem for states , and it can actually be seen as its counterpart for cloning of observables .furthermore , it permits a comparison among cloning machines for observables and for states .let us write the generic input signal as , where , is the bloch vector and is the identity operator . the state cloning counterpart of the statement above can then be obtained by considering the class of observables : if a cloning machine existed , then the components and of bloch vector would be cloned for any input signal .the same situation occurs in the case of a two parameter class generated by any pair of pauli operators . in other words, it is not possible to simultaneously copy a pair of components of the bloch vector of a generic state , even completely disregarding the third one .let us stress the fact that , even if a perfect cloning of observables could be performed ( for example , in the case of a class generated by commuting observables ) , correlations between the measurements would exist , otherwise state estimation limits would be overcame .now , a question arises on whether , analogously to state cloning , we may introduce the concept of approximate cloning machines , _i.e. _ cloning of observables involving added noise .indeed this can be done and optimal approximate cloning machines corresponding to minimum added noise may be found as well .an approximate cloning machine for the class of observables is defined as the triple such that and _ i.e. _ = g_1 \hbox{tr}_{12}\left [ r\ : { { \rm x}}\otimes { \mathbb{i}}\right ] = g_2 \hbox{tr}_{12}\left [ r\ : { \mathbb{i}}\otimes { { \rm x}}\right ] \label{appcl } \:,\end{aligned}\ ] ] for any .the quantities , are independent on the input state and are referred to as the noises added by the cloning process .let us begin by again considering the class . by using the decomposition of a generic matrix in eq .( [ su4 ] ) one may attempt to find an approximate cloning machine considering only the action of the entangling kernel .unfortunately , it can be shown that no , and exist which realize approximate cloning for .a further single - qubit transformation should be introduced after . in particular , the unitary flips the pauli matrices and ( _ i.e. _ , ) and permits the realization of an approximate cloning machine .indeed , the unitary realizes the approximate cloning machine with added noises in order to prove the cloning properties of let us start from the unitary , where is a generic entangling unitary of the form given in eq .( [ su4 ] ) .then , by imposing approximate cloning for any , one obtains the following system of equations :  [ nccm_aux ] &={{\sigma_1}}\\  g_1{{\rm tr}_2}[({\mathbb{i}}\otimes{{\varrho_p}})\,u_e^\dag\,({{\sigma_2}}\otimes{\mathbb{i}})\,u_e]&={{\sigma_2}}\\  g_2{{\rm tr}_2}[({\mathbb{i}}\otimes{{\varrho_p}})\,u_e^\dag\,({\mathbb{i}}\otimes{{\sigma_2}})\,u_e]&={{\sigma_1}}\\  g_2{{\rm tr}_2}[({\mathbb{i}}\otimes{{\varrho_p}})\,u_e^\dag\,({\mathbb{i}}\otimes{{\sigma_1}})\,u_e]&={{\sigma_2}}\:.\end{aligned}\ ] ]  system ( [ nccm_aux ] ) admits the solution , .e. _ , with free parameter with and .notice that other solutions for the s parameters may be found , which however give the same added noise as the one considered above .remarkably , similar cloning machines may be obtained for any class of observables generated by a pair of operators unitarily equivalent to and .in fact , given the two - parameter classes of noncommuting observables defined as , with , and generic unitary one has that an approximate cloning machine is given by the triple , with , with added noises and .the statement easily follows from the fact that is a cloning machine and from unitary covariance .similar results hold for any class of observables unitarily generated by any pair of ( noncommuting ) pauli operators .let us now consider if the approximate cloning machine for observables introduced above is optimal . in order to assess the quality and to define optimality of a triple we consider it as a tool to perform a joint measurements of noncommuting qubit observables .for example , consider the cloning machine and suppose to measure and on the two qubits at the output .we emphasize again that the cloning machine clones every observable belonging to , while we are now considering only the observables and which , in a sense , generate the class . the difference with a joint measurement specifically designed to measure only a couple of observables is evident . indeed_ a priori _ it is not clear at all if a cloning for observables , being in this sense far more general than a joint measurement , can reach the minimum disturbance necessary to perform an optimal joint measurement .nevertheless , this turns out to be the case , as we will see in the following .we have that the measured expectation values of and at the output are given by ( with ) , where the s are the input mean values .it follows that the measured uncertainties ( ) at the output are given by where denote the intrinsic uncertainties for the two quantities at the input . since for any pauli operators we have one may rewrites as a consequence, the measured uncertainty product is given by : since the arithmetic mean is bounded from below by the geometric mean we have with the equal sign iff , then it follows that if the initial signal is a minimum uncertainty state_i.e ._ , finally has that the measured uncertainty product is bounded by .notice that an optimal joint measurement corresponds to have . in our casethis is realized when is chosen such that . as already recalled in sec .[ intro ] , such an _ a priori _ knowledge of the initial state is a standard requirement of joint measurements .therefore , since adds the minimum amount of noise in a joint measurement performed on minimum uncertainty states we conclude that it is an optimal approximate cloning machine for the class under investigation .an optimal approximate cloning machine for the more general class may be also defined , using the concept of joint measurement for noncanonical observables .let us now consider the comparison with a joint measurement of and performed with the aid of an optimal universal cloning machine for states .it is easy to show that the best result in this case is given by , indicating that cloning of observables is more effective than cloning of states to perform joint measurements ( for the case of three observables see refs . ) . in fact, a symmetric cloning machine for states shrinks the whole bloch vector by a factor , whereas a cloning machine for observables shrinks the components and of only by a factor ( considering equal noise ) .notice that such a behavior is different from what happens in the case of continuous variables , for which the optimal covariant cloning of coherent states also provides the optimal joint measurements of two conjugated quadratures .this is due to the fact that coherent states are fully characterized by their complex amplitude , that is by the expectation values of two operators only , whereas the state of a qubit requires the knowledge of the three components of the bloch vector .  as a final remarkwe notice that if the requirement of universality is dropped , then cloning machines for states can be found that realize optimal approximate cloning of observables .for example , an approximate cloning machine for the two parameter class can be obtained by considering a phase covariant cloning for states .in order to clarify this point , let us recall that a phase covariant cloning machine for states of the form uses a probe in the state and performs the following transformation : where , in general , $ ] . if we now consider the class it is straightforward to show that eqs .( [ appcl ] ) are satisfied for any using the machine ( [ eqc ] ) , with the optimal added noises given by eq .( [ addn2 ] ) . this can be intuitively understood by considering that a phase covariant cloning machine extracts the optimal information about states lying on the equatorial plane of the bloch sphere , which in turn include the eigenstates of and .nevertheless , notice that there exist optimal approximate cloning machines for observables which do not coincide with phase covariant cloning for states . as an example , considering a cloning machine with added noise and one would still obtain optimal cloning of observables , even if such a machine could not perform a phase covariant cloning of states .this remark shows that , in general , one can not simply transfer the results for state cloning into the cloning of observables case .we have analyzed in details the cloning for classes of observables with focus on classes generated by two noncommuting observables .we have elucidated the relationship between cloning of observables and joint measurements and shown that even if the cloning machine is designed to act on the whole class generated by two noncommuting observables , the same optimal performances of a joint measurement can be attained .this work has been supported by miur through the project prin-2005024254 - 002 .af acknowledges the financial support from universit di milano under grant `` borse di perfezionamento allestero '' .99 e. arthurs , j. l. kelly , bell .j. * 44 * , 725 ( 1965 ) .n. j. cerf et al , phys . rev .lett . * 85 * , 1754 ( 2000 ) .n. gisin and s.  massar , phys .lett . * 79 * , 2153 ( 1997 ) ; d. bruss , a. ekert , and c. macchiavello , phys .lett . * 81 * , 2598 ( 1998 ) ; j. bae and a. acin , e print quant - ph/0603078 .e. arthurs , m. s. goodman , phys .lett . * 60 * , 2447 ( 1988 ) .h. p. yuen , phys .a*91 * , 101 ( 1982 ) .a. trifonov et al . , phys .rev . lett . *86 * , 4423 ( 2001 ) .v. buzek and m. hillery , phys .a * 54 * , 1844 ( 1996 ) . c. -s .niu and r. b. griffiths , phys .a * 60 * , 2764 ( 1999 ) .d. bruss _ et al _ , phys .a * 62 * , 012302 ( 2000 ) .a. ferraro , m. galbiati , and m.g.a .paris , j. phys .a : math . gen . * 39 * , l219 ( 2006 ) .see e.g. r. r. tucci , e print quant - ph/0507171 .notice that a cloning machine of the form with , where is an arbitrary unitary matrix also assures that . on the other hand , in order to have at the same time , the relation should hold .p. kienzler , int .* 37 * , 257 ( 1998 ) .d. m. appleby , int .j. th . phys . *39 * , 2231 ( 2000 ) .v. buzek et al , phys .a * 56 * , 3446 ( 1997 ) .g. m. dariano et al , j. opt .b * 3 * , 44 ( 2001 ) .

Abstract: cloning of observables , unlike standard cloning of states , aims at copying the information encoded in the statistics of a class of observables rather then on quantum states themselves . in such a process the emphasis is on the quantum operation ( evolution plus measurement ) necessary to retrieve the original information . we analyze , for qubit systems , the cloning of a class generated by two noncommuting observables , elucidating the relationship between such a process and joint measurements . this helps in establishing an optimality criterion for cloning of observables . we see that , even if the cloning machine is designed to act on the whole class generated by two noncommuting observables , the same optimal performances of a joint measurement can be attained . finally , the connection with state dependent cloning is enlightened .


this paper considers the infinite - time discounted optimal control problem for continuous - time input - affine polynomial dynamical systems subject to polynomial state constraints and box input constraints .this problem has a long history in both control and economics literature .various methods to tackle this problem have been developed , often based on the analysis of the associated hamilton - jacobi - bellman equation .  in this workwe take a different approach : we first lift the problem into an infinite - dimensional space of measures with continuous densities where this problem becomes convex ; in fact a linear program ( lp ) .this lifting is a _ tightening _ , i.e. , its optimal value is greater than or equal to the optimal value of the original problem , and under suitable technical conditions the two optimal values coincide .this infinite - dimensional lp is then further tightened by restricting the class of functions to polynomials of a prescribed degree and replacing nonnegativity constraints by sufficient sum - of - squares ( sos ) constraints .this leads to a hierarchy of semidefinite programming ( sdp ) tightenings of the original problem indexed by the degree of the polynomials .the solutions to the sdps yield immediately a sequence of _ rational _ controllers , and we prove that , under suitable technical assumptions , the value functions associated to these controllers converge _ from above _ to the value function of the original problem .we also describe how to obtain a sequence of polynomial approximations converging from above and from below to the value function associated to each rational controller .combined with existing techniques to obtain polynomial under approximations of the value function of the original problem ( adapted to our setting ) , this method can be viewed as a design tool providing a sequence of rational controllers asymptotically optimal in the original problem with explicit estimates of suboptimality in each step .the idea of lifting a nonlinear problem to an infinite - dimensional space dates back at least to the work of l. c. young  and subsequent works of warga  , vinter and lewis  , rubio  and many others , both in deterministic and stochastic settings .these works typically lift the original problem into the space of measures and this lifting is a _ relaxation _( i.e. , its optimal value is less than or equal to the optimal value of the original problem ) and under suitable conditions the two values coincide .more recently , this infinite - dimensional lifting was utilized numerically by _ relaxing _ the infinite - dimensional lp into a finite - dimensional sdp  or finite - dimensional lp  . whereas the lp relaxations are obtained by classical state- and control - space gridding , the sdp relaxations are obtained by optimizing over truncated moment sequences ( i.e. , involving only finitely many moments ) of the measures and imposing conditions _ necessary _ for these truncated moment sequences to be feasible in the infinite - dimensional lifted problem .these finite - dimensional relaxations provide _ lower bounds _ on the value function of the optimal control problem and seem to be difficult to use for control design with strong convergence guarantees ; a controller extraction from the relaxations is possible although no convergence ( e.g. ,  ) or only very weak convergence can be established ( e.g. ,  in the related context of region of attraction approximation ) .contrary to these works , in this approach we tighten the infinite - dimensional lp by optimizing over polynomial densities of the measures and imposing conditions _ sufficient _ for these densities to be feasible in the infinite - dimensional lifted problem , thereby obtaining upper bounds as opposed to lower bounds .crucially , to ensure that polynomial densities of arbitrarily low degrees exist for our problem ( and therefore the resulting sdp tightenings are feasible ) , we work with free initial and final measures and set up the cost function and constraints such that this additional freedom does not affect optimality .importantly , we do _ not _ assume that the state constraint set is control invariant , a requirement that is often imposed in the existing literature ( e.g. , ) but rarely met in practice .the presented approach bears some similarity with the density approach of  for global stabilization later extended to optimal control ( in a purely theoretical setting ) in  and recently generalized to optimal stabilization of a given invariant set in  ( providing both theoretical results and a practical computation method ) .however , contrary to  we consider the problem of optimal control , not stabilization and moreover we work under state constraints .contrary to  we work in continuous time , consider a more general problem ( optimal control , not optimal stabilization of a given set ) and our approach of finite - dimensional approximation is completely different in the sense that it is based purely on convex optimization and it does not rely on state - space discretization .moreover , and importantly , our approach comes with convergence guarantees .finally , let us mention that this work is inspired by  , where a converging sequence of upper bounds on _ static _ polynomial optimization problems was proposed , as opposed to a converging sequence of lower bounds as originally developed in  .we use to denote the space of all lebesgue measurable functions defined on a set and taking values in the set . if the space is not specified it is understood to be  .the spaces of integrable functions and essentially bounded functions are denoted by and , respectively .the spaces of continuous respectively -times continuously differentiable functions are denoted by respectively . by a ( borel ) measure we understand a countably additive mapping from ( borel ) sets to nonnegative real numbers .integration of a continuous function with respect to a measure on a set is denoted by or also when the variable and domain of integration are clear from the context .a probability measure is a measure with unit mass ( i.e. , ) .the support of a measure , defined as the smallest closed set whose complement has zero measure , is denoted by .the ring of all multivariate polynomials in a variable is denoted by  ] , and the vector space of -dimensional polynomial vectors is denoted by ^m ] is nonnegative on a set with  ] be strictly positive on .then for some .  combining with the stone - weierstrass theorem , as an immediate corollary we get :  [ cor : densnonneg ] let be nonnegative on and let for some and .then for every there exists and such that .corollary [ cor : densnonneg ] says that polynomials in are dense ( with respect to the norm ) in the space of continuous functions nonnegative on when we let tend to infinity .  in the rest of the text we use standard algebraic operations on sets .for instance if we write that ] .the inclusion of for a fixed is equivalent to the existence of a positive semidefinite matrix such that , where is a basis of {d/2} ] , ^n ] . ]input constraints  [ eq : xu ] ^m,\end{aligned}\ ] ]  where ^{n_g} ] and no stopping function . ] discounted infinite - horizon value function for all initial conditions for which the trajectories can be kept within the state constraint set forever using admissible controls , i.e. , for all in the maximum control invariant set associated to the dynamics ( [ eq : sys ] ) and the constraints ( [ eq : xu ] ) . to see the first claim , set for all . to see the second claim notice that with chosen as in ( [ eq : m ] ) , it is always beneficial to continue the time evolution whenever possible and therefore for all in the maximum controlled invariant set associated to ( [ eq : sys ] ) and ( [ eq : xu ] ) .[ rem : m ] a constant satisfying ( [ eq : m ] ) can be found either by analytically evaluating the supremum in ( [ eq : m ] ) or by using the techniques of  to find an upper bound .given a lipschitz continuous _controller and a stopping function ) ] denote the value function attained by in ( [ opt : main ] ) , i.e. , setting . by denote the value function , where ) ] feasible in ( [ opt : main ] ) such that and such that for every there exist a function and a scalar such that if and where denotes the solution to ( [ eq : sys ] ) controlled by .[ rem : l1 ] note that on by construction and therefore ( [ eq : valconv ] ) is equivalent to the convergence of to .assumption [ as : smoothcont ] says that the optimal control inputs and stopping functions for ocp ( [ opt : main ] ) can be well approximated by lipschitz continuous _controllers and measurable stopping functions such that the resulting densities of the discounted occupation measures are continuously differentiable and vanish near the boundary of .note that the existence of an optimal feedback controller , as well as whether it can be well approximated by lipschitz controllers , are subtle issues .similarly it is a subtle issue whether asymptotically optimal stopping functions can be found such that the associated densities in ( [ eq : smoothdens2 ] ) are continuously differentiable and vanish near the boundary of ( note , however , that the left hand side of ( [ eq : smoothdens2 ] ) can always be represented as for some nonnegative measure  ) .this problem is of rather technical nature and has been studied in the literature ( e.g. , ( * ? ? ?* section 1.4 ) or  ) , where affirmative results have been established in related settings .we do not undertake a study of this problem here and rely on assumption [ as : smoothcont ] , which is , for ease of reading , not stated in its most general form .for example , the functions do not need to be but only weakly differentiable and the integration on the left - hand side of ( [ eq : smoothdens2 ] ) can be weighted by a nonnegative function satisfying on and in .in addition , we conjecture that it is enough to require on and not necessarily on some neighborhood of ; this is in particular the case when is a box or a ball but we expect all the results of the paper to hold with a general semialgebraic set for which the defining functions satisfy ( [ eq : barg ] ) .  the main result of this paper is a hierarchy of sum - of - squares ( sos ) problems providing an explicit sequence of _ rational _ feedback controllers such that , under assumption [ as : smoothcont ] , ( [ eq : valconv ] ) holds with , i.e. , a sequence of asymptotically optimal rational controllers in the sense of the convergence of the associated value functions ( see remark [ rem : l1 ] ) .in this section we present an infinite - dimensional linear program ( lp ) in the space of continuous functions whose sum - of - squares ( sos ) approximations provide a sequence of _ rational _ controllers satisfying ( [ eq : valconv ] ) .this infinite - dimensional lp is closely related ( and in a weak sense equivalent ) to ocp ( [ opt : main ] ) ; the rationale behind the derivation of the lp and its relation to ocp ( [ opt : main ] ) is detailed in section [ sec : rationale ] .the infinite - dimensional lp reads the optimization in ( [ opt : dual_inf ] ) is over functions with .  the optimal value of ( [ opt : dual_inf ] )will be denoted by .the value attained in ( [ opt : dual_inf ] ) by any tuple of densities feasible in ( [ opt : dual_inf ] ) will be denoted by  .[ rem : nonunifweight ] note that we could have imposed for any polynomial nonnegative on .choosing a different has no impact on the asymptotic convergence of the value functions established in the rest of the paper as long as is strictly positive on  .it may , however , influence the speed of convergence in different subsets of  .in general we expect faster convergence where is large and slower convergence where it is small . choosing a non - constant allows to assign a different importance to different subsets of .the infinite - dimensional lp ( [ opt : dual_inf ] ) is then approximated by a hierarchy of sum - of - squares ( sos ) problems , which immediately translate to finite - dimensional semidefinite programs ( sdps ) .the sos approximation of degree of ( [ opt : dual_inf ] ) reads ^{3+m } } &  \int_x l_x(x ) \rho(x ) \ , dx + \sum_{i=1}^m\int_x l_{u_i}(x)\sigma_i(x)\ , dx + m\int_x \rho_t(x ) \ , dx \\ & \hspace{0.0 cm } \mathrm{s.t . }  & \rho_t  - \rho_0 + \beta\rho + \mathrm{div}(\rho f ) + \sum_{i = 1}^m\mathrm{div}(\sigma_i f_{u_i } ) = 0\\ & & -\rho \in q_d(x ) + g_i\mathbb{r}[x]_{d-\mathrm{deg}\ , g_i } + \bar{g}\mathbb{r}[x]_{d-\mathrm{deg}\ , \bar{g } }  & \hspace{-2 cm } i=1,\ldots , n_g\\ & & \rho_0 - 1 \in q_d(x )  \\ & & \bar{u}\rho - \sigma_i \in q_d(x ) + \bar{g}q_{d-\mathrm{deg}\ , \bar{g}}(x ) & \hspace{-2 cm } i=1,\ldots , m\\ & & \rho_t \in q_d(x)\\ & & \sigma_i\in q_d(x ) + \bar{g}q_{d-\mathrm{deg}\ , \bar{g}}(x ) ,  & \hspace{-2 cm } i=1,\ldots , m . \end{array}\ ] ] once a basis for ] , and family of trajectories of ( [ eq : sys ] ) generated by a lipschitz controller give rise to a triplet of measures defined by  [ eq : meas_def ]   the measure is called _ discounted occupation measure _ , the measure _ terminal measure _ and the measures , , _ control measures_. these measures satisfy the _ discounted liouville equation _ for all .this follows by direct computation ; see , e.g. , .notice also that , i.e. , is absolutely continuous with respect to with radon - nikodm derivative equal to .crucially , the converse statement is also true , although we have to go from stopping functions to stopping measures :  [ superposition][thm : meas_rep_crucial ] if measures , , and , , satisfy ( [ eq : discountliouville_general ] ) with , and and for some lipschitz , then there exists an ensemble of probability measures ( i.e. , measures with unit mass ) and an ensemble of trajectories of the system ( [ eq : sys ] ) controlled with such that for all and  [ eq : meas_traj ]   for all .  see appendix b.  theorem 2 says that any measures satisfying ( [ eq : discountliouville_general ] ) are generated by a superposition of the trajectories of the dynamical system , where the superposition is over the final time of the trajectories .note that there is a unique trajectory corresponding to each initial condition ( since the vector field is lipschitz ) but this unique trajectory can be stopped at multiple times ( in fact at a whole continuum of times ) allowing for superposition ; this superposition is captured by the stopping measures .for example , if the is a dirac measure at a given time , then there is no superposition ; if has a discrete distribution , then there is a superposition of finitely or countably many overlapping trajectories starting at stopped at different time instances ; if has a continuous distribution then there is a superposition of a continuum of overlapping trajectories starting from stopped at different time instances .  if in addition the measures , , satisfying the discounted liouville equation ( [ eq : discountliouville_general ] ) are absolutely continuous with respect to the lebesgue measure with densities , , such that on , then these densities satisfy with , follows directly by substituting , , and in ( [ eq : discountliouville_general ] ) and using integration by parts .equation ( [ eq : disc_liouville_functional ] ) holds almost everywhere in with a lipschitz controller , since lipschitz functions are differentiable almost everywhere and the integration by parts formula applies to them , and everywhere with .[ rem : termmeasure ] an important feature of the sos tightenings ( [ opt : dual_sos ] ) is that they are feasible for arbitrarily low degrees ( see remark [ rem : feasibility ] ) , which is crucial from a practical point of view and is not satisfied with other , more obvious , formulations ( e.g. , those not involving a stopping function in ( [ opt : main ] ) ) ; the reason for this is that , in the absence of a terminal measure ( i.e. , ) , the discounted liouville equation ( [ eq : disc_liouville_functional ] ) may not have a solution with a polynomial even though and the dynamics are polynomial .indeed , for example with , , , on  ] . moreoverthe state - control trajectories and are feasible in ( [ opt : main ] ) in the sense that and for all .let be feasible in ( [ opt : dual_inf ] ) and let denote the value attained by in ( [ opt : dual_inf ] ) . the equality constraint of ( [ opt : dual_inf ] )is exactly ( [ eq : disc_liouville_functional ] ) .since the constraint of ( [ opt : dual_inf ] ) implies on , equation ( [ eq : discountliouville_general ] ) holds with , , and , where , . by theorem [ thm : meas_rep_crucial ] ( setting in ( [ eq : meas_mu ] ) , in ( [ eq : meas_mut ] ) and in ( [ eq : meas_nu ] ) ) we obtain the result ( noticing that the constraints of ( [ opt : dual_inf ] ) imply that for all ) .[ cor : costub ] if is feasible in ( [ opt : dual_inf ] ) and , then if in addition the stopping measures in ( [ eq : costrep ] ) are equal to the dirac measures for some stopping function ) ] , then the integrals with respect to  in ( [ eq : costrep ] ) become evaluations at and hence ( [ eq : costexact ] ) holds .corollary [ cor : costub ] immediately implies that the problem ( [ opt : dual_inf ] ) ( and hence problem ( [ opt : dual_sos ] ) ) is a tightening of the original problem ( [ opt : main ] ) :  [ thm : tighten ] the optimal value of ( [ opt : dual_inf ] ) of satisfies   follows from corollary [ cor : costub ] since and .now we are in a position to prove the following crucial lemma linking problems ( [ opt : main ] ) and ( [ opt : dual_inf ] ) .[ lem : conv ] if and )\}_{k=1}^{\infty} ] from assumption [ as : smoothcont ] .by the first part of lemma [ lem : conv ] the sequence of associated densities generated by is feasible in ( [ opt : dual_inf ] ) and satisfies ( [ eq : opteq ] ) . by assumption [ as : smoothcont ] , and on ( since ) with .hence by lemma [ lem : polyapprox ] there exist polynomial densities , for some degrees such that    for all ( since ^m ] and  ] and ] , the objective functions of ( [ opt : val_u_ub ] ) and ( [ opt : val_u_lb ] )become linear in the coefficients of respectively in this basis .problems ( [ opt : val_u_ub ] ) and ( [ opt : val_u_lb ] ) are therefore convex sos problems and immediately translate to sdps ( see section [ sec : sosprog ] ) .[ thm : vu_ub_lb ] let and denote solutions to ( [ opt : val_u_ub ] ) and ( [ opt : val_u_lb ] ) of degree . then on and   see appendix a.  as a simple corollary we obtain a converging sequence of polynomial over - approximations to , the optimal value function of ( [ opt : main ] ) :  let denote the degree polynomial approximation from above to the value function associated to the rational controller obtained from ( [ opt : dual_sos ] ) using ( [ eq : ud_def ] ) .then on and   now we describe a hierarchy of lower bounds on : ,\ ;  p \in \mathbb{r}[x]_d^m } &  \int_x \underline{v } ( x)\,dx  \\ & \hspace{0.0 cm } \mathrm{s.t . }  & l_x - \beta \underline{v } + \nabla \underline{v } \cdot f + \bar{u } \sum_{i=1}^m p_i \in q_d(x ) \vspace{1 mm } \\ & & l_{u_i } + \nabla \underline{v } \cdot f_{u_i } - p_i \in q_d(x)\\ & & - p_i \in q_d(x)\\ & & m - \underline{v }  \in q_d(x ) + \bar{g}\mathbb{r}_{d-\mathrm{deg}\,\bar{g}}. \end{array}\ ] ]  if ] and and stage costs and .the discount factor was set to 1 ; the constant to .first we obtain a rational controller of degree six by solving ( [ opt : dual_inf ] ) with .the graph of the controller is shown in figure [ fig : nl_doubleinteg_cont ] .next we obtain a polynomial upper bound of degree 14 on the value function associated to by solving ( [ opt : val_u_ub ] ) with .to assess suboptimality of the controller we compare it with a lower bound on the optimal value function of the problem ( [ opt : main ] ) obtained by solving ( [ opt : val_opt_lb ] ) with .the graphs of the two value functions are plotted in figure [ fig : nl_doubleinteg_valfun ] .we see that the gap between the upper bound on and lower bound on is relatively small , verifying a good performance of the extracted controller .quantitatively , the average performance gap defined as is equal to .( 140,210 )  ( 120,0 )  ( 180,09) ( 290,10) ( 110,85) ( 240,-20 )  ( 140,210 )  ( 120,00 ) ( 195,10) ( 312,14)   in our second example we apply the proposed method to a population model governed by -dimensional controlled lotka - volterra equations where is the vector of ones and denotes the componentwise ( hadamard ) product .each component of the state represents the size of the population of species .the vector contains the intrinsic growth rates of each species and the matrix captures the interaction between the species .if , then species is harmful to species ( e.g. , competes for resources ) and if , then species is helpful to species ( e.g. , species feeds on species ) ; the diagonal components are normalized to one .the control inputs ^n ] represent , respectively , the inflow and outflow of new species from the outside . for our numerical examplewe select and model parameters which results in a system with four states and eight control inputs .the economic objective is to harvest species number one while ensuring that no species goes extinct .more specifically the cost function is and , where the vector is associated with the control input vector .therefore there is a reward for harvesting species number one and cost associated with both introduction and hunting of all other species , the cost of hunting being lower than the cost of introduction .the reason for choosing is in order to make the joint stage cost  ( [ eq : joinstagecost ] ) nonnegative ; this choice does not affect optimality since irrespective of the control input applied .the non - extinction constraint is expressed as with and .we choose and .we apply the coordinate transformation and solve solve obtain a rational controller of degree eight by solving ( [ opt : dual_sos ] ) .figure [ fig : lotkavolterra ] we shows plots for two different initial conditions , one with low population size of the first species and one with high .finally , we evaluate the suboptimality of the extracted controller using the polynomial lower bound on the optimal value function of degree 11 obtained from ( [ opt : val_opt_lb ] ) . using monte carlo simulation with 1000 samples of initial conditions drawn from a uniform distribution over the constraintset we obtain average cost of the extracted controller to be 0.89 whereas the lower bounds predicts average cost of 0.72 ; hence the extracted controller is no more than suboptimal ( modulo the statistical estimation error ) .note that we could also obtain a deterministic suboptimality estimate using the upper bound on the value function of the extracted controller obtained from ( [ opt : val_u_ub ] ) . in this case , however , the upper bound ( [ opt : val_u_ub ] ) is not informative .nevertheless , the monte carlo simulation along with the lower bound ( [ opt : val_opt_lb ] ) is a viable alternative in this case , since the extracted controller is simple and hence trajectories of the controlled system can be simulated rapidly .( 140,180 ) ( 0,90 ) ( 120,90 ) ( 240,90 ) ( 360,90 )  ( 0,0 ) ( 120,0 ) ( 240,0 ) ( 360,0 )this paper presented a method to obtain a sequence of rational controllers asymptotically optimal ( under suitable technical assumptions ) in a discounted optimal control problem and a method to explicitly estimate suboptimality of each controller .the rational controller of a given degree is obtained by solving a single sum - of - squares problem with no extraction step .the sos problem solved is feasible for any degree and therefore this method allows to trade off complexity of the controller against performance .the approach is based on lifting the nonconvex optimal control problem into an infinite dimensional space of measures with continuous densities , where this problem becomes linear . crucially , this problem is a tightening of the original problem , which follows immediately from the representation result for solutions of the discounted liouville s equation with a terminal measure ( theorem [ thm : meas_rep_crucial_thm ] ) .asymptotic optimality of the extracted controllers then follows by approximating the asymptotically optimal continuous densities ( guaranteed to exist by assumption [ as : smoothcont ] ) with polynomial densities in such a way that these densities correspond to the densities of the dynamical system ( this is the essence of the proof of theorem [ thm : main ] ) .this appendix contains the proof of theorem [ thm : vu_ub_lb ] ; we use the same notation as in section [ sec : valapprox ] .the inequalities follow from gronwall s lemma by noticing that the constraints of ( [ opt : val_u_ub ] ) and ( [ opt : val_u_lb ] ) imply that  on and , on .we detail the argument for the inequality , the inequality being similar .given the inequality ( [ eq : ubgron ] ) implies that ,\ ] ] and therefore by gronwall s lemma \ , ds\ ] ] and hence \ , ds\ ] ] for all  ] is the first exit time of .next we observe that , the value function associated to , is equal to , & \tau = \infty \\me^{-\beta \tau } + \int_0^\tau e^{-\beta s}\big [ l_x(x(s\ ! \mid \ !x_0 ) ) + \sum_{i=1}^m l_{u_i}(x(s\ ! \mid \ !x_0))u_i(x(s\ ! \mid \ !x_0 ) ) \big ] ds , & \tau < \infty . \end{cases}\ ] ] in view of ( [ eq : grownaux ] ) , we conclude that if since is polynomial and hence bounded on ( and hence ) ; and we conclude that if since and on .convergence of the upper and lower bounds ( [ eq : convublb ] ) follows from theorem [ thm : meas_rep_crucial_thm ] using infinite - dimensional lp duality and standard results on the convergence of moment relaxations .the proof is similar to the proof of theorem 5 in  or theorem 3.6 in  and therefore we only outline it .the hierarchy of sos programming problems ( [ opt : val_u_ub ] ) and ( [ opt : val_u_lb ] ) is dual to the hierarchy of moment relaxations of an infinite - dimensional lp in the cone of nonnegative measures whose dual is an infinite - dimensional lp in and feasible solutions of this dual provide upper or lower bounds on .crucial to applying infinite - dimensional duality results ( e.g. , ( * ? ? ?* theorem 3.10 ) ) is the boundedness of measures satisfying the discounted liouville equation ( [ eq : discountliouville_general ] ) with and , where is the restriction of the lebesgue measure to . plugging in ( [ eq : discountliouville_general ] ) we have . since and conclude that and are indeed bounded , which implies that is also bounded for . equally importantis the absence of duality gap between the finite - dimensional moment relaxations and sos tightenings ( which are both sdp problems ) ; this follows immediately from the presence of the constraint among the constraints describing , which implies the boundedness of the truncated moment sequences feasible in the moment relaxations .the absence of duality gap then follows from ( * ? ? ?* lemma 2 ) . appendix presents a proof of theorem [ thm : meas_rep_crucial ] . we will prove a slightly more general version of the result from which theorem [ thm : meas_rep_crucial ] immediately follows :  [ thm : meas_rep_crucial_thm ] let be globally lipschitz and let the nonnegative measures , , on satisfy for all .then there exists an ensemble of probability measures with  ] such that on and on . then the left hand side of ( [ eq : derivaux ] ) is greater than or equal to whereas the right hand side is less than or equal to . since and we arrive at a contradiction with ( [ eq : derivaux ] ) by picking a sufficiently large .therefore there exists a such that ( i.e. , and are on the same trace of the flow associated to ) .now we prove that .since and are on the same trace we have using again if we get  consider the set since and are on the same trace ( and follows ) there exists , , such that on and elsewhere ( e.g. , let with appropriate mollification ) . with this choice of equation ( [ eq : asmall ] ) gives and therefore since the integral is strictly positive .this proves the first two claims .to finish we observe that any solution to ( [ eq : trans_disc ] ) satisfies therefore using ( [ eq : v_rep ] ) we get since we conclude that i.e. , is indeed generated by trajectories of ( in this case by two trajectories , both starting at , one stopping at , the other one continuing to infinity with weights given by the ratio of masses of and ) .that is the measure is given by as expected .next we treat the case where for some and . using the same argument as in the previous casewe can show that for all and that the condition holds with being the times to reach from .then we have    in the same way as before we can show that the support of must be on the trace . then we can define the measure by and show that it has to satisfy the condition .next , using the fact that the mapping is invertible , we obtain the conclusion of the theorem then holds with defined by i_a(\infty ) , \quad a\subset [ 0,\infty].\ ] ]     l. ambrosio , g. crippa .existence , uniqueness , stability and differentiability properties of the flow associated to weakly differentiable vector fields . transport equations and multi - d hyperbolic conservation laws , lecture notes of the unione matematica italiana , vol . 5 , springer , 2008 .m. korda , d. henrion , c. n. jones .controller design and region of attraction estimation for nonlinear dynamical systems .ifac symp .nonlinear control systems , toulouse , france , 2013 . extended version : ` arxiv:1310.2213 ` , 2013 .

Abstract: this work considers the infinite - time discounted optimal control problem for continuous time input - affine polynomial dynamical systems subject to polynomial state and box input constraints . we propose a sequence of sum - of - squares ( sos ) approximations of this problem obtained by first lifting the original problem into the space of measures with continuous densities and then restricting these densities to polynomials . these approximations are tightenings , rather than relaxations , of the original problem and provide a sequence of rational controllers with value functions associated to these controllers converging ( under some technical assumptions ) to the value function of the original problem . in addition , we describe a method to obtain polynomial approximations from above and from below to the value function of the extracted rational controllers , and a method to obtain approximations from below to the optimal value function of the original problem , thereby obtaining a sequence of asymptotically optimal rational controllers with explicit estimates of suboptimality . numerical examples demonstrate the approach .  optimal control , nonlinear control , sum - of - squares , semidefinite programming , occupation measures , value function approximation


over the last few years , model checking based on abstraction - refinement has become a popular technique for the analysis of software . in particularthe abstraction technique of choice is a property preserving over - approximation called predicate abstraction  and the refinement removes spurious behavior based on automatically analyzing abstract counter - examples .this approach is often referred to as cegar ( counter - example guided automated refinement ) and forms the basis of some of the most popular software model checkers .furthermore , a strength of model checking is its ability to automate the detection of subtle errors and to produce traces that exhibit those errors .however , over - approximation based abstraction techniques are not particularly well suited for this , since the detected defects may be spurious due to the over - approximation hence the need for refinement .we propose an alternative approach based on refinement of under - approximations , which effectively preserves the defect detection ability of model checking in the presence of aggressive abstractions .the technique uses a combination of ( explicit state ) model checking , predicate abstraction and automated refinement to efficiently analyze increasing portions of the feasible behavior of a system . at each step , either an error is found , we are guaranteed no error exists , or the abstraction is refined . more precisely , the proposed model checking technique traverses the concrete transitions of the system and for each explored concrete state , it stores an abstract version of the state .the abstract state , computed by predicate abstraction , is used to determine whether the model checker s search should continue or backtrack ( if the abstract state has been visited before ) .this effectively explores an under - approximation of the feasible behavior of the analyzed system .hence all counter - examples to safety properties are preserved .refinement uses weakest precondition calculations to check , with the help of a theorem prover , whether the abstraction introduces any loss of precision with respect to each explored transition .if there is no loss of precision due to abstraction ( we say that the abstraction is _ exact _ ) the search stops and we conclude that the property holds .otherwise , the results from the failed checks are used to refine the abstraction and the whole verification process is repeated anew . in general , the iterative refinement may not terminate .however , if a finite bisimulation quotient  exists for the system under analysis , then the proposed approach is guaranteed to eventually explore a finite structure that is bisimilar to the original system .the technique can also be used in a lightweight manner , without a theorem prover , i.e. the refinement guided by the exactness checks is replaced with refinement based on syntactic substitutions or heuristic refinement .the proposed technique can be used for systematic testing , as it examines increasing portions of the system under analysis .in fact , our method extends existing approaches to testing that use abstraction mappings  , by adding support for automated abstraction refinement .our approach can be contrasted with the work on predicate abstraction for modal transition systems  , used in the verification and refutation of branching time temporal logic properties . an abstract model for such logicsdistinguishes between _ may _ transitions , which over - approximate transitions of the concrete model , and _ must _ transitions , which under - approximate the concrete transitions ( see also  ) . as we show in the next section ( and we discuss in more detail in section [ sec : discussions ] ) , the technique presented here explores and generates a structure which is more precise ( contains more feasible behaviors ) than the model defined by the _ must _ transitions , for the same abstraction predicates .the reason is that the model checker explores transitions that correspond not only to _ must _ transitions , but also to _ may _ transitions that are feasible .moreover , unlike  and over - approximation based abstraction techniques , the under - approximation and refinement approach does not require the a priori construction of the abstract transition relation , which involves exponentially many theorem prover calls ( in the number of predicates ) , regardless of the size of ( the reachable portion of ) the analyzed system . in our case, the model checker executes concrete transitions and a theorem prover is only used during refinement , to determine whether the abstraction is exact with respect to each executed transition .every such calculation makes at most two theorem prover calls , and it involves only the _reachable _ state space of the system under analysis .another difference with previous abstraction techniques is that the refinement process is not guided by the spurious counter - examples , since no spurious behavior is explored . instead, the refinement is guided by the failed exactness checks for the explored transitions .to the best of our knowledge , the presented approach is the first predicate abstraction based analysis which focuses on automated refinement of under - approximations with the goal of efficient error detection .we illustrate the application of the approach for checking safety properties in concurrent programs .the rest of the paper is organized as follows .section [ sec : example ] shows an example illustrating our approach .section [ sec : background ] gives background information .section [ sec : algorithm ] describes the main algorithm for performing concrete model checking with abstract matching and refinement .section [ sec : correctnesstermination ] discusses correctness and termination ; section [ sec : discussions ] discusses other interesting properties for the algorithm .section [ sec : extensions ] proposes extensions to the algorithm .section [ sec : applications ] illustrates applications of the approach , section [ sec : related - work ] discusses related work , and section [ sec : conclusions ] concludes the paper .= 4.9 in  the example in figure [ fig : comparison ] illustrates some of the main characteristics of our approach .figure [ fig : comparison ] * ( a ) * shows the state space of a concrete system that has only one variable ; states are labelled with the program counter ( e.g. , , , ) and the concrete value of  . figure [ fig : comparison ] * ( b ) * shows the abstract system induced by the _ may _ transitions for predicate .figure [ fig : comparison ] * ( c ) * shows the abstract system induced by the _ must _ transitions for predicate .figure [ fig : comparison ] * ( d ) * shows the state space explored using our proposed approach , for an abstraction specified by predicate .dotted circles denote the abstract states which are stored , and used for matching , during the concrete execution of the system .the approach explores only the _ feasible _ behavior of the concrete system , following transitions that correspond to both _ may _ and _ must _ transitions , but it might miss behavior due to abstract matching .for example , state is not explored , assuming a breadth - first search , since was matched with  both have the same program counter and both satisfy . notice that , with respect to reachable states , the produced structure is a better under - approximation ( it `` covers '' more states ) than the _ must _ abstraction .figure [ fig : comparison ] * ( e ) * illustrates concrete execution with abstract matching , after a refinement step , which introduced a new predicate . the resulting structure is an exact abstraction of the concrete system .to make the presentation simple , we use as a specification language a guarded commands language over integer variables . most of the results extend directly to more sophisticated programming languages .let be a finite set of integer variables .expressions over are defined using standard boolean ( ) and binary ( ) operations .  a _ model _is a tuple . is a finite set of transitions , where , is a guard and are assignments to the variables represented by tuple .  throughout the paper ,we write concurrent assignments as sequences , to improve readability .the semantics of program models uses transition systems .a _ transition system _ over a finite set of atomic propositions is a tuple where is a ( possibly infinite ) set of states , is a finite set of deterministic transition relations : , is an initial state , and is a labeling function .state is _ reachable _ if there exists a sequence of zero or more transitions from the initial state such that ( denoted ) .the set of _ reachable labelings _ is .the notation means that there is no transition from the state .[ def : concretesemantics ] the _ concrete semantics _ of model is transition system  , , over , where :  * , i.e. states are valuations of variables , * ; the semantics of guards ( boolean expressions ) and updates is as usual ; guards are functions , written as ; updates are functions , * is the zero valuation ( ) , * .let be a predicate representing a set of states .then the _ strongest postcondition _ of with respect to transition is ; defines the successors by transition of the states characterized by .the _ weakest precondition _ of with respect to transition is ; characterizes the largest set of states whose successors by transition satisfy . for guarded commands ,the weakest precondition can be expressed as ) ] .therefore is equivalent to ) ] , equivalent to )$ ] , is valid for each such that and ( 2 )  is valid for each such that .checking the validity for these formulas is in general undecidable .as is customary , if the theorem prover can not decide the validity of a formula , we assume that it is not valid . this may cause some unnecessary refinement , but it keeps the correctness of the approach . if the abstraction can not be proved to be exact with respect to some transition , then the new predicates from the failed formula are added to the set of abstraction predicates . intuitively , these predicates will be useful for proving exactness in the next iteration .in this section we discuss the main properties of the iterative refinement algorithm .we first state the main theorems , after which we give the technical lemmas and proofs ( the reader may wish to skip this technical material on the first reading ) .we first show that , if the iterative algorithm terminates then the result is correct and moreover , if the error state is unreachable , the output structure is bisimilar to the system under analysis :  ( correctness ) [ theorem : correctness ] if  terminates then :  * if it returns a counter - example , then it is a real error , * if it returns ` unreachable ' , then the error state is indeed unreachable in and moreover the computed structure is bisimilar to .  in general , the proposed algorithm might not terminate ( the reachability problem for our modeling language is undecidable ) .however , the algorithm is guaranteed to eventually find all the reachable labelings ( including all the reachable errors ) of the concrete program , although it might not be able to detect that ( to decide termination ) . moreover ,if the ( reachable part of the ) system under analysis has a finite bisimulation quotient , then the algorithm eventually produces a finite bisimilar structure .( termination ) [ th : termination ] let the use breadth - first search order and let , , be a sequence of transition systems generated during iterative refinement performed by .then  * there exists such that , * if the reachable part of the bisimulation quotient is finite , then there exists such that .note that a consequence of this theorem is that if an error is reachable it is eventually reported by our algorithm . also note that for the second part of the theorem , we do not require that both the reachable and unreachable parts of the system have a finite bisimulation quotient , but only the reachable part needs to be finite ( of course , if both the reachable and unreachable parts are finite , then it follows that the reachable part is also finite ; the converse is not true ) .here we provide several technical lemmas and the proofs for the two main theorems .we use the following notation : a state is _ visited _ during the search if it is inserted into ; a state is _ considered _ during the search if it is generated as a successor of some state in the loop ; a state is _ matched _ to a state if the check fails because and was visited before .we say that transition is exact if is exact with respect to it .note that sometimes we let denote just the structure computed by the algorithm and not the tuple .also note that starts with as the initial set of predicates .for the proofs , we need to refine the definition of bisimulation .  a symmetric relation is a -bisimulation relation iff :  * for all * if then there exists -bisimulation relation such that for all   the -bisimulation is the largest -bisimulation relation , denoted .note that the bisimulation is a -bisimulation relation for every .we first show that the reachable labelings computed by the iterative algorithm is indeed an _ under - approximation _ of the reachable labelings of the program under analysis ( lemmas [ lemma : must - sequences ] and [ lemma : must - search - concrete ] ) .therefore , all the reported counter - examples correspond to real errors .we then show that when reports unreachable ( i.e. when the set of predicates returned for the current iteration is equal to the set of predicates from the previous iteration ) then the computed structure is bisimilar to ( lemmas [ lemma : bisim - ok ] and [ lemma : check - correctness ] ) .[ lemma : must - sequences ] if a state is reachable in via exact transitions with respect to , then there exists such that is visited during the and .  by induction with respect to the number of exact transitions from the initial state necessary for reaching the state .basic step ( ) is trivial . for the induction step , suppose that state is reachable via sequence of exact transitions : . by the induction hypothesis thereexists such that is visited and .because the abstraction is exact with respect to , there must be such that and .this successor is considered during the visit of .there are two cases to be analyzed .  1 . is added to and later visited , 2 . is matched to a previously visited state such that .  in both caseswe get that some state with the same abstract counterpart as is visited during the search .  [ lemma : must - search - concrete ] .it is easy to verify that the following is an invariant of the search : is a subset of reachable states in .the lemma follows .  [ lemma : bisim - ok ] let .if for all reachable states it holds that , then .consider relation defined as : iff or is matched to .then is a bisimulation relation between and .  [ lemma : check - correctness ] let .if , then .  due to lemma[ lemma : bisim - ok ] it is sufficient to show that if then induces a bisimulation relation on the reachable part of the transition system  . we first show that every reachable state in is reached by exact transitions .we proceed on induction by the number of transitions from the initial state to .basic step ( ) is trivial . for the induction step , suppose that state is reachable via a sequence of exact transitions of length . by lemma [ lemma : must - sequences ]some state such that is visited during the search . during the visit of the state we check exactness of the abstraction( see section [ sec : check - exact - abstr ] ) .since it follows that the abstraction is exact for , i.e. , iff and for every outgoing transition and there exists such that and . since is deterministic , it follows that is the only successor of by transition and transition is also exact .moreover , it satisfies the same criterion for bisimulation , i.e. for all such that there exists such that and .  the first claim follows from the fact that produces an under - approximation ( lemma [ lemma : must - search - concrete ] ) .the second claim follows from lemma [ lemma : check - correctness ] .   in order to prove theorem 2 , we study sequences of transition systems generated during .we assume that uses breadth - first search order .the basic idea of the proof is that any two states that are in different bisimulation classes ( ) are eventually distinguished by the abstraction function , i.e. such that ( lemma [ lemma : states - distinguieshed ] ). moreover , each bisimulation class of is eventually visited by ( lemma [ lemma : each - bisim - class - reached ] ) and the finite set of reachable labelings emerges ( lemmas [ lemma : all - labelings ] and [ lemma : finite - quotient ] ) .[ lemma : states - distinguieshed ] let be a sequence of transition systems generated during an infinite run of and there exists infinitely many such that .if and then there exists such that for all .  by induction with respect to where is the smallest number such that .basic step : for it means that and therefore .induction step : let be such that and .since is visited in infinitely many iterations of , is considered in infinitely many iteration of and therefore one of the following must hold :  1 .. then we can apply induction hypothesis , i.e. there exits such that for all .state is matched to some state in infinitely many runs of .since we use breadth - first order , there are only finitely many states to which it can be matched ( with breadth - first search order the state can be matched only to states with lower or equal distance from the initial state ) . therefore , there exists a state such that is matched to in infinitely many runs of , this means that for all . from the induction hypothesiswe get that and hence . moreover , from the induction hypothesis we get that there exists such that for all . therefore for all .  in both caseswe get that there exists such that is not exact with respect to , therefore will be included in and therefore .  [ lemma : each - bisim - class - reached ] for each reachable bisimulation class of there exists a state such that is visited by infinitely often .by induction with respect to the length of the shortest path by which some state from is reachable .basic step is obvious .induction step : let state from be reachable via path . by induction hypothesis some state is reached during the refinement search infinitely often .consider state such that .it holds that and from lemma [ lemma : states - distinguieshed ] we get that is visited infinitely often .  [ lemma : all - labelings ] let be a sequence of transition systems generated during an infinite run of .there exists such that .  for each choose some bisimulation class such that . in this waywe obtain a finite set of bisimulation classes which covers all labels in .note that is finite because is finite .now we show that there exists an iteration in which at least one state from each of these classes is visited .this is done similarly to the proof of lemma [ lemma : each - bisim - class - reached ] .  [ lemma : finite - quotient ] let be a sequence of transition systems generated during an infinite run of . if the reachable part of the bisimulation quotient is finite , then there exists such that .  by contradiction .suppose that . from lemma[ lemma : bisim - ok ] we get that there exists reachable such that and .we show ( similarly to the proof of lemma [ lemma : must - sequences ] ) that there exists such which is visited infinitely often . from lemma [ lemma : states - distinguieshed ]we get that eventually which is the contradiction .  this theorem is a direct consequence of lemmas [ lemma : all - labelings ] and [ lemma : finite - quotient ] .  discussed correctness and termination , we now turn to other interesting properties of the algorithm .we should note that the proposed iterative algorithm is not guaranteed to terminate even for a finite state program .this situation is illustrated by the example from figure [ fig : incomplete - refinement ] ; and are initialized to zero .the property that we check is that `` pc=1 '' is unreachable .although the program is finite state ( and therefore the problem can be easily solved with classical explicit model checking ) , it is quite difficult to solve using abstraction refinement techniques .the iterative algorithm does not terminate on this example : it keeps adding predicates . note that , in accordance with theorem [ th : termination ] , it eventually produces a bisimilar structure .however , the algorithm is not able to detect termination , and it keeps refining indefinitely .the reason is that the algorithm keeps adding predicates that refine the unreachable part of the system under analysis .also note that the same problem occurs with over - approximation based abstraction techniques that use refinement based on weakest precondition calculations  .those techniques introduce the same predicates .moreover , unlike our technique , they will keep generating _ spurious _ counter - examples . for this example no may / must abstraction based on predicates and refinement with weakest precondition calculations can produce a structure that is bisimilar to the concrete system ( the concrete system is rather trivial it has only one state ) .this example also illustrates another difference between the method presented here and over - approximation based predicate abstraction with refinement , in particular  . if the analyzed system has a _ reachable _ finite bisimulation quotient then our algorithmis guaranteed to find it ( see theorem [ th : termination ] and lemma [ lemma : finite - quotient ] ) .in contrast , the method in  will fail to compute a finite state abstraction for the example ; this result seems to contradict the bisimulation completeness claim ( theorem 3 ) from  .we conjecture that the method in  is not guaranteed to compute a finite state abstraction unless both the _ reachable and unreachable _ quotient is finite .  to solve the problem of non - termination for finite state systems , we propose to use the following heuristic . if there is a transition for which we can not prove that the abstraction is exact in several subsequent iterations of the algorithm , then we add predicates describing the concrete state ; i.e. in the example from figure [ fig : incomplete - refinement ] we would add predicates and .the abstraction eventually becomes exact with respect to each transition . andsince the number of reachable transitions is finite , the algorithm must terminate .if the reachable part of is finite state then the modified algorithm terminates .   the search order used in (depth - first or breadth - first ) influences the size of the generated structure , the newly computed predicates , and even the number of iterations of the main algorithm . if there are two states and such that but then , depending on whether or is visited first , different parts of the transition system will be explored . for our implementation ,we use breadth - first search order .  [ cols="^,^,^ " , ]   this is another well - known protocol for mutual exclusion .the protocol is similar to the ticket protocol ( the ticket protocol requires special hardware instruction like fetch - and - add , whereas the bakery protocol is applicable without any special instructions ) .the model has 10 variables .the property of interest is again mutual exclusion .the state space is infinite with a finite bisimulation quotient .the property can be proved by the algorithm in three iterations , using 31 predicates .for this example , we seeded an error by changing a guard into which creates a nontrivial error in the protocol .the tool can find the error in the first iteration .this is a `` classic '' example analyzed using predicate abstraction  .the property of interest is the correct use of a lock .our tool can prove that the property holds after one iteration ( using just the predicates from guards ) : the algorithm explores 10 concrete states , 9 abstract states and casts 3 queries to the theorem prover .for an erroneous version of the driver , the tool finds an error in the first iteration as well .these preliminary experiments show the merits of our approach .the approach proves to be effective in computing finite bisimilar structures of non - trivial infinite state systems and in finding errors using under - approximation based predicate abstraction . of course , much more experimentation is necessary to really assess the practical benefits of the proposed technique and a lot more engineering is required to apply it to real programming languages .extensions for handling complex features such as pointers , arrays and procedures , are tedious but conceptually not very hard .we also note that in some cases ( e.g. ticket2 , ticket3 and rax ) the number of explored concrete and abstract states stays the same after the first iteration ; however our algorithm needs more than two iterations to discover all the necessary abstraction predicates , according to the exactness criteria that we defined .the results suggest that it is possible to relax these criteria and still provide a guarantee that the relevant state space of the analyzed program has been explored .we leave this topic for future work .we should mention that the application of over - approximation based predicate abstraction to a java version of rax is described in detail in  . in that work ,four different predicates were used to produce an abstract model that is bisimilar to the original program .in contrast , the work presented here allowed more aggressive abstraction to recover feasible counter - examples .our technique explores transitions that are guaranteed to be feasible .in contrast , the over - approximation based techniques such as the ones from  may also explore transitions that are spurious and therefore could require additional refinement before reporting a real counter - example .as mentioned , over - approximation based abstraction techniques involve exponentially many theorem prover queries ( in the number of predicates ) , at each iteration .this computation is performed regardless of the size of ( the reachable portion of ) the analyzed system . in our case ,theorem prover queries are only performed during refinement and they involve only the _reachable _ state space of the system under analysis . on the other hand ,over - approximation based techniques are good at proving properties ( as they compute abstractions that are coarser than the bisimulation quotient but sufficient to prove safety properties ) .we believe however that the technique presented here is _ complementary _ to over - approximation abstractions and it should _ combined _ ( rather than compared ) with such techniques . our technique could be used for discovering efficiently feasible counter - examples in the space bounded by the abstraction predicates ( that are used in the over - approximation analysis ) . in the future, we plan to study more the strengths and weaknesses of each approach and to investigate their _throughout the paper , we have already discussed the relationship between our work and predicate abstraction ( see the previous section and also section [ sec : discussions ] , where we compared our work with over - approximation approaches , in particular the work of namjoshi and kurshan  , and with under - approximation approaches using _ must _ transitions  ) .we discuss here other approaches that are closely related to ours .the work of grumberg et al . uses a refinement of an under - approximation to improve analysis of multi - process systems . the procedure in checks models with an increasing set of allowed interleavings of the given processes , starting from a single interleaving .it uses sat - based bounded model checking for analysis and refinement , whereas here we focus on explicit model checking and predicate abstraction , and we use weakest precondition calculations for abstraction refinement .another closely related work is that of lee and yannakakis  , which proposes an on - the - fly algorithm for computing the bisimulation quotient of an ( infinite state ) transition system .similar to our approach , the algorithm from  traverses concrete transitions while computing _ blocks _ of equivalent states ; if some transition is found to be _ unstable _ the block is _ split _ into sub - blocks .note however that unlike  our algorithm is geared towards error detection and it is formulated in terms of predicate abstraction with a clear separation between state exploration and refinement .there are other important differences between our approach and the work presented in  .we use refinement globally while the block splitting in  is local .this makes the approach in  more efficient in the number of visited states . on the other hand ,the global refinement has the advantage of faster propagating the new predicates across the system but it may lead to unnecessary refinement . as a consequence of this global refinement , our algorithm may not compute _ the _ bisimulation quotient ( as in  ) but rather just _ a _ bisimilar structure ( due to extra refinement ) .we view the experimental comparison of the two approaches as an interesting topic for future work .  in previous work , we developed a technique for finding feasible counter - examples in abstracted programs .the technique essentially explores an under - approximation defined by the _ must _ abstract transitions ( although the presentation is not formalized in these terms ) .the work presented here explores an under - approximation which is more precise than the abstract system defined by the _ must _ transitions . hence it has a better chance of finding bugs while enabling more aggressive abstraction and therefore more state space reduction .model - driven software verification  advocates the use of abstraction mappings during concrete model checking in a way similar to what we present here . in their approach, the abstraction function needs to be provided by the user .the cmc model checking tool  also attempts to store state information in memory using aggressive compressing techniques ( which can be seen as a form of abstraction ) , while the detailed state information is kept on the stack .these techniques allow the detection of subtle bugs which can not be discovered by classical model checking , using e.g. breadth first search or by state - less model checking  .while these techniques use abstractions in an ad - hoc manner , our work contributes the automated generation and refinement of abstractions .directed automated random testing ( dart )  performs a concrete execution on random inputs and it collects the _ path constraints _ along the executed paths .these path constraints are then used to compute new inputs that drive the program along alternative paths .the approach in  is similar to ours as it combines concrete program execution with a symbolic analysis .however , dart applies only to sequential programs , not to concurrent programs as we do here .moreover , dart attempts to cover all the feasible paths through the program , not the reachable ( abstract ) states as we do in our approach .dart does not perform any state matching , and therefore it can not detect if an ( abstract ) state has been visited before . as a result , dart can potentially explore redundant states , e.g. for looping , reactive , programs . another ( methodological ) difference is that dart uses symbolic evaluation while our method uses predicate abstraction with refinement .dataflow and type - based analyzes have been used to check safety properties of software ( e.g.  ) . unlike our work, these techniques analyze over - approximations of system behavior and may generate false reports due to infeasible paths .we presented a model checking algorithm based on refinement of under - approximations , which effectively preserves the defect detection ability of model checking in the presence of powerful abstractions .the under - approximation is obtained by traversing the concrete transition system and performing the state matching on abstract states computed by predicate abstraction .the refinement is done by checking exactness of abstractions with the use of a theorem prover .we illustrated the application of the algorithm for checking safety properties of concurrent programs . in the future, we plan to investigate whether we can extend the algorithm with property driven refinement and with checking liveness properties .we also plan to investigate the integration of our approach with over - approximation based abstraction refinement and to do an extensive evaluation on large systems .we thank the anonymous reviewers for their detailed comments that helped us to improve this article significantly .t. ball , a. podelski , and s. k. rajamani .boolean and cartesian abstraction for model checking c programs . in _ proceedings of tools and algorithms for the construction and analysis of systems ( tacas 2001 ) _ , volume 2031 of _ lncs _ , pages 268283 .springer , 2001 .t. bultan , r. gerber , and w. pugh .symbolic model checking of infinite state systems using presburger arithmetic . in _ proceedings of computer aided verification ( cav97 )_ , volume 1254 of _ lncs _ , pages 400411 .springer , 1997 .d. dams and k. s. namjoshi .the existence of finite abstractions for branching time model checking . in _ proceedings of logic in computer science ( lics 2004 ) _ , pages 335344 .ieee computer society , 2004 .l. de alfaro , p. godefroid , and r. jagadeesan .three - valued abstractions of games : uncertainty , but with precision . in _ proceedings of logic in computer science ( lics 2004 ) _ , pages 170179 .ieee computer society , 2004 .r. giacobazzi and e. quintarelli .incompleteness , counterexamples and refinements in abstract model checking .in _ proceedings of the 8th static analysis symposium ( sas01 ) _ , volume 2126 of _ lncs _ , 2001 .p. godefroid , m. huth , and r. jagadeesan .abstraction - based model checking using modal transition systems . in _ proceedings of conference on concurrency theory ( concur 01 ) _ , pages 426440 .springer - verlag , 2001 .p. godefroid , n. klarlund , and k. sen .dart : directed automated random testing . in _ proceedings of the 2005 acm sigplan conference on programming language design and implementation _ , pages 213223 .acm , 2005 .w. grieskamp , y. gurevich , w. schulte , and m. veanes .generating finite state machines from abstract state machines . in _ proceedings of international symposium on software testing and analysis ( issta04 ) _ ,july 2002 .o. grumberg , f. lerda , o. strichman , and m. theobald .proof - guided underapproximation - widening for multi - process systems . in _ proceedings of the 32nd symposium on principles of programming languages ( popl05 ) _ , 2005 .m. musuvathi , d. y. w. park , a. chou , d. r. engler , and d. l. dill . : a pragmatic approach to model checking real code . in _ proceedings of the 5th symposium on operating systems design and implementation ( osdi02 ) _ , 2002 .     c. s. psreanu , r. pelnek , and w. visser .concrete search with abstract matching and refinement . in _ proceedings of computer aided verification ( cav 2005 )_ , volume 3576 of _ lncs _ , pages 5266 .springer , 2005 .s. shoham and o. grumberg . a game - based framework for ctl counterexamples and 3-valued abstraction - refinement .in _ proceedings of computer aided verification ( cav 2003 ) _ , volume 2725 of _ lncs _ , pages 275287 .springer , 2003 .s. shoham and o. grumberg .monotonic abstraction - refinement for ctl . in _ proceedings of tools and algorithms for the construction and analysis of systems ( tacas 2004 ) _ ,volume 2988 of _ lncs _ , pages 546560 .springer , 2004 .

Abstract: we propose an abstraction - based model checking method which relies on refinement of an under - approximation of the feasible behaviors of the system under analysis . the method preserves errors to safety properties , since all analyzed behaviors are feasible by definition . the method does not require an abstract transition relation to be generated , but instead executes the concrete transitions while storing abstract versions of the concrete states , as specified by a set of abstraction predicates . for each explored transition the method checks , with the help of a theorem prover , whether there is any loss of precision introduced by abstraction . the results of these checks are used to decide termination or to refine the abstraction by generating new abstraction predicates . if the ( possibly infinite ) concrete system under analysis has a finite bisimulation quotient , then the method is guaranteed to eventually explore an equivalent finite bisimilar structure . we illustrate the application of the approach for checking concurrent programs .