in this paper we give a detailed analysis of the asymptotic dynamics and stationary distribution for a special class of metastable markov chains .loosely speaking , a metastable markov chain is one that , on short time scales , looks like a stationary markov chain exploring only a small subset of its state space ; on longer time scales , however , it performs fast and rare transitions between different such subsets .the topic of metastability is an old one .its origins can be traced back at least to the works of eyring and kramers , who studied it in the context of chemical reaction rates . in the context of perturbed dynamical systems , freidlin andwentzell developed a systematic approach based on large deviation theory .this approach was extended by berglund and gentz to cover stochastic bifurcation and stochastic resonance , and by olivieri and scoppola to study dynamics of markov chains with exponentially small transition probabilities .bovier , eckhoff , gayrard and klein developed a systematic approach based on capacities , and gave a precise mathematical definition for metastability .the transition path theory investigates the most probable paths that the markov chain uses when travelling between different metastable states .recent books on various aspects of metastability include the monograph , and the lecture notes .  as we will discuss in section [ dynamics ] , the chains treated in the present paper are metastable in the sense of bovier et al .our situation is considerably simpler than the general one : the state space is of fixed finite ( but possibly large ) size , and the metastability enters via an explicit parameter in the transition matrix .in contrast , the theory described in is built to accommodate the difficult situation where metastability is not necessarily a consequence of some transition probabilities becoming small , but may also arise from a limit where the number of states diverges . in the case of reversible markov chains ,many of our main results can be deduced from the theory of , although our proofs are different and do not rely on the variational methods used there .the benefit of this is that our methods also cover the non - reversible situation where dirichlet - form techniques are less useful .let us describe our setup and results in some more detail .consider a family of discrete time markov chains with finite state space and transition matrices .we assume that the map is continuous at for all , and that the markov chain is irreducible when .for however , the chain may have several essential communicating classes .such a family of markov chains is called an irreducible perturbation of , or simply an irreducibly perturbed markov chain .  the first main result of the paper is a description of the multi - scale metastable behavior of the chain .let be the essential classes of the chain at parameter .we pick for all and define an effective chain with state space .we prove that this chain captures the effective dynamics of the original chain on the shortest metastable time scale , in the sense that its escape probabilities and stationary distribution are asymptotically independent of the choice of the representatives , and asymptotically equal to those of the original chain .for the stationary distribution , this means that , where and are the stationary distributions of the respective chains . a central tool is a natural , _reversible _ chain that has the same stationary distribution as and is interesting in its own right .  in order to explore longer metastable time scales, we renormalize the effective chain : for , all transitions between different states will vanish in the limit . by rescaling time under suitable conditions ,we obtain a new perturbed markov chain , where at least one transition probability between distinct states is of order one as .we can now iterate the procedure described above , yielding effective chains on smaller and smaller state spaces and encoding the dynamics of the original chain on longer and longer metastable time scales .a similar program has been carried out before by olivieri and scoppola .the difference to our approach is that relies on ( and extends ) the theory of freidlin and wentzell , while our approach is closer to the potential theoretic methods of bovier et .this allows us to avoid many of the technical complications found in . also , olivieri and scoppola only consider markov chains with exponentially small transition probabilities , and study asymptotics on a logarithmic scale .in contrast , our methods allow for much more general families of transition matrices , and our results are asymptotically sharp in the sense that we identify the correct prefactors for all our asymptotic identities .the last fact is particularly useful in practice , since it allows us to devise numerically stable algorithms for computing the asymptotic stationary distribution for all states .alternatively , we can compute the ratio of the stationary distributions for two given states without computing the full stationary distribution , thus potentially decreasing the computational cost considerably .these algorithms are the second main result of our work .  to see why numerically computing the asymptotic stationary distribution might be a problem , consider the following simple examplelet , and with elements , , for some . for , and are the essential classes of the chain , so both and are metastable .the stationary distribution of the chain is , .thus depends very sensitively on the behavior of the elements of the transition matrix at small .the reason for this is that the space of solutions to the defining equation is one - dimensional in the case , but multidimensional in the case .this also means that this linear equation is ill - conditioned for small .thus computing numerically by solving an eigenvalue problem is infeasible if the state space is large and the transition matrix is somewhat complicated .metastability also means that a monte carlo simulation of , i.e. running the chain and recording the relative occupation times of the states , will fail for small . in the reversible case, the detailed balance equation can be used to compare the relative importance of and for neighboring , and by iterating for all , but there is no detailed balance equation for irreversible markov chains .therefore , it is not immediately clear how to compute the asymptotic stationary distribution of an irreversible perturbed markov chain in any numerically efficient way .  efficiently computingthe stationary distribution of a large markov chain is an extremely important problem in many areas of applied science .maybe the most prominent example where it is needed is the computation of the page rank in search engines , where metastability also plays a role .it is therefore not surprising that a large body of literature is devoted to the topic , mainly in the computer science community .the seminal paper here seems to be by simon and ando , where they introduce a method for treating what is now known as almost decomposable markov chains , and derive the metastable behavior and some information on the asymptotic stationary measure for such chains .subsequently , the method was clarified and extended , and meyer realized that many of the extensions have a common foundation that he called the theory of the stochastic complement .many further extensions and refinements of the method have been given since .we can not give a full review of the literature here , but rather point the reader , by way of example , to the recent papers and the references therein .an apparently independent effort to treat metastable markov chains took place in the context of game theory and mathematical economy . herethe start was made by hp young .he basically advocated using the markov chain tree theorem , as given in or .up to normalization , it gives the stationary measure as the sum of terms indexed by the directed spanning trees of rooted in , where the weight of a tree is the product of all transition probabilities along its edges ; for details see .as has been pointed out in , the problem with this formula is that while it is in principle not numerically unstable , it involves computing all spanning trees , which is exponentially expensive and thus becomes non tractable for large state spaces .moreover , all of the are usually tiny , and so we are trying to add an astronomical number of tiny terms , which is not a good idea .a different approach was taken by wicks and greenwald who offer a solution that is closer to the one described in , but differs in some important details . at the center of their methodis what they call the quotient construction on stochastic matrices , which allows them to recursively simplify the state space and , by keeping track of the various simplifications , to compute in the end .as can already be guessed from the above discussion , the citation graph on metastable markov chains and their stationary distributions is somewhat disconnected .while some mathematicians , e.g.  or , are aware of the theory of ando and simon , it does not seem to be well known in the probability theory community . on the other hand ,the mathematical theory of metastability following is virtually unknown in the applied community , and the approaches by young and wicks and greenwald appear to be completely disjoint from the others .we hope that , among its other purposes , this paper helps connect these communities . for this reason we review the results related to simon / ando and those of wicks / greenwald at the end of our paper , translate their statements from the language of matrices to probabilistic terminology , and comment on how their results relate to the present paper .the paper is organized as follows : in section [ general escape ] , we collect some results on escape times for irreducible markov chains that seem hard to find in the literature . in section [ asymptotic escape ] , we introduce perturbed markov chains and show how the results from section [ general escape ] can be used to obtain asymptotic expressions of various important quantities .these will be used in section [ dynamics ] to describe the multi - scale effective dynamics of the chain .finally , in section [ algorithms ] , we present our numerical algorithms and compare them to those present in the computer science and economics literature .here we collect the main tools that we will use . in this section, is a general discrete time markov chain .in contrast to the remainder of the paper , we do not assume the state space to be finite , but we will assume that is irreducible and recurrent unless stated otherwise .all of the results below are relatively transparent , explicit identities involving hitting times . given the sheer amount of material on the subject , it is reasonable to assume that some or all of them have been derived elsewhere .we were unable to find an explicit reference for any of them , but will comment on related results where appropriate .  for a markov chain on a state space , the hitting time of a set denoted by , and the return time by . as usual , we will write instead of for , and similarly for .[ basic fact ] assume that is irreducible and positive recurrent , and write for the unique stationary distribution .then for all , [ useful equation ] ( x)p^x(^+_y < ^+_x ) = ( y)p^y(^+_x < ^+_y ) .  proposition [ basic fact ] looks like it should be part of every textbook on discrete time markov chains , but somewhat surprisingly it is not . before we comment on the status of proposition [ basic fact ] in the literature , note that is reminiscent of the detailed balance equation .let us write and for the moment assume that for all .then the quantities can be completed to become the transition matrix of a _reversible _ markov chain that has the same stationary distribution as . in general, will not hold for all , but below we will encounter a situation in the context of perturbed markov chains where it does .when the markov chain itself is reversible , proposition [ basic fact ] is a direct consequence of the well established theory of electrical networks : for example , from proposition 9.5 in it follows that , where is the effective conductance between and , and a global constant . since is symmetric in and , follows in the reversible case.  for the non - reversible case , proposition [ basic fact ] appears much less well known , although it can also be quickly deduced from a known result : corollary 8 of chapter 2 in the unfinished , but brilliant , monograph by aldous and fill directly implies it . as we have not found that statement anywhere else , we give a short proof here for the convenience of the reader .our proof differs somewhat from the one given in and uses the following more general lemma :  [ lem : ht ] let be irreducible and positive recurrent . for all states ,   since the chain is irreducible and positive recurrent , for all and in the state space , so in particular almost surely .thus the first two terms of the last line above sum up to .the last term is equal to , where denotes the time shift by steps . indeed , the random variables and , when nonzero , both count the number of steps from the first occurrence of until the first occurrence of . by the strong markov property , and the claim follows .if , the claim boils down to , so let us assume that .by using lemma [ lem : ht ] in two different ways we obtain [ eq : prophsr ]  ^y(^+_y ) & = ^y((^+_y,^+_x ) ) + ^y(^+_x <^+_y)^x(^+_y ) , + ^y(^+_x ) & = ^y((^+_x,^+_y ) ) + ^y(^+_y < ^+_x)^y(^+_x ) .+  we rearrange the second equation above to obtain the last equality being due to . plugging this back into the first line of , using the fact that , and rearranging , gives [ lemma8 ] ( y ) p^y(^+_x <^+_y ) = , which is essentially corollary 8 of chapter 2 of . for our purposes, we note that the right - hand side of is invariant under swapping and , which proves the claim .* remark : * in the continuous time setting , the whole proof of lemma [ lem : ht ] and almost all of the proof of proposition [ basic fact ] goes through unchanged if we define the only difference is that the formula for the stationary measure in that case is given by , where is the exponential rate with which the process jumps away from .this gives the formula which is a special case of the symmetry result on capacities for non - reversible continuous time markov chains derived by gaudillire and landim , and applied to investigate metastability by beltrn and landim .their proof is quite different from the one presented here .a direct consequence of proposition [ basic fact ] is  [ stat dist representation ] the stationary distribution of fulfills the set of equations [ generic u expression ] = _ y s .since if , is equivalent to for all .we have by irreducibility for all , and so we can divide both sides by it . summing over andrearranging now shows the claim .  to get the most out of corollary [ stat dist representation ] , we need find a way to compute the escape probabilities appearing in .we will now collect some tools that will help us to do this , asymptotically , in the context of perturbed markov chains . unlike the statement of proposition [ basic fact ], we have not been able to find them in the literature , but we still suspect that they are not completely new .[ first formula ] let be an irreducible , recurrent markov chain . for , and , we have [ hitting prob 1a ] p^x(x__a^+ = y ) = p(x , y ) + _ z s a p(z , y ) .when , simplifies to [ hitting prob 1b ] p^x(x__a^+ = y ) = _ z s a p(z , y ) .  for ,let us write for the set of paths that visit precisely times before entering , and in addition move directly from to .more formally , we put , and for , where denotes the cardinality of a set in this case .then , we have , and the sets are disjoint . as the chain is irreducible and recurrent, holds , and thus [ prelim 1 ] p^x(x__a^+ = y ) = p(x , y ) + _ z s a _ k 1 p^x(_y , k , z ) .now for we compute in the second line , we used the strong markov property , and in the third line , finite induction .we now sum up the geometric series in , use , and obtain in the last equality we used that implies . plugging this into proves .  for ,let us start from and note that for , we have . for the term with , we have the first equality holds because for , thus is shown .a variant of proposition [ first formula ] is well known and is the basis of many algorithms for computing stationary distributions of large markov chains .it is called the quotient construction by wicks and greenwald , and the stochastic complement by meyer . while in all those references ,it is written in matrix language , we give here the probabilistic formulation , which also has the benefit that we can give a short and transparent proof . below and in what follows denotes the complement of a set .  [quotient ] ( ) assume that the state space is finite , , and .then [ quot ] p^x ( x__a^+ = y ) = p(x , y ) + _ z , w a^c p(x , w ) ( 1 - p|_a^c)^-1(w , z ) p(z , y ) , where is the restriction of the transition matrix to .  clearly , .now , standard results state that is the unique harmonic extension of the function from to .in other words , is the unique function so that for all , and on .this can be rewritten as for all .since is irreducible , there exists with , where is the operator norm of a matrix .thus is invertible .the claim follows .* remark : * in , the probability of the set of all paths moving from to in and then entering from there is expressed as . in the probability of the set of all paths that leave at but enter anywhere is expressed as the quotient of two escape probabilities .comparing the two and varying over leads to the amusing identity for all , and . + for the following result , we do not assume irreducibility of the chain .[ committor formula ] let be an arbitrary markov chain , .assume and .then   we have where in the last step we have used the strong markov property . by our assumption , we have .since we assumed , we must have ; otherwise the strong markov property would give .thus we can rearrange and obtain the result .  for our next statement ,fix a proper subset , and define for all [ effective p ] p(x , y ) : = p(x , y ) x c , p(x , y ) : = p^x(x__c^c = y ) x c.  [ effective chain ] let be an irreducible , recurrent markov chain. then is the transition matrix of a markov chain .denoting its path measure by , we have [ delete c ] p^x(_b < _ a ) = p^x(_b < _ a ) . for all with , and all .  since is irreducible and recurrent and , for all .thus it is obvious that is a stochastic matrix .the statement is also intuitively obvious , since all we do is replace the motion inside with the effective motion from to its exterior .we nevertheless give the short formal proof .we write for the -th time that the chain travels between two states that are not both in , i.e. on , we define .then for all by recurrence and irreducibility of , and is a markov chain by the strong markov property of . since ,the transition probabilities of are given by .since is disjoint from and , we have and follows by taking expectations .  for our final general statement ,we introduce the notion of a direct path which will be useful in several places below .let , and be subsets of .a tuple is called a _ direct -path of length from to _ if , , and for all , if then and .note that we allow .the set of all direct -paths from to will be denoted by , and the components of will be written , . will denote the length of . for or we will use the notations instead of etc , and speak of direct -paths from to , from to or from to .the probability of a direct -path is defined by .[ second formula ] let be a finite subset of .then for all and , [ hitting prob 2 ] ^x(x__sj = y ) = _ _ j(x , y ) _ i=1^|| -1  the idea of the proof is to start at state and run the markov chain until it either hits or returns to . in the first casewe have reduced the problem to computing and we iterate the argument for the smaller set ; in the second case we use the strong markov property to restart the process .formally , let us proceed by induction on .the claim trivially holds for ; so now let .the third equality below is obtained by the strong markov property . as the markov chain is recurrent and irreducible , we have .thus the last equation can be rearranged to where the numerator may be decomposed as .finally , we use the induction hypothesis for the set to rewrite for all , and obtain re - indexing yields the claim .let be a markov chain on a finite state space .a family of markov chains on indexed by is called a _ perturbation _ of if for all , where denotes the elements of the transition matrix of the chain , .we will speak of an _irreducible perturbation _ of ( or , alternatively , call the family an _ irreducibly perturbed markov chain _ ) if the chain is irreducible for all .note that in the definition of irreducibly perturbed markov chains , we do not require that be irreducible , and indeed the case where has several ergodic components is the interesting one .recall that is called _ accessible _ from under if for some .we write if is accessible from , and say that two states and _ communicate _ if and . the property to communicate forms an equivalence relation , and the respective equivalence classes are called _ communicating classes_. a state is called _ essential _ if for all such that , otherwise transient .it is easy to see that either all members of a communicating class are essential , or all are transient . in the first case, is called an essential ( communicating ) class , or ergodic component . can thus be decomposed into finitely many disjoint essential classes and the set of transient states . to emphasize that a nontrivial ergodic decomposition only exists for , we will always speak of -essential classes and -transient states . will denote the set of all -essential classes .the sets and can be conveniently described in terms direct paths .the following statement could be taken as a definition of -essential classes and -transient states ; the proof of equivalence to the traditional definition of essential classes ( see e.g.  ) is very easy , and omitted here . here and below, we will say that a direct path is _ -relevant _ if , otherwise -irrelevant .[ direct path lemma ] let be an irreducibly perturbed markov chain .+ a ) are in the same -essential class if and only if there exists a -relevant direct -path from to , and a -relevant direct -path from to .+ b ) is in the transient set if and only if all direct -paths from to are -irrelevant .  in much of what follows , we will use the following concept of asymptotic equivalence .two functions and from to are _ asymptotically equivalent _ , if either and are identically zero , or for all with some and .note that in the latter case , we do not assume convergence of or .we write if is asymptotically equivalent to .it is easy to see that is indeed an equivalence relation , and in particular this implies whenever and is not identically zero .we will also need to know that is stable under addition and multiplication in the following sense : if and , then , and .stability under multiplication is trivial , and stability under addition follows from latexmath:[\[|\tfrac{a_\eps + c_\eps}{b_\eps + c_\eps } - 1| =   and transitivity of . note that we did not assume that in either case .let be a -essential class .the restriction of to is the markov chain with state space and transition matrix .it is irreducible , and thus has a unique strictly positive stationary distribution .the trivial extension of to ( by putting for ) will be denoted by the same symbol , and is an extremal point of the convex set of stationary distributions for .the following lemma shows that when we focus our attention on a single -essential class , the unperturbed chain gives a faithful asymptotic description of both the dynamics and the stationary distribution . here andbelow we will write for the unique stationary distribution of , when .[ like unperturbed ] let be a -essential class .then for all and all , [ unperturbed1 ] _ 0 p^x _ ( _ y^+ < _ z^+ ) = p_0^x ( _ y^+ < _ z^+ ) , _ 0 p^x _( _ y < _ z ) = p_0^x ( _ y < _ z ) , and [ unperturbed 2 ] _ 0 = . in particular , .we only prove the first equality from , the proof for the second one is identical .we decompose [ decomp ] p_^x(_y^+ < _ z^+ ) = p_^x(_y^+< _ \{z } e^c^+ ) + p_^x(_e^c < _ y^+ < _ z^+ ) .the second term is equal to , and proposition [ first formula ] gives now for each , there is a -relevant direct -path from to , and so thus , and thus the second term on the right - hand side of vanishes as .for the first term of , fix and decompose we have the first equality is because the probability on the left - hand side is a finite sum of at most -fold products of transition probabilities .the elementary markov property at time gives for each , there is a -relevant direct -path from to , and thus there exists with for all sufficiently small . we conclude that for all sufficiently small and thus as was arbitrary , follows . for , we apply and obtain since the right - hand side above is strictly positive , this implies .  as an immediate corollary, we obtain some information on the structure of the stationary distribution in the limit .recall that is the collection of -essential classes , and is the set of transient states .[ structure ] let . + a ) if , then .+ b ) if for some , then . + c ) in particular if exists for all , then exists for all , and   for all , there exists a -relevant path from to , and thus there is with . on the other hand , by , .so by proposition [ basic fact ] , proving a ) . now let for some -essential class . summing the asymptotic equality over b ) , and c ) is immediate from a ) and b ) .the practical usefulness of corollary [ structure ] depends on our ability to compute asymptotic expressions for the .we now give two statements that will play a key role in all that follows .the first says that hitting probabilities are asymptotically equivalent when the transition matrices are .the second describes how a perturbed markov chain leaves a -essential class , with or without the additional condition that it can not return to its starting point .[ chain comparison ] let and be perturbed markov chains with finite state space , but not necessarily irreducible .let us assume that for the elements of the respective transition matrices .then for all and all , we have   we will first show that the statement holds in the case where and only differ in one row , i.e. where [ one row ] p_(z , y ) p_(z , y ) z s , p_(x , y ) = p_(x , y ) x s. once this is done , we can exploit the assumption that is finite , iteratively change row after row , and prove the full claim .for the case where holds , first note that for all , this can be seen by considering a coupling of the chains and by observing that by , for all .so , the first time when the chains and can differ is after they hit .thus , since , we can now use the strong markov property to find again , and it remains to show that . if , this is trivial . for , , and .we are aiming to use lemma [ committor formula ] , and thus need to deal with the possibility that .we assumed for all , and so we also have for each direct path from to . by the definition of , a direct path from to fulfills for all in a neighborhood of if and only if in a neighborhood of .let us first assume that no such direct path exists .then for all , and thus .now let us assume that such direct paths do exist .since , we can use lemma [ committor formula ] to get now , and the same argument shows .the claim follows .the statement of theorem [ chain comparison ] is rather surprising .the reason is that even though in each step that the chain takes from on its way to , the probabilities for the chains and differ only by a factor that becomes negligibly close to one as , in the same limit the number of steps needed to reach can diverge .indeed , imagine two -essential classes and that are linked by direct paths with , but are linked to and only by paths with . then, starting from a point in , both and will be visited many times before either or is hit .so one could fear that the errors committed by changing each transition probability to an asymptotically equivalent one will pile up ; but as theorem [ chain comparison ] shows , this is not the case .[ magic formula ] let be a -essential class , and .then [ magic formula 1 ] p^x_(_e^c^+ < _ x^+ , x__e^c = z ) _ y e _ e(y ) p_(y , z ) , and [ magic formula 2 ] p^x_(x__e^c = z ) _ y e _e(y ) p_(y , z ) , with normalizing constant   * remark : * is intuitively clear : for small , the markov chain spends such a long time in before exiting that it essentially exits from its -stationary distribution .formula on the other hand is rather remarkable , since a return to happens in a time of order one , so there is no time for the chain to become stationary .  to prove , choose in proposition [ first formula ] .then we decompose [ simple decomp ] p_^x(_y^+ < _ e^c \{x}^+ ) = p_^x(_y^+ < _ e^c \{x}^+ , _ e^c^+ > _ x^+ ) + p_^x(_y^+ < _ e^c \{x}^+ , _ e^c^+ < _ x^+ ). the first term is equal to . the second term in this decomposition as well as the second term inare bounded by and thus vanish , due to lemma [ like unperturbed ] and the finiteness of .the same lemma then yields similarly , for we have by proposition [ basic fact ] , we conclude here , we have used that for all . since the right - hand side above is strictly positive ,we conclude that , and is shown .to see , we use proposition [ first formula ] with .since now , we can use and obtain [ easy ] p^x_(x__e^c^+ = z ) = _ y e p_(y , z ) . as before , for all . by summing over all ,we get plugging these into , we obtain .here we describe the metastable dynamics of a perturbed markov chain . as in the previous section , we will restrict our attention to a finite state space throughout .first of all , we have to define what we mean by metastable dynamics .we follow the theory of bovier et al . in the case of perturbed markov chains on a finite state space , definition 2.1 from ( see also ) goes as follows : a set is called a _ set of metastable points _ if for all and , [ bovier def ] _ 0 = 0 . in words, this means that reaching from the outside of is much easier than traveling between different points of , in both cases with the restriction not to return to one s starting point first .  using lemma [ direct path lemma ] and lemma [ like unperturbed ], it is easy to see that if we choose precisely one point from each of the -essential classes , then the set is a set of metastable points . also, is maximal in the sense that adding a further point to will result in a set no longer fulfilling . on the other hand , removing points from or replacing them with points from may in certain cases still result in a metastable set , depending on the structure of the markov chain and the points in question .we will not pursue this further since is the most natural choice .of course , when some of the contain more than one point , the choice of is not unique .one of our main results is that when defining the effective chain by the transition matrix [ p hat ]  p_(x_i , x_j ) & : = _ e_i(x_i ) p^x_i_(x__s_0^+ = x_j ) i j , + p_(x_i , x_i ) & : = _ e_i(x_i ) p^x_i_(x__s_0^+ = x_i ) + 1 - _ e_i(x_i ) ,  then the relevant dynamical quantities will be asymptotically independent of the choice of the representatives .the occurrence of the expression in is intuitively obvious , since it means that we just monitor the chain when it hits one of our reference points .the factor may be less obvious . to motivate it ,note that by , [ explanation ]  p^x_i_(x__s_0^+ = x_j ) & = _ z s e_i p_^x(_e^c < _ x_i^+ , x__e^c = z ) p^z_(x__s_0 = x_j ) + & _ w e_i , z e_i _ e_i(w ) p_(w , z ) p^z_(x__s_0 = x_j ) .this shows that the factor in cancels one of the dependencies of on the choice of our set . while the terms still do depend on the choice of , we will see below that including the factor in the definition is enough to obtain the asymptotically correct stationary distribution and escape probabilities .this justifies the following definition :  let be an irreducibly perturbed markov chain on a finite state space .the markov chain with state space and transition matrix is called the * effective metastable representation * of corresponding to .  in order to show the properties of the chain announced above, we define a second effective markov chain , this time without reference to a set of representatives . for with put [ q hat ] q_(e , e ) : = _ x e _e(x)^2 p_^x ( _ e^+ < _ x^+ ) , and .the are the elements of a transition matrix when is sufficiently small . as the following proposition shows , this chain is reversible and the reversible measure of is :  [ asymptotic detailed balance ] the quantities satisfy the asymptotic detailed balance equation   the proof of proposition [ asymptotic detailed balance ] rests on the following simple lemma :  [ e instead of x ] let be -essential classes , , , , and . then [ all the same ] p_^z(_y^+< _ x^+ ) p_^z(_e^+ <_ x^+ ) .  from lemma[ like unperturbed ] , we have for all .since for all , the strong markov property gives   when , the claim holds trivially .for , pick and .we use corollary [ structure ] b ) , proposition [ basic fact ] , and corollary [ structure ] b ) again to find thus by , [ no x , y dependence ] _ ( e ) _e(x ) p_^x(_e^+ < _x^+ ) _ ( e ) _e(y ) p_^y(_e^+ < _ y^+ ) , for all .since the right - hand side is independent of , and the left - hand side is independent of , we find [ really no x dependence ] _e(x ) p_^x(_e^+ < _ x^+ ) _e(x ) p_^x ( _ e^+ < _ x^+ ) for all , and similarly for .thus when we multiply with and sum over and , we obtain the claim .the next result shows that the effective metastable representation indeed describes the metastable dynamics of correctly , in the sense that asymptotically it has the right escape probabilities and thus the right stationary distribution .let us write for the stationary distribution and for the path measure of .[ escape probs ] for , we have . in particular .  from and lemma [ e instead of x ], we see that the markov property and the definition of then gives we will show below that for , [ remains ] p_^x_k(_x_j^+ < _ x_i^+ ) = p_^x_k(_x_j^+ < _ x_i^+ ) .once this is done , the markov property for shows the first claim , and from proposition [ asymptotic detailed balance ] we get since by proposition [ basic fact ] , we get since by corollary [ structure ] a ) , we can sum over and obtain , and thus .to show , we introduce the shorthand from , we get .now a standard application of the markov property with the stopping time shows that for , is the unique solution of the harmonic equation for all with boundary conditions , .likewise , is the unique solution of the harmonic equation for all with boundary conditions , .but since we must have , and the claim follows .  the advantage of the chain is that its transition matrix is almost diagonal in the sense that . in particular , is an irreducible perturbation of the trivial ( identity ) markov chain .it is now natural to rescale time so that the most likely transition between two different states becomes of order one .more precisely , we set [ p check ] p_(x_i , x_j ) : = , p_(x_i , x_i ) : = 1 - _ j : j i p_(x_i , x_j ) . since , for each at least one of the terms in the finite sum must be large .the problem is that at this point we can not guarantee that the quantities converge . to see what could happen ,consider the example , , .then , but does not converge .of course , this also implies that does not exist .so far , we did not have to pay attention to that type of problem - all of our results above are valid as asymptotic equivalences , whether or not the quantities in question converge . now however , we need proper convergence to carry on , and will give a sufficient criterion .let , be two functions of .we say that and are _ asymptotically comparable _ , and write , if either both of them are strictly positive and exists in $ ] , or if one or both of them are identically zero .note that we allow and as possible limits .we caution the reader that unlike asymptotic equivalence , asymptotic comparability is not transitive , and is not stable under multiplications . on the other hand , it is obviously symmetric , and we have the following summability property : if , and are mutually asymptotically comparable , and if and have strictly positive , finite limits as , then [ asym comp summ ] _ a_+ _ b_~_c_.we say that an irreducibly perturbed markov chain is _ regular _ if for all and all sequences of pairs , with , we have [ regular def ] _i=1^n p_(x_i , y_i ) ~_i=1^m p_(z_i , w_i ) .we will call a transition matrix regular if the generated markov chain is regular .examples of regular perturbed markov chains include those treated in , where the transition elements are of the form with either converging to a strictly positive limit or identically zero , and independent of .they also include those with property introduced in .  [ conservation of regularity ] for a regular perturbed markov chain with transition matrix ,define as in , and as in .then and are transition matrices of regular perturbed markov chains .  by , for [ aa0 ] p_(x_i , x_j )_ w e_i , ze_i _ e_i(w ) p_(w , z ) p^z_(x__s_0 = x_j ) , and proposition [ second formula ] gives [ aaa ] p^z_(x__s_0 = x_j ) = _ _ s_0^c(z , x_j ) _ i=1^||-1 in lemma [ regularity lemma ] below we will show that if contains one representative of each -essential class then exists and is strictly smaller than one for all .thus each exists .in other words , is given as a sum of terms of the form with , where exists for all .when plugging this into , we can apply the extension of to finite sums to show that is the transition matrix of a regular markov chain . by , this immediately implies convergence of the transition probabilities . rewriting the second equation in in the form we see in addition that the chain is a regular perturbed markov chain .it remains to prove the claim used in the proof above .[ regularity lemma ] let be a perturbed markov chain .assume that a set contains one element of each -essential class .let with .then for all , exists and is strictly smaller than .as contains a representative of each -essential class , there must be a -relevant direct path from to some .so , .for the existence of the limit , let first . for , we have as .let us now assume that the claim holds for all such that with some .let be such that .then , by the induction hypothesis , exists and is strictly smaller than .thus also exists and is strictly smaller than .the claim follows by induction .we have thus found a way to successively describe the multi - scale metastable dynamics of regular perturbed markov chains : starting with the original chain , we derive and then . by theorem[ conservation of regularity ] , and are again regular perturbed markov chains .moreover , all of the -essential classes consist of exactly one element , and whenever .so , describes the effective metastable dynamics , but still in the original time scale .the transformation from to means that we go to a time scale where the most likely transitions between different states become of order one . in other words , there exist with . by lemma [ direct path lemma ] , this implies that will no longer be a -essential class on its own : it will either form a larger -essential class together with some , , or it will have become -transient . in any case , the number of -essential classes will be smaller than the number of -essential classes .thus by applying the transformations to the matrix , and iterating the procedure , we can recursively explore longer and longer time scales of the dynamics .  on a purely theoretical level ,our theory of multi - scale metastable dynamics for regular perturbed markov chains is thus complete .however , if one attempts to ( numerically ) compute the transition probabilities at the different time scales , the problem arises that all relevant expressions in our theory still contain terms of the form . in the next section, we will show why naive attempts to compute this quantity numerically are likely to fail , and present a numerically stable algorithm for computing them .a byproduct of our algorithm is a numerically stable method to compute the matrix elements of the transition matrix , and thus the stationary weights for all -essential classes .this section deals with aspects of the numerical computation of the transition probabilities and given in and , respectively . before we proceed we would like to make clear that subtle issues coming from the field of _ computable analysis _ fall beyond the scope of this article . intuitively though, we mean the following by `` numerical computation '' : if someone enumerates , step by step , all members of an infinite sequence of transition matrices that converge towards , we are able to process each using only a computer and produce , step by step , an infinite sequence that converges towards the matrices ( or ) .this corresponds roughly to the property of being _computably approximable_. note that in this case we do not know how fast the sequence is converging to the limit . said otherwise , if we want a precise approximation of , say , , we have no idea until which we should process the . this is a usual issue in numerical analysis . if in addition we would know that the -th approximation is , _e.g. _ , at most away from the limit , we would know when to stop to obtain the desired precision .this corresponds roughly to the property of being _computable_.  the starting point of our considerations are the formulae [ p eps numeric ] p_(x_i , x_j ) = _ e_i(x_i ) ( _ j i p_(x_i , x_j ) + _ z s_0 p_(x_i , z ) p_^z ( x__s_0 = x_j ) ) , and [ q eps numeric ] q_(e , e ) = _ x e _ e(x)^2 ( _ y e p_(x , y ) + _ z \{x } e p_(x , z ) p_^z(_e^+ < _ x^+ ) ) , both of which are obtained from the definition of the respective quantities using the strong markov property . in both cases , the task is to compute a hitting probability of the form [ committor def ] h_a , b(z ) : = p_^z(x__a b a ) , where and . in the case of , and . such hitting probabilities are well understood in the theory of markov processes : is called the committor function in and the equilibrium potential of the capacitator in , and is the unique harmonic continuation from to of the indicator function of .this means that is the unique solution of the linear system [ possibly ill conditioned ] _ z s c(p_(x , z ) - _ x , z ) h_a , b(z ) = - r(x ) , x c^c = s c , where .let us write for the restriction of to .if and is irreducible , we have seen in the proof of proposition [ quotient ] that is invertible .we thus find the committor function by matrix inversion : [ matrix formulation ] h_a , b(x ) = [ ( 1-   the problem with this formula is that as , the matrix may converge to a non - invertible matrix . in that case , some matrix elements of will diverge , and even though the quantities themselves are bounded by for all , computing them numerically becomes unreliable as .our first result will identify situations where this can not happen .we call a state an _ asymptotic dynamical trap _ ( or simply a trap ) with respect to if a necessary condition for to be a trap is that there exists no direct -relevant path from into . on the other hand , for all is at least one -relevant path from to some -essential class .thus if intersects all -essential classes , no traps will exist .recall that for a matrix , the _ condition number _ is given by , where is the operator norm with respect to any norm on the underlying vector space . in our case, it is convenient to use the supremum norm on the vector space .   since is substochastic , clearly . on the other hand ,the absence of traps with respect to allows us to find and , both of them independent of , and so that for all .the strong markov property then implies for all and all .thus for and bounded , we find consequently , the left - hand side above is absolutely summable , and for all . taking the supremum over , the claim follows .  by construction, contains precisely one point of each -essential class , and thus there are no asymptotic dynamical traps with respect to . by proposition [ condition nr ] andwe can thus compute in a numerically stable way .in fact , the perturbative nature of the problem makes the following newton scheme particularly useful .let denote the restriction of to , and set . by , we need to find .we use as a seed for the newton iteration , and employ the usual recursion . by putting , we find and .so , is a polynomial in , and we can use the resulting commutativity to obtain [ newton 2 ] b_k+1 - a_^-1 = - a_0 ^ -1 a_(b_k - a_^-1 ) a_0 ( b_k - a_^-1 ) for all . in the special case , this can be transformed to [ newton 1 ] b_1 - a_^-1 = a_0 ^ -1 ( a_- a_0 ) ( a_^-1 - a_0 ^ -1 ) = a_0 ^ -1 ( |p_0 - |p _ ) ( a_^-1 - a_0 ^ -1 ) .thus and proposition [ condition nr ] guarantees that we can choose sufficiently small so that converges to very quickly . to illustrate this , we restrict ourselves to the special case where with the matrix bounded uniformly in .then , for some , and this means that when we are interested only in transitions of size or bigger , we only have to calculate logarithmically ( in ) many .therefore , it might seem that all is well , but this is not entirely so .the reason is that a subtle problem arises from the multi - scale structure of the dynamics : at a given metastable time scale , it is in general not obvious what computational accuracy we need to achieve in order to obtain the asymptotically correct dynamics on longer metastable time scales .this phenomenon can best be explained by an example .figure [ figure example ] shows a graphical representation of a couple of metastable markov chains . for both of them , , andboth of them have transition probabilities corresponding to the solid arrows : , , and .only one of them has the dashed arrows , i.e.  .all other transition probabilities are zero except those mapping a point to itself , which are adjusted to give a stochastic matrix . with or without the dashed arrows , , and are the -essential classes , while is -transient . also in both cases , , while .so on the first metastable time scale , transitions from to play no role .but whether or not we can stop our computation of after reaching order depends on the presence of the dashed arrows .if the dashed arrows are present , we can stop the computation of after reaching order : on the next ( and final ) metastable time scale , we will have and . will be connected to via , by transition probabilities of order one .  however, if the dashed arrows are absent , stopping the calculation at order leads to an effective markov chain where can not be reached from , and thus to wrong results on the next metastable time scale . in the correct dynamics on that time scale and form a new effective metastable state , and transitions between it and are ( after rescaling ) of order .for this to be resolved correctly , the transition from to of order needs to be present already in the effective dynamics on the first metastable time scale .  in the simple example at hand it is easy to directly figure out what is going on , but to decide when a given approximation of is good enough to give correct dynamical results on all further metastable time scales for general chains on large state spaces is a subtle problem .here we only give a necessary condition , about which we conjecture that it is also sufficient , and which is accessible to numerical validation .let us write for path measure of a given approximation to the chain . by theorem [ escape probs ] , when is the representative from and the representative from , and thus for all .so in order to obtain the correct asymptotic stationary distribution for our approximate chain , we have to increase the accuracy at least until [ necessary cond ] q_(e_i , e_j ) p_,a^x_i ( _ x_j < _ x_i ) .it would not be surprising if this were already sufficient for some sort of agreement of the metastable dynamics on all further metastable time scales .since in general the escape probabilities do not characterize the transition probabilities of a markov chain , a proof of this conjecture is not immediate , and we do not pursue this any further here .instead , we discuss how to check numerically .  by , the numerically tricky part in computing is .since will not intersect all -essential classes unless there are only two of them , we can not use proposition [ condition nr ] this time , and indeed in most situations a direct calculation of will be numerically unreliable .however , for the very same reason , namely since intersects only two -essential classes , we can successively lift these traps and arrive at a simplified chain without traps for which the probability of hitting before is asymptotically equivalent to the original one .  the basic step in this procedure is the following .assume that is a -essential class of a perturbed markov chain , and that .we define a new markov chain on the state space by its transition probabilities , where whenever , and [ hat transitions ] p_(x , e ) : = _ z e p_(x , z ) , p_(e , x ) : = _ z e _e(z ) p_(z , x ) , p(e , e):=0 for all .here , is the normalization that ensures that is a stochastic matrix .we say that the traps in ( with respect to ) have been lifted in .this terminology is justified by  [ lifting the trap ] let be a perturbed markov chain , a -essential class of , and the markov chain where has been lifted . + a ) let .then for all , , while for , . + b ) if is regular , then is a regular perturbed markov chain .+ c ) if is regular , then either is a -transient state , or is an element of a -essential class that contains at least one further element . in the latter case ,the number of -transient states is strictly smaller than the number of -transient states .consider the chain with state space and transition matrix , where when and when .denoting its path measure by , proposition [ effective chain ] gives for all .we now define by replacing with for , and keeping them the same if .then implies that for all , and thus theorem [ chain comparison ] gives .finally , noting that does not depend on whenever , we can replace all by a single state , and claim a ) follows .for b ) , note that by regularity of the chain , for all .so the quotient in either converges or diverges to infinity as .since it is bounded by by construction , the latter is not an option , and the converge .so the markov chain defined by them is a perturbed markov chain . finally , this markov chain is again regular , since products of its elements can be written as weighted sums of products of the with nonnegative weights .we have shown b ) .  for c ) ,note that by b ) exists , and since , there must be at least one state with .lemma [ direct path lemma ] implies that if is a -essential class with , then all direct paths from to are -irrelevant .so if one of the elements with is connected to a different -essential class via a -relevant direct path , then is -transient . on the other hand ,if no is connected to any by such a direct path , then each such must be an element of , and must be connected to by a -relevant direct path .it follows that is in the same -essential class as , and thus not a -transient state .the claim follows .using theorem [ lifting the trap ] , we can now give a general recursive algorithm for numerically computing expressions of the form given in simultaneously for all , up to asymptotic equivalence :  1 .determine the set of all -essential classes not intersecting .2 .  if , compute by solving the well - conditioned linear system .finish the algorithm .3 .  compute the -stationary measures for each .4 .  lift all the traps in by .this results in a new state space , where all elements of are replaced by a single state .keep track of the elements of the original state space that become lumped into .return to ( 1 ) with the new state space .we note that steps ( 3 ) and ( 4 ) are trivial to parallelize . by theorem [ lifting the trap ]c ) , each step either decreases the number of -essential classes in the chain , or leaves it unchanged and decreases the number of transient states .we thus see that the algorithm terminates .once it does ( in step 2 ) , we know for all in the final state space .theorem [ lifting the trap ] a ) now guarantees that for all states of the chain from the previous step that were collapsed into .thus we can recursively go backwards until we reach the original state space , where we now know all up to asymptotic equivalence .in particular , this gives a stable algorithm for the asymptotic numerical approximation of the coefficients .since the expressions are also escape probabilities ( for a different markov chain ) , we can compute them by the same algorithm .if they agree with to leading order in , our necessary criterion is met and the approximate chain has the same asymptotic stationary measure as the true one .another useful aspect of our algorithm is that the determine the limiting stationary distribution of the chain through the formula which is derived in analogy to , using proposition [ asymptotic detailed balance ] .computing the stationary distribution of a large markov chain with many metastable sets is a very important problem in practice .for example , it is how internet search engines compute page importance ranks . as a consequence ,there has been tremendous activity in the computer science community on the topic .most of the developments seem to be based on a seminal paper by simon and ando .seemingly independently , the problem has been treated by a much smaller group of people in mathematical economy , starting with and with significant recent progress by wicks and greenwald .both approaches are based on formula , which itself is closely related to . in the literature following and , this leads to what is known as the method of the stochastic complement . for a finite markov chain on a state space , the first step of the method is to decompose into disjoint sets .equation with then allows to compute [ andos ] p(x , y ) : = p^x(x__s_j^+=y ) for by using matrix multiplications and by computing the inverse of the matrix .the are the transition probabilities of an effective markov chain only running inside .writing for the stationary distribution of the effective chain , and for the full stationary distribution , it can be shown that [ ando1 ] ( x ) = _ j _ j(x ) for all , where is the stationary distribution of the markov chain with state space and transition probabilities [ ando2 ] q(s_i , s_j ) = _ x s_i , y s_j _ i(x ) p(x , y ) .equation is similar to the statements of our corollary [ structure ] , with the taking the role of , and the the role of .equation is in analogy to the expression [ fo1 ] p_(x_i , x_j ) _ w e_i , z e_i_ e_i(w ) p_(w , z ) p_^z(x__s_0 = x_j ) that we get for when combining and .the drawback of the method is that a priori , we have no control over the numerical difficulty of computing . for example , let consist of two elements .then , and thus the computation of is no easier than the problem we have treated in the present paper ; in particular , if is a perturbed markov chain and and are in different -essential classes , the matrix will become singular as . therefore without any further assumptions , the theory of simon and ando as it stands gives no numerically feasible way of computing .a suitable such further assumption is to choose the decomposition in a way that makes all transitions between different small .the situation where this is possible has been treated already in , and is nowadays known as a the theory of nearly reducible ( or nearly decomposable ) markov chains . in the framework of the present paper ,a perturbed markov chain is nearly reducible if for each there exists a unique -essential class so that all -relevant paths from to end in . in the terminology of ,this means that the local valleys corresponding to the maximal metastable set from section [ dynamics ] do not intersect .when a markov chain is nearly reducible , it is known ( and follows from in our case ) that we can ignore transitions between different for the approximate computation of the ; in the case of perturbed markov chains and when each contains exactly one -essential class , this means .the reduced chain is then similar to our , and by a recursive algorithm similar to the one given in the present section , the stationary measure can be computed .so in the context of nearly reducible markov chains , the contribution of our work is on the one hand a systematic , rigorous asymptotic theory , and on the other hand an extension to the case where the markov chain no longer needs to be nearly reducible : in the latter case , the take the role of the , and the presence of the transient set is accounted for by replacing by , together with a recipe to compute the escape probabilities contained in the latter equation .the second approach that we are aware of which uses is the recent work by wicks and greenwald , who call their approach the method of the stochastic quotient .they work in the situation where with bounded corrector matrix , and they do not need to assume almost decomposability . as we do , they pick a representative from each -essential class .then they apply with , i.e. they compute the probabilities to either leave at a given , or to return to .the leading order of this quantity can be computed efficiently by a matrix calculation , since the matrices remain bounded as thanks to the absence of from . indeed , as wicks and greenwald note, it suffices to invert .this construction leads to an effective chain where the class is replaced by a -essential class containing just the one element .they do this construction for all -essential classes , and indeed also for transient communicating classes .after that , they rescale transition probabilities out of each of the ( now trivial ) -essential classes much like we do in , keeping track of the factors by which they speed up each individual trap .this results in a markov chain with fewer -essential states , or fewer transient states .recursively iterating the procedure while always keeping track of the rescaling factors , they arrive at a stable algorithm for computing the stationary distribution .it is obvious that the algorithm of wicks and greenwald and ours share quite similar ideas .the difference is that while our algorithm lifts metastable traps completely , the wicks - greenwald algorithm keeps one point in each trap .the advantage of the wicks - greenwald algorithm is that the whole stationary distribution can be computed at once , while in our algorithm one has to compute separately for each pair .the advantage of our approach is that it is local : if we are only interested in the relative importance of two given states and , we need only compute the ratio . depending on the structure of the chain ,this can be done by lifting only a tiny fraction of the traps present in the state space .an additional advantage of our approach is of course that we also obtain information about the metastable dynamics , information which is not contained in the stationary distribution alone .a. milias - argeitis , j. lygeros .efficient stochastic simulation of metastable markov chains , _50th ieee conference on decision and control and european control conference ( cdc - ecc ) orlando , fl , usa , december 12 - 15 _ , 2011          j.r .wicks , a. greenwald : an algorithm for computing stochastically stable distributions with applications to multiagent learning in repeated games . _proceedings of the 21st conference in uncertainty in artificial intelligence , 2005_. auai press 2005  j.r .wicks , a. greenwald : a quotient construction on markov chains with applications to the theory of generalized simulated annealing . _international symposium on artificial intelligence and mathematics ( isaim 2006 ) , fort lauderdale , florida , usa_.

Abstract: we consider a simple but important class of metastable discrete time markov chains , which we call perturbed markov chains . basically , we assume that the transition matrices depend on a parameter , and converge as . we further assume that the chain is irreducible for but may have several essential communicating classes when . this leads to metastable behavior , possibly on multiple time scales . for each of the relevant time scales , we derive two effective chains . the first one describes the ( possibly irreversible ) metastable dynamics , while the second one is reversible and describes metastable escape probabilities . closed probabilistic expressions are given for the asymptotic transition probabilities of these chains , but we also show how to compute them in a fast and numerically stable way . as a consequence , we obtain efficient algorithms for computing the committor function and the limiting stationary distribution .   escape times , non - reversible markov chains , asymptotics   60j10 , 60j22


fast template or pattern matching  is a critical step in many remote sensing applications such as object detection and recognition  , image registration and alignment  , glacier surface movement detection  , road detection  , seismic monitoring  , shadow detection  , cloud detection and tracking  , sea ice tracking  , stereo image matching for space born imagery  , dem generation  , image super resolution  , and content based image retrieval . in template matching , a smaller template or target image is matched at multiple locations of a larger search image to find the best match location that maximizes an appropriate similarity measure .  in most of the remote sensing applications ,the search space is large which increases the computational complexity of the matching process .numerous techniques have been proposed in literature to make the matching process fast .based on the search accuracy , these approaches may be broadly divided into approximate accuracy and exhaustive accuracy techniques .the first category obtains fast speedup at the cost of some loss of accuracy and often incorporates one or more approximations . for example, the search space may be approximated with a smaller search space , the target may be approximated with a simple representation , or the similarity measure approximated with a simpler measure .the exhaustive accuracy techniques obtain fast speedup without losing accuracy .this category includes domain transformation techniques using fft and bound based computation elimination algorithms in which unsuitable search locations are skipped from computations . in this paperwe argue on maintaining exhaustive accuracy in the proposed algorithm which skips unsuitable search locations based on bound comparisons . to make the process fast ,the search image is processed in a controlled way to avoid any loss of accuracy .bound based computation elimination algorithms are exhaustive accuracy fast template matching techniques . most of these algorithms are based on the sum of absolute differences ( sad ) or the sum of squared differences ( ssd )  .elimination algorithms using similarity measures invariant to the intensity and contrast variations such as normalized cross correlation ( ncc ) or correlation coefficient ( ) ( or zero - mean ncc ) are relatively few including  .algorithms in this category are the main focus of this paper . in most of the remote sensing applications ,the images to be matched are acquired at different time of the day , often with significant time lag . therefore simple measures like sad and ssd are more vulnerable to errors as compared to the correlation coefficient .correlation coefficient ( or zncc ) is more robust to the linear photo - metric variations between the two images to be matched .correlation coefficient between template image and a location in search image is denoted as and defined as  where and are the means of and respectively .if are two shifted blocks in the same image , then in will represent local auto - correlation with a shift . in this paperwe use local auto - correlation to speedup the matching process .local auto - correlation is enhanced by inducing controlled non - uniform image blur in the search images . as a result, we obtain high matching speed with a robust match measure .  in the category of correlation coefficientbased fast template matching algorithms , we consider transitive algorithm ( tea )  mainly because this algorithm gives the opportunity to incorporate non - uniform blur to obtain high speed template matching .also tea has been shown to be faster than the previous algorithms of the same category  .however , tea can not be directly used for this purpose because the design parameters of this algorithm are currently user defined .especially the group size parameter and the initial threshold .we develop an efficient group size ( egs ) algorithm which maximizes an estimation of the the eliminated computations over group size parameter .we then incorporate egs algorithm within a novel algorithm for optimization of auto - correlation ( opta ) which computes non - uniformly blurred search image . a scheme for early detection of the high initial threshold is also proposed .all these algorithms combined with tea make a fast template matching system which has the same accuracy as the original algorithm but obtains significant speedup .experiments are performed with a wide variety of the template images on real satellite and aerial image datasets .we observe significant speedup over existing techniques .form groups of size .( b ) two known correlations and bound the unknown correlation .,width=321 ]  , , } and are quadratic in shape .if and then third side is bounded by .,width=302 ]in transitive algorithm the search image is divided into small groups of contiguous search locations .local auto - correlation of the group center is computed with all other locations within that group and transitive bounds are computed . at run time , the template image is only matched with the group center , while transitive bounds are used to skip all other unlikely locations ( fig .[ fig : tea2 ] ) .transitive bounds  are defined for three images , template , outer block and central block .pairwise correlations are given by \{ , , } ( fig .[ fig : threeblocks ] ) .each correlation is bounded by the other two correlations by the transitive inequality .considering as bounded correlation and the other two as bounding correlations , the upper bound is given by and the lower bound is given by transitive gap is the difference between the upper and the lower transitive bounds we observe that is contained within the transitive gap .the bound tightness can be defined as : the smaller the transitive gap ( ) , the tighter the transitive bounds .transitive gap will be minimized if the magnitude of at least one of the two bounding correlations \{ , is maximized .taking the derivative of w.r.t any one of the two bounding correlations and setting it equal to zero . from [ eq : eps ] we get  . since therefore or   in the remote sensing images , consecutive search locations are often highly correlated , therefore the nearby search locations can be grouped such that intra - group auto - correlations remain high . fig .[ fig : tea2 ] shows the search image divided into groups of search locations .each group has a central location and others are outer locations .auto - correlation of each outer location with the group center is computed by using a very efficient algorithm [ algo : lsa ] . at run time , the template image ( ) is only matched with , while the remaining locations can be skipped if the sufficient elimination condition given below is satisfied .[ prop : sec ] without loss of accuracy , a search location can be skipped if there exists another search location such that where is the correlation coefficient between and .  if the sufficient elimination condition is satisfied then it is guaranteed that , therefore can not exhibit better similarity than .in this section we analyze different types of costs involved in the transitive algorithm .these costs include the auto - correlation cost , central correlation cost and the cost of computing correlation on the locations with failed elimination condition . based on this analysis , we develop an efficient group size algorithm which takes as input the auto - correlation matrix , initial threshold and computes a group size that minimizes the sum of all costs . , template size , group size auto - correlation matrix  running - sum( )   [ algo : lsa ]   auto - correlation is efficiently computed by algorithm [ algo : lsa ] .the reference image is multiplied with its shifted versions and running sum approach is used to compute the sum of products over each block requiring only four operations .the overall complexity of this algorithm is , where is the group size or the number of shifts applied to the reference image .note that this cost is significantly smaller than the cost of a single template matching , because .central correlation cost is required to match the template image with the group centers , therefore is proportional to the number of groups in the search image where is the one time matching cost . remains fixed for a particular group size and can be reduced by increasing the group size .this is the cost of computing the correlation at the locations with failed sufficient elimination condition : , where are the number of retained locations . as the group size increases the within group auto - correlation reduces due to increased distance between and .as a result , the transitive bound becomes loose , causing the sufficient elimination condition to fail more often and an increase in .a relatively precise estimate of retained locations can be found by proposition [ lem : autocor1 ] while a relaxed but easy to pre - compute estimate can be found by proposition [ prop : simplebound ] .[ lem : autocor1 ] for a fixed threshold value , the sufficient elimination condition will be satisfied on all search locations with auto - correlation satisfying the following inequality   elimination will be obtained if or simplifying, being quadratic , has two roots . in the practically useful ranges of and , the upper root mostly remains positive while the lower root is mostly negative .since auto - correlation of the natural images for small lags is positive ( ) therefore in most of the cases only the upper root is the valid solution .hence elimination will be obtained if is larger than the upper root :   note that in narrow ranges , lower root may also become positive resulting in more elimination than the estimation based on . however, these cases being less frequent may be ignored from the estimation without inducing significant error .proposition [ lem : autocor1 ] can be used to estimate the number of search locations with failed elimination condition if is known which limits the beforehand estimation of the retained locations .however , proposition [ prop : simplebound ] enables us to estimate the number of retained locations without knowing .we empirically observe that the estimation error in proposition [ prop : simplebound ] is small and the result is used to formulate the efficient group size algorithm .[ prop : simplebound ] a search location will be eliminated if \ge \sqrt{1-\rho_{tb}^2},\ ] ] where the expectation is computed over a neighborhood around that location .taking expectation of both sides of and considering as a fixed value , \ge \rho_{tb}e[\rho_{tc } ] + \sqrt{1-\rho_{tb}^2}e\big[\sqrt{1-\rho_{tc}^2}\big].\ ] ] since is uncorrelated with most of the search locations in , therefore =0 ]             \leftarrow$ ] egs( )   [ algo : blur ]a high initial threshold at the start of the search process may significantly increase the probability of success of the sufficient elimination condition and significantly reduce the execution time .however , a very high threshold may result in skipping the best match location while a very small value may result in increased computational cost . in the previous version of tea the initial threshold was a user defined parameter . we propose a simple strategy to automatically find a suitable threshold value . in the previous tea , the search space was scanned only once .for each group , the template was matched with the group center and the bounds were computed for the remaining patches in the group .all patches for which the elimination test failed were processed before moving to the next group . as a result, the search space was required to be scanned only once .in contrast , we propose two scans of the search space . during the first scan ,the template is only matched with the group centers and the maximum correlation value is tracked . once all group centers are exhausted , the maximum value of the central correlation is used as initial threshold in the second scan .  [tab : addlabel ]in order to test the proposed fast template matching algorithms , we have performed extensive experimentation on real satellite and aerial image datasets .performance of the proposed efficient group size ( egs ) algorithm and the optimal auto - correlation ( opta ) algorithm ( with efficient group size ) is separately reported .both of these algorithms include the proposed strategy for early detection of the high initial threshold .egs and opta are compared with six current algorithms including previous tea  , pce  , fft  , zebc  , amwu  , sad based on successive elimination and partial distortion elimination  .the execution time comparisons are performed on intel core i3 cpu 2.10ghz and 3.00 gb ram .experiments are performed on satellite image dataset ( si ) of a low population density seaport ( fig .[ fig : datasi ] ) and aerial image dataset ( ai ) of a densely populated area ( fig .[ fig : dataai ] ) . for each of the two datasets , templates of 11 different sizesare used ( table 1 ) .total number of templates in the si dataset are 2168 and in the ai dataset are 2086 . the templates are obtained from a different view point at a different time .therefore both datasets contain projective distortions as well as illumination variations .the dataset , c++ code and the experimental setup will soon be made publicly available at www.csse.uwa.edu.au/~arifm/opta.htm .622 pixels and the template images have sizes varying from 21 to 121 . , width=340 ]  2389 pixels and the template images vary from 29 to 129 pixels . , width=340 ]  [ tab : sispeed ]  [ tab : aispeed ]  the opta ( algorithm [ algo : blur ] ) is executed by varying \{0.05 , 0.20 , 0.50 , 0.20 , 0.05 } , and .the algorithm is not much sensitive to because the image regions with more initial blur start failing the proposition 4.3 and blurring is stopped in those regions .therefore , the three values of resulted in approximately the same optimized auto - correlation .the results in tables [ tab : sispeed ] and [ tab : aispeed ] are reported for .  in the comparison of opta with egs, we observe that opta has outperformed egs in all experiments with a maximum speedup of 2.46 times .no loss of accuracy is observed in the opta algorithm .maximum localization error is observed up to pixels .both egs and opta algorithms outperformed existing tea  ( group size and ) on all datasets ( tables [ tab : sispeed ] and [ tab : aispeed ] ) .opta has obtained maximum 3 times speedup over tea and on the average , 2.7% more elimination on ai dataset ( table [ tab : elim ] ) .opta outperformed pce  ( initial partitions= , , two - stage initialization strategy ) on all datasets obtaining maximum speedup of 7.13 times and average computation elimination up to 9.5% on ai dataset .this experiment demonstrates significant improvement over current state of the art algorithm .the opta outperformed zebc  with relatively bigger margins .for zebc , the partition parameter is kept close to 8 and as recommended by  . since the template rows must be divisible by , for each experimentthe most efficient value of is chosen .maximum speedup of opta over zebc is 12.13 times . on si dataset zebcobtained higher computation elimination . howeverdue to the larger cost of the elimination test , zebc was not able to get the benefit of high computation elimination .we have also compared opta with fft  and we observe up to 22.04 times speed up .speed up over fft is generally larger for smaller template sizes and lesser for the larger sizes .it is because the computational cost of fft has relatively less variation with template size increments .for the three ai dataset , , and initial threshold is varied as \{0.70 , 0.75 , 0.80 , 0.85 , 0.90 } and the variation of total execution time over these two datasets for the opta algorithm is observed as \{294 , 287 , 287 , 287 , 286 } seconds respectively .the execution time is almost constant for the threshold 0.75 .this experiment shows that the sensitivity of the proposed algorithm to the initial threshold is low ..average percent computation elimination on the si and ai datasets [ cols="^,^,^,^,^,^,^",options="header " , ]   [ tab : elim ]  the proposed algorithm ( opta ) is also compared with amwu  . for this comparison , we prepared two new ai datasets with template sizes and , because amwu requires the template dimensions to be in the powers of 2 . for the ai reference of size 1549 , amwu is more than an order of magnitude slower than our proposed algorithm .amwu is much efficient on the small search space  , however for the large search space , its performance degrades probably significantly long lists of the temporary winners make the hashing scheme inefficient .  among the two datasets , opta obtained more speed up on the ai dataset mainly because it contains significant details and high frequency contents .opta was able to induce blurring at most of the ai reference image locations without loss of image quality .less speedup on si dataset is because a big portion of si reference image is sea , having no details .opta was not able to blur that portion because blurring caused image quality degradation .  in both datasets ground truthwas manually marked and a match was considered correct if it was within pixels of the ground truth location . using this criterion , opta and egs exhibited 100% accuracy on all datasets while sad has an average accuracy of 81.0% over ai and 76.11% over si datasets .maximum speedup of opta over sad with sea and pde is 26.93 times .although more efficient implementations of sad exist , however accuracy of sad can not be improved over the exhaustive accuracy .a fast template matching system was proposed to find a target in a large search space .fast match speedup was obtained by inducing non uniform blur in the search space .exhaustive math accuracy was ensured by maintaining a minimum image quality at all locations .for this purpose an optimal auto - correlation ( opta ) algorithm was proposed .opta algorithm was based on efficient group size ( egs ) algorithm which made the transitive elimination algorithm more efficient .moreover , a technique for early detection of the high initial threshold was also proposed .the fast template matching system was compared with six existing algorithms on two different datasets .the proposed system consistently outperformed the existing techniques .this research was supported by australian research council grants dp1096801 and dp110102399 .s. rousseau , d. helbert , p. carre , and j. blanc - talon , `` compressive pattern matching on multispectral data , '' _ geoscience and remote sensing , ieee transactions on _ , vol .52 , no . 12 , pp .75817592 , dec 2014 .y. bentoutou , n. taleb , k. kpalma , and j. ronsin , `` an automatic image registration for applications in remote sensing , '' _ geoscience and remote sensing , ieee transactions on _ , vol .43 , no .. 21272137 , sept 2005 .r. caves , p. harley , and s. quegan , `` matching map features to synthetic aperture radar ( sar ) images using template matching , '' _ geoscience and remote sensing , ieee transactions on _ , vol . 30 , no . 4 , pp . 680685 , jul 1992 .x. chen , j. guan , z. bao , and y. he , `` detection and extraction of target with micromotion in spiky sea clutter via short - time fractional fourier transform , '' _ geoscience and remote sensing , ieee transactions on _ , vol .52 , no . 2 ,10021018 , feb 2014 .j. ma , j. chan , and f. canters , `` fully automatic subpixel image registration of multiangle chris / proba data , '' _ geoscience and remote sensing , ieee transactions on _ , vol . 48 , no . 7 , pp .28292839 , july 2010 .l. bandeira , j. saraiva , and p. pina , `` impact crater recognition on mars based on a probability volume created by template matching , '' _ geoscience and remote sensing , ieee transactions on _ , vol .45 , no . 12 , pp .40084015 , dec 2007 .r. mayer , f. bucholtz , and d. scribner , `` object detection by using `` whitening / dewhitening '' to transform target signatures in multitemporal hyperspectral and multispectral imagery , '' _ geoscience and remote sensing , ieee transactions on _ , vol .41 , no . 5 , pp .11361142 , may 2003 .m. dufour , l. miller , and p. galatsanos , `` template matching based object recognition with unknown geometric parameters , '' _ image processing , ieee transactions on _ , vol .11 , no .12 , pp . 13851396 , 2002 .m. uss , b. vozel , v. dushepa , v. komjak , and k. chehdi , `` a precise lower bound on image subpixel registration accuracy , '' _ geoscience and remote sensing , ieee transactions on _ , vol .52 , no . 6 , pp . 33333345 , june 2014 .y. ahn and i. howat , `` efficient automated glacier surface velocity measurement from repeat images using multi - image / multichip and null exclusion feature tracking , '' _ geoscience and remote sensing , ieee transactions on _ , vol .49 , no . 8 , pp . 28382846 , aug 2011 .x. hu , y. li , j. shan , j. zhang , and y. zhang , `` road centerline extraction in complex urban scenes from lidar data based on multiple features , '' _ geoscience and remote sensing , ieee transactions on _ , vol .52 , no . 11 , pp . 74487456 , nov 2014 .s. gibbons and f. ringdal , `` seismic monitoring of the north korea nuclear test site using a multichannel correlation detector , '' _ geoscience and remote sensing , ieee transactions on _ , vol .50 , no . 5 , pp .18971909 , may 2012 .h. zhang , k. sun , and w. li , `` object - oriented shadow detection and removal from urban high - resolution remote sensing images , '' _ geoscience and remote sensing , ieee transactions on _ , vol .52 , no . 11 , pp . 69726982 , nov 2014 .g. vivone , p. addesso , r. conte , m. longo , and r. restaino , `` a class of cloud detection algorithms based on a map - mrf approach in space and time , '' _ geoscience and remote sensing , ieee transactions on _ , vol .52 , no . 8 , pp .51005115 , aug 2014 .a. berg and l. eriksson , `` investigation of a hybrid algorithm for sea ice drift measurements using synthetic aperture radar images , '' _ geoscience and remote sensing , ieee transactions on _ , vol .52 , no .50235033 , aug 2014 .v. n. radhika , b. kartikeyan , b. gopala krishna , s. chowdhury , and p. srivastava , `` robust stereo image matching for spaceborne imagery , '' _ geoscience and remote sensing , ieee transactions on _ , vol .45 , no . 9 , pp . 29933000 , sept 2007 .j. piedra - fernandez , g. ortega , j. wang , and m. canton - garbin , `` fuzzy content - based image retrieval for oceanic remote sensing , '' _ geoscience and remote sensing , ieee transactions on _ , vol .52 , no . 9 , pp . 54225431 , sept 2014 .d. espinoza - molina and m. datcu , `` earth - observation image retrieval based on content , semantics , and metadata , '' _ geoscience and remote sensing , ieee transactions on _ , vol .51 , no . 11 , pp .51455159 , nov 2013 .w. ouyang , f. tombari , s. mattoccia , l. di stefano , and w .- k .cham , `` performance evaluation of full search equivalent pattern matching algorithms , '' _ pattern analysis and machine intelligence , ieee transactions on _ , vol .34 , no . 1 ,pp . 127143 , 2012 .f. tombari , s. mattoccia , and l. di stefano , `` full - search - equivalent pattern matching with incremental dissimilarity approximations , '' _ pattern analysis and machine intelligence , ieee transactions on _ , vol .31 , no . 1 ,pp . 129141 , 2009 .wei and s .- h .lai , `` fast template matching based on normalized cross correlation with adaptive multilevel winner update , '' _ image processing , ieee transactions on _ , vol .17 , no . 11 , pp . 22272235 , 2008 .b. montrucchio and d. quaglia , `` new sorting - based lossless motion estimation algorithms and a partial distortion elimination performance analysis , '' _ ieee trans .circuits syst .video technol ._ , vol . 15 , no . 2 , pp .210220 , 2005 .

Abstract: in remote sensing image - blurring is induced by many sources such as atmospheric scatter , optical aberration , spatial and temporal sensor integration . the natural blurring can be exploited to speed up target search by fast template matching . in this paper , we synthetically induce additional non - uniform blurring to further increase the speed of the matching process . to avoid loss of accuracy , the amount of synthetic blurring is varied spatially over the image according to the underlying content . we extend transitive algorithm for fast template matching by incorporating controlled image blur . to this end we propose an efficient group size ( egs ) algorithm which minimizes the number of similarity computations for a particular search image . a larger efficient group size guarantees less computations and more speedup . egs algorithm is used as a component in our proposed optimizing auto - correlation ( opta ) algorithm . in opta a search image is iteratively non - uniformly blurred while ensuring no accuracy degradation at any image location . in each iteration efficient group size and overall computations are estimated by using the proposed egs algorithm . the opta algorithm stops when the number of computations can not be further decreased without accuracy degradation . the proposed algorithm is compared with six existing state of the art exhaustive accuracy techniques using correlation coefficient as the similarity measure . experiments on satellite and aerial image datasets demonstrate the effectiveness of the proposed algorithm .   fast template matching , fast pattern matching , large search space , transitivity of correlation , auto - correlation