We consider stochastic processes on complete, locally compact tree-like metric spaces (T,r)(T,r) on their “natural scale” with boundedly finite speed measure νν. Given a triple (T,r,ν)(T,r,ν) such a speed-νν motion on (T,r)(T,r) can be characterized as the unique strong Markov process which if restricted to compact subtrees satisfies for all x,y∈Tx,y∈T and all positive, bounded measurable ff, Ex[∫τy0dsf(Xs)]=2∫Tν(dz)r(y,c(x,y,z))f(z)<∞, Ex[∫0τydsf(Xs)]=2∫Tν(dz)r(y,c(x,y,z))f(z)<∞, where c(x,y,z)c(x,y,z) denotes the branch point generated by x,y,zx,y,z. If (T,r)(T,r) is a discrete tree, XX is a continuous time nearest neighbor random walk which jumps from vv to v′∼vv′∼v at rate 12⋅(ν( v )⋅r(v,v′))−112⋅(ν( v )⋅r(v,v′))−1. If (T,r)(T,r) is path-connected, XX has continuous paths and equals the νν-Brownian motion which was recently constructed in [Trans. Amer. Math. Soc. 365 (2013) 3115–3150]. In this paper, we show that speed-νnνn motions on (Tn,rn)(Tn,rn) converge weakly in path space to the speed-νν motion on (T,r)(T,r) provided that the underlying triples of metric measure spaces converge in the Gromov–Hausdorff-vague topology introduced in [Stochastic Process. Appl. 126 (2016) 2527–2553].
It was shown in Remark 3.1 in @cite_17 that the unique @math -symmetric strong Markov process associated with @math is the speed- @math motion on @math
@cite_17
2002850151 2134711723 1991238514 2911793117
A discrete uncoupling process for finite spaces is introduced, called the Markov Cluster Process or the MCL process. The process is the engine for the graph clustering algorithm called the MCL algorithm. The MCL process takes a stochastic matrix as input, and then alternates expansion and inflation, each step defining a stochastic matrix in terms of the previous one. Expansion corresponds with taking the @math th power of a stochastic matrix, where @math . Inflation corresponds with a parametrized operator @math , @math , that maps the set of (column) stochastic matrices onto itself. The image @math is obtained by raising each entry in @math to the @math th power and rescaling each column to have sum 1 again. In practice the process converges very fast towards a limit that is invariant under both matrix multiplication and inflation, with quadratic convergence around the limit points. The heuristic behind the process is its expected behavior for (Markov) graphs possessing cluster structure. The process is typically applied to the matrix of random walks on a given graph @math , and the connected components of (the graph associated with) the process limit generically allow a clustering interpretation of @math . The limit is in general extremely sparse and iterands are sparse in a weighted sense, implying that the MCL algorithm is very fast and highly scalable. Several mathematical properties of the MCL process are established. Most notably, the process (and algorithm) iterands posses structural properties generalizing the mapping from process limits onto clusterings. The inflation operator @math maps the class of matrices that are diagonally similar to a symmetric matrix onto itself. The phrase diagonally positive semi-definite (dpsd) is used for matrices that are diagonally similar to a positive semi-definite matrix. For @math and for @math a stochastic dpsd matrix, the image @math is again dpsd. Determinantal inequalities satisfied by a dpsd matrix @math imply a natural ordering among the diagonal elements of @math , generalizing the mapping of process limits onto clusterings. The spectrum of @math is of the form @math , where @math is the number of endclasses of the ordering associated with @math , and @math is the dimension of @math . This attests to the uncoupling effect of the inflation operator. We consider a symmetric random walk on a connected graph, where each edge is labeled with the probability of transition between the two adjacent vertices. The associated Markov chain has a uniform equilibrium distribution; the rate of convergence to this distribution, i.e., the mixing rate of the Markov chain, is determined by the second largest eigenvalue modulus (SLEM) of the transition probability matrix. In this paper we address the problem of assigning probabilities to the edges of the graph in such a way as to minimize the SLEM, i.e., the problem of finding the fastest mixing Markov chain on the graph. We show that this problem can be formulated as a convex optimization problem, which can in turn be expressed as a semidefinite program (SDP). This allows us to easily compute the (globally) fastest mixing Markov chain for any graph with a modest number of edges (say, @math ) using standard numerical methods for SDPs. Larger problems can be solved by exploiting various types of symmetry and structure in the problem, and far larger problems (say, 100,000 edges) can be solved using a subgradient method we describe. We compare the fastest mixing Markov chain to those obtained using two commonly used heuristics: the maximum-degree method, and the Metropolis--Hastings algorithm. For many of the examples considered, the fastest mixing Markov chain is substantially faster than those obtained using these heuristic methods. We derive the Lagrange dual of the fastest mixing Markov chain problem, which gives a sophisticated method for obtaining (arbitrarily good) bounds on the optimal mixing rate, as well as the optimality conditions. Finally, we describe various extensions of the method, including a solution of the problem of finding the fastest mixing reversible Markov chain, on a fixed graph, with a given equilibrium distribution. The behavior of the stationary distribution of a Markovian process of the form [( * ) X_ n + 1 ^a = _S (X_n^a - aY_n^a ) ] as a tends to zero is studied. The process @math is a constant step-size (constant gain) constrained stochastic approximation process, with @math being the projection onto the convex set of constraints S. If the gains a are held constant, the process @math does not converge to the point of solution but the stationary distribution of @math converges to the unit mass at the solution, if @math . Properly normalized the stationary distribution converges to a nondegenerated limit. This limit depends on the smoothness of the feasible set S in the neighborhood of the solution (in particular on the dimension of the largest linear subspace contained in the tangential cone). Consider a Markov decision process (MDP) that admits a set of state-action features, which can linearly express the process's probabilistic transition model. We propose a parametric Q-learning algorithm that finds an approximate-optimal policy using a sample size proportional to the feature dimension @math and invariant with respect to the size of the state space. To further improve its sample efficiency, we exploit the monotonicity property and intrinsic noise structure of the Bellman operator, provided the existence of anchor state-actions that imply implicit non-negativity in the feature space. We augment the algorithm using techniques of variance reduction, monotonicity preservation, and confidence bounds. It is proved to find a policy which is @math -optimal from any initial state with high probability using @math sample transitions for arbitrarily large-scale MDP with a discount factor @math . A matching information-theoretical lower bound is proved, confirming the sample optimality of the proposed method with respect to all parameters (up to polylog factors).