The present paper considers leveraging network topology information to improve the convergence rate of ADMM for decentralized optimization, where networked nodes work collaboratively to minimize the objective. Such problems can be solved efficiently using ADMM via decomposing the objective into easier subproblems. Properly exploiting network topology can significantly improve the algorithm performance. Hybrid ADMM explores the direction of exploiting node information by taking into account node centrality but fails to utilize edge information. This paper fills the gap by incorporating both node and edge information and provides a novel convergence rate bound for decentralized ADMM that explicitly depends on network topology. Such a novel bound is attainable for certain class of problems, thus tight. The explicit dependence further suggests possible directions to optimal design of edge weights to achieve the best performance. Numerical experiments show that simple heuristic methods could achieve better performance, and also exhibits robustness to topology changes.
In the setting of distributed averaging, the optimal weights and step size are considered in @cite_20 . Decentralized ADMM over weighted graphs was also considered @cite_36 , which offers the flexibility to optimize convergence rate by tuning weights. The importance of edges was incorporated into hybrid ADMM as well @cite_3 and its convergence rate bound was shown to be related to spectral properties of the communication graph.
@cite_36 @cite_3 @cite_20
2009531227 2068628888 2919745732 2763281404
We derive the optimal step-size and over-relaxation parameter that minimizes the convergence time of two ADMM-based algorithms for distributed averaging. Our study shows that the convergence times for given step-size and over-relaxation parameters depend on the spectral properties of the normalized Laplacian of the underlying communication graph. Motivated by this, we optimize the edge-weights of the communication graph to improve the convergence speed even further. The performance of the ADMM algorithms with our parameter selection are compared with alternatives from the literature in extensive numerical simulations on random graphs. This paper addresses the optimal scaling of the ADMM method for distributed quadratic programming. Scaled ADMM iterations are first derived for generic equality-constrained quadratic problems and then applied to a class of distributed quadratic problems. In this setting, the scaling corresponds to the step-size and the edge-weights of the underlying communication graph. We optimize the convergence factor of the algorithm with respect to the step-size and graph edge-weights. Explicit analytical expressions for the optimal convergence factor and the optimal step-size are derived. Numerical simulations illustrate our results. Distributed optimization has gained popularity in many areas because it can to cope with the increasing volume of data, and the corresponding demand for computational resources. A popular distributed solver is the alternating direction method of multipliers (ADMM). Fully decentralized (D) ADMM arises when node-to-node communications have to abide by the underlying network connectivity. DADMM’s convergence however, slows down as the network diameter grows large. To deal with this challenge, the recently proposed hybrid (H) ADMM provides considerable performance boost over DADMM by exploiting local topology information. But HADMM only applies to unweighted graphs. The present contribution develops a weighted hybrid (WH) consensus-based ADMM approach that can deal with weighted graphs. The resultant scheme further improves the performance of HADMM through graph-aware weight tuning. Theoretical analysis offers convergence guarantees and establishes linear convergence rate, while numerical tests on various graphs demonstrate the WHADMM merits. When solving consensus optimization problems over a graph, there is often an explicit characterization of the convergence rate of Gradient Descent (GD) using the spectrum of the graph Laplacian. The same type of problems under the Alternating Direction Method of Multipliers (ADMM) are, however, poorly understood. For instance, simple but important non-strongly-convex consensus problems have not yet being analyzed, especially concerning the dependency of the convergence rate on the graph topology. Recently, for a non-strongly-convex consensus problem, a connection between distributed ADMM and lifted Markov chains was proposed, followed by a conjecture that ADMM is faster than GD by a square root factor in its convergence time, in close analogy to the mixing speedup achieved by lifting several Markov chains. Nevertheless, a proof of such a claim is is still lacking. Here we provide a full characterization of the convergence of distributed over-relaxed ADMM for the same type of consensus problem in terms of the topology of the underlying graph. Our results provide explicit formulas for optimal parameter selection in terms of the second largest eigenvalue of the transition matrix of the graph's random walk. Another consequence of our results is a proof of the aforementioned conjecture, which interestingly, we show it is valid for any graph, even the ones whose random walks cannot be accelerated via Markov chain lifting.