
We offer a new and unified perspective for understanding existing distributed optimization algorithms with constant step-sizes, i.e., most distributed optimization algorithms with constant step-sizes can be seen as gradient tracking methods in essence, which leads to the birth of UGT.

UGT can unify most existing distributed algorithms with constant step-sizes, without assuming the symmetry of weight matrices. In this sense, UGT is more general than the primal-dual frameworks proposed in [1]} and [2]}. Meanwhile, the permission of asymmetric weight matrices implies that UGT can be applied to digraphs, which is remarkably different from [1]} and [2]}. Furthermore, the linear convergence of UGT can be guaranteed when \(f_i(x)\)  is smooth and \(F(x)\)  is strongly convex, without the need of the strong convexity even convexity of \(f_i(x)\) . However, the linear convergences of the primal-dual frameworks proposed in [1]} and [2]} both rely on the strong convexity of \(f_i(x)\) .

As a generalized gradient tracking method, we can easily obtain lots of different versions of UGT, benefiting from its special structure. In numerical experiments, it is found that some versions of UGT have far better performance than classical gradient tracking algorithms.

