The above observation inspires us to ask that if there exists a more general framework, which does not require the symmetry of weight matrices. In [1]} and [2]}, gradient tracking algorithms
[3]}, [4]}, [5]}, [6]}, [7]}, [8]} are seen as primal-dual methods, where the symmetry of weight matrices is further assumed. However, the convergence of all these gradient tracking methods do not rely on the symmetry of weight matrices. When the symmetry of weight matrices is absent, in fact, we can not treat gradient tracking algorithms as primal-dual methods, which is also pointed out in [3]}. Different from [1]} and [2]}, our framework is built from a pure primal perspective and does not rely on the symmetry of weight matrices. The key in developing the proposed framework is that we regard most existing distributed algorithms with constant step-sizes as gradient tracking methods but with different kinds of tracking mechanisms: explicit tracking and implicit tracking. To be more specific, the tracking of the global gradient exists generally in most distributed algorithms with constant step-sizes. Though some algorithms do not have the explicit gradient tracking structure [12]}, [13]}, [14]}, [15]}, [16]}, they still possess certain implicit structure that can track the global gradient, which we call the implicit tracking mechanism. The idea originates from our earlier work [17]}, where the implicit tracking mechanism is proposed and further used to design efficient algorithms for distributed constraint-coupled optimization problems.
Therefore, our framework can be seen as a generalized gradient tracking method, which is the reason we call it unified gradient tracking (UGT).
