@inproceedings{
li2023marina,
title={{MARINA} Meets Matrix Stepsizes: Variance Reduced Distributed Non-Convex Optimization},
author={Hanmin Li and Avetik Karagulyan and Peter Richt{\'a}rik},
booktitle={International Workshop on Federated Learning in the Age of Foundation Models in Conjunction with NeurIPS 2023},
year={2023},
url={https://openreview.net/forum?id=YqqWQP8POe}
}
Matrix-stepsized gradient descent algorithms have been demonstrated to exhibit superior efficiency in non-convex optimization compared to their scalar counterparts. 
  The det-CGD algorithm, as introduced by [LKR23], leverages matrix stepsizes to perform compressed gradient descent for non-convex objectives and matrix-smooth problems in a federated manner. 
  The authors establish the algorithm's convergence to a neighborhood of the weighted stationarity point under a convex condition for the symmetric and positive-definite stepsize matrix. 
  In this paper, we propose a variance-reduced version of the det-CGD algorithm, incorporating the MARINA method. 
  Notably, we establish theoretically and empirically, that det-MARINA outperforms both MARINA and the distributed MARINA algorithms
MARINA Meets Matrix Stepsizes: Variance Reduced Distributed Non-Convex Optimization
Hanmin Li Avetik Karagulyan Peter Richtárik