Algorithms. We compare COLD and Dyna-COLD with NIDS [1]}, LEAD [2]}, DQOA [3]}, and CHOCO-SGD [4]}. NIDS involves no compression and serves as a baseline. In all experiments, we fix \(s^k=3\Vert X^1\Vert _\text{max}\cdot 0.99^k\)  in Dyna-COLD. We tune all hyperparameters of these algorithms from the grid \([0.01, 0.05, 0.1, 0.3, 0.5, 0.7, 1, 1.5]\)  (1 hyperparameter for CHOCO-SGD; 2 for COLD, Dyna-COLD, and NIDS; 3 for LEAD and DQOA).
