Recently, [1]} presented an improved version of convergence analysis which matches FedDyn rate for convex functions. The comparison is given in Remark 11 of [1]}. For the nonconvex setting, the convergence rate of SCAFFOLD builds upon 'lag in the control variates' for each round, (Lemma 17 page 32). We construct an example with quadratic functions and numerically evaluate Lemma 17. However, we can not empirically verify the claim in Lemma 17 Please see the example in https://github.com/alpemreacar/FedDynClarification/blob/main/Lemma17_SCAFFOLD.pdf.. Since Lemma 17 is essential for the convergence rate, it is not clear if SCAFFOLD gets better rate for nonconvex functions compared to FedDyn.
