Proximal Parameter \(\rho \). To counter client drift in non-IID data distributions, FedADMM and FedProx both use a quadratic proximal term for the local training problem. The proximal coefficient \(\rho \)  in FedProx has to be carefully tuned to achieve competitive performance across different settings [1]}, [2]}. Such tuning is dependent on system sizes and data distributions, as we also demonstrate in Table REF . Note that the best \(\rho \)  value (0.01) for FedProx in the FMNIST dataset gives the worst performance for MNIST in the 200-client setting. Moreover, the performance of FedProx with respect to \(\rho \)  is not monotone, which makes tuning even more challenging. On the other hand, FedADMM dominates all tested instances of FedProx with fixed \(\rho =0.01\) . This is also supported by our theoretical analysis (Theorem 1 and Remark 1 supports a constant \(\rho \) ).
Additional insights can be gained by a simple dynamic adaptation of \(\rho \)  for FedADMM in Fig. REF . A smaller value (0.01) at initial stages of training allows efficient incorporation of local data when the global model is not informed, while an increase of \(\rho \)  at later stages reduces discrepancies between client models and the global model.
<TABLE><FIGURE>