In this section, we present the full proof of our theoretical results. Our proof of convergence is inspired by [1]} but differs in the following aspects allusive to our specific design: (i) the server updates at each round; (ii) the local training problems are solved inexactly with the level of inexactness captured by the local parameter \(\varepsilon _i\)  as in (REF ); (iii) an additional step size \(\eta \)  for server aggregation is included. 

For ease of exposition, we fix \(\eta ={S^t}/m\)  in the analysis and stack variables as follows:
\(w^{t} = [(w_1^t)^\top ,\dots ,(w_m^t)^\top ]^\top ,y^{t} = [(y_1^t)^\top ,\dots ,(y_m^t)^\top ]^\top \in \mathbb {R}^{md}\) . We define the virtual dual update as follows:
\({\nabla _{w_i} \mathcal {L}_i(\widehat{w}_i^{t+1},y_i^t,\theta ^t)}^2 \le \varepsilon _i, \\\widehat{y}_i^{t+1} = y_i^t+\rho (\widehat{w}_i^{t+1}-\theta ^t).\) 
