Consider the case where there are 100 clients performing a regression task.
The server stores 22 pre-trained models, including kernel based regression models with 5 Gaussian kernels, 5 Laplacian kernels, 5 polynomial kernels, 5 sigmoid kernels and 2 feed-forward neural networks. The bandwidth of Gaussian, Laplacian kernels, and the slope of sigmoid kernels are \(0.01, 0.1, 1, 10, 100\) . Also, the degree of polynomial kernels are \(1, 2, 3, 4, 5\) . The feedforward neural networks have 1 and 2 hidden layers respectively, where each hidden layer consists of 25 neurons with ReLU activation functions. Each model is trained with \(10\%\)  of each dataset. Furthermore, the cost of sending each model is considered to be the number of parameters associated with model divided by the number of parameters associated with the model with maximum number of parameters. Hence, the maximum cost of sending a model is 1. The budget for sending models is \(B=3\) .
The learning rate \(\eta \)  and exploration rate \(\xi \)  for all methods are set to be \(\frac{1}{\sqrt{T}}\) . The performance of ensemble federated learning methods is evaluated based on mean square error (MSE) at learning round \(t\)  defined as
\(\text{MSE}_t = \frac{1}{t}\sum _{\tau =1}^{t}{\frac{1}{|{\mathbb {C}}_\tau |}\sum _{i \in {\mathbb {C}}_\tau }{(\hat{y}_{i,\tau }-y_{i,\tau })^2}} \) 
where \(\hat{y}_{i,t}\)  denote the prediction made by the \(i\) -th client at learning round \(t\) . In order to derive dominating sets for feedback graphs in Algorithm REF , greedy set cover algorithm is employed [1]}. Note that in this experiments it is assumed that clients are not able to store their observed data in batch. Therefore, FedBoost implementation is modified to cope with this situation for fair comparison. Specifically, at learning round \(t\) , the \(i\) -th client transmits its update with respect to newly observed sample \({\mathbf {x}}_{i,t}\)  instead of the whole batch of data.
<TABLE><FIGURE>