Ensemble learning methods are known to be effective for learning large-scale models, which combine several base predictors or experts to generate more accurate ensemble model. However, conventional ensemble learning methods (see e.g. [1]}) are not directly applicable for communication efficient federated learning. To adapt ensemble learning to federated setting, FedBoost has been proposed by [2]}, where the server constructs a model by combining a subset of pre-trained models. In this way, the server only needs to transmit a subset of pre-trained models to the clients at each learning round. Pre-trained models can be trained on publicly available data without observing clients' data. FedBoost imposes a budget constraint on the number of pre-trained models that can be transmitted to the clients; however it cannot guarantees that the budget constraint is not violated at each learning round. Instead, it only guarantees the expected cost of model transmission satisfies the budget constraint. Moreover, ensemble learning techniques has been employed in vertical federated learning [3]}.
