We find the optimal weights of the class model logistic regressions using sci-kit learn's LogisticRegression with the L-BFGS [1]} optimizer. For baselines, we chose Federated Ensemble Distillation (FedD) and Federated Averaging (FedAVG), which we pretrain (+P) in the same fashion as FedAUXfdp. For FedAUXfdp and FedD+P, the full server model is trained for 10 distillation epochs using the Adam optimizer with a learning rate of \(5\cdot 10^{-5}\)  and a batch size of 128. For FedAVG+P, the average of the weights of the clients' logistic regressions is used as a classification head on top of the frozen feature extractor on the server.
