HANF: Hyperparameter And Neural Architecture Search in Federated Learning
Automated machine learning (AutoML) is an important step to make machine
learning models being widely applied to solve real world problems. Despite
numerous research advancement, machine learning methods are not fully utilized
by industries mainly due to their data privacy and security regulations, high
cost involved in storing and computing increasing amount of data at central
location and most importantly lack of expertise. Hence, we introduce a novel
framework, HANF - $\textbf{H}$yperparameter $\textbf{A}$nd $\textbf{N}$eural
architecture search in $\textbf{F}$ederated learning as a step towards building
an AutoML framework for data distributed across several data owner servers
without any need for bringing the data to a central location. HANF jointly
optimizes a neural architecture and non-architectural hyperparameters of a
learning algorithm using gradient-based neural architecture search and
$n$-armed bandit approach respectively in data distributed setting. We show
that HANF efficiently finds the optimized neural architecture and also tunes
the hyperparameters on data owner servers. Additionally, HANF can be applied in
both, federated and non-federated settings. Empirically, we show that HANF
converges towards well-suited architectures and non-architectural
hyperparameter-sets using image-classification tasks.