Fair and Green Hyperparameter Optimization via Multi-objective and Multiple Information Source Bayesian Optimization
There is a consensus that focusing only on accuracy in searching for optimal
machine learning models amplifies biases contained in the data, leading to
unfair predictions and decision supports. Recently, multi-objective
hyperparameter optimization has been proposed to search for machine learning
models which offer equally Pareto-efficient trade-offs between accuracy and
fairness. Although these approaches proved to be more versatile than
fairness-aware machine learning algorithms -- which optimize accuracy
constrained to some threshold on fairness -- they could drastically increase
the energy consumption in the case of large datasets. In this paper we propose
FanG-HPO, a Fair and Green Hyperparameter Optimization (HPO) approach based on
both multi-objective and multiple information source Bayesian optimization.
FanG-HPO uses subsets of the large dataset (aka information sources) to obtain
cheap approximations of both accuracy and fairness, and multi-objective
Bayesian Optimization to efficiently identify Pareto-efficient machine learning
models. Experiments consider two benchmark (fairness) datasets and two machine
learning algorithms (XGBoost and Multi-Layer Perceptron), and provide an
assessment of FanG-HPO against both fairness-aware machine learning algorithms
and hyperparameter optimization via a multi-objective single-source
optimization algorithm in BoTorch, a state-of-the-art platform for Bayesian
Optimization.