Hybrid bayesian evolutionary optimization for hyperparameter tuning
In this paper, we present a Hybrid Bayesian Evolutionary tuning algorithm (HBEtune) for tuning machine learning algorithms or evolutionary algorithms, and analyze its performance. HBEtune combines meta EA and Bayesian optimization techniques. As hyperparameter tuning is a noisy, black box optimization problem with expensive target functions, practical tuners must aim to minimize the number of necessary samples. In our method, we guide the EA's recombination operator towards more promising samples by employing the expected improvement acquisition criterion commonly used in Bayesian optimization. The expected improvement is evaluated on a surrogate model using a Gaussian process regression. HBEtune shows generally competitive performance when compared with the state of the art irace tuner. Performance is analyzed across a suite of synthetic and real world benchmark problems.
GECCO Companion