TransBO: Hyperparameter Optimization via Two-Phase Transfer Learning
With the extensive applications of machine learning models, automatic
hyperparameter optimization (HPO) has become increasingly important. Motivated
by the tuning behaviors of human experts, it is intuitive to leverage auxiliary
knowledge from past HPO tasks to accelerate the current HPO task. In this
paper, we propose TransBO, a novel two-phase transfer learning framework for
HPO, which can deal with the complementary nature among source tasks and
dynamics during knowledge aggregation issues simultaneously. This framework
extracts and aggregates source and target knowledge jointly and adaptively,
where the weights can be learned in a principled manner. The extensive
experiments, including static and dynamic transfer learning settings and neural
architecture search, demonstrate the superiority of TransBO over the
state-of-the-arts.