Hybrid Models for Mixed Variables in Bayesian Optimization
We systematically describe the problem of simultaneous surrogate modeling of
mixed variables (i.e., continuous, integer and categorical variables) in the
Bayesian optimization (BO) context. We provide a unified hybrid model using
both Monte-Carlo tree search (MCTS) and Gaussian processes (GP) that
encompasses and generalizes multiple state-of-the-art mixed BO surrogates.
Based on the architecture, we propose applying a new dynamic model selection
criterion among novel candidate families of covariance kernels, including
non-stationary kernels and associated families. Different benchmark problems
are studied and presented to support the superiority of our model, along with
results highlighting the effectiveness of our method compared to most
state-of-the-art mixed-variable methods in BO.