The use of Bayesian optimisation (BO) can alleviate the issue of high computational cost [1]}, [2]}. The framework of BO is concluded in the sequential model-based algorithm [3]} that iteratively exploits historical data to fit surrogate models or other transformations, while the most promising candidate is drawn according to a predefined criterion (i.e. acquisition function). The work presented in [4]}, [5]}, [6]}, [7]} makes use of Gaussian processes (GPs) to approximate the distribution over the objective function, and the candidate is selected by the acquisition function: probability of improvement (PI) or expected improvement (EI). In contrast, TPE [4]} employs Parzen estimators as a surrogate to directly model promising and unpromising hyperparameters, and the candidate solutions are drawn according to EI that is proportional to the ratio of two density functions. During searching optimal solutions, the acquisition function plays a significant role in balancing exploration and exploitation.
