There are many state-of-the-art HPO approaches with different search strategies. For example, TPE [1]} and BOHB [2]} are proposed based on Bayesian optimisation (BO) which are suitable for the functions that are expensive to evaluate. However, the former uses density functions to model two sets (good and bad) of hyperparameters, instead of using Gaussian process as in [3]}, [4]} as a standard surrogate model to approximate the distribution over the objective function. Furthermore, BOHB combines Bayesian optimisation and bandit-based methods to ensure good performance at anytime and speed up the process towards optimal solutions. In general, BO-based methods balance the exploration and exploitation by acquisition function. On the contrary, CMA-ES [5]} and HESGA [6]} are evolutionary approaches that exploit high-quality individuals to guide the generation of promising offspring, while they rely on evolutionary operators to explore search space. CMA-ES employs an adaptive multivariate distribution to sample individuals, and HESGA uses a genetic algorithm that relies on crossover and mutation operators to generate better individuals. To deepen the work of HESGA [6]}, our research focuses on improving the exploration capability of the mutation operator.
