Black-box optimisation [1]}, [2]} implies that the details of the objective function and its gradient information is unknown during searching optimal solutions. In HPO, hyperparameters can be evaluated on the objective function, but we do not have access to any information about the model. Random search [3]} and grid search are two simple and commonly used methods for HPO. Bergstra et al. [4]} hold the point that random search is more efficient than grid search given a fixed limited computation budget for HPO. However, facing the problems with expensive evaluations, it is very challenging for random search and grid search to achieve satisfactory performance.
