This problem can be tackled using Bayesian Optimization (BO) [1]}.
BO approximates \(\mathbf {s}^*\)  with a sequence of evaluations, \(y_{1}, y_{2},\ldots , y_t\)  at \(\mathbf {s}_1, \mathbf {s}_2, \ldots , \mathbf {s}_t \in \) , which maximizes an utility metric, with each new \(\mathbf {s}_{t+1}\)  depending on the previous function values. BO achieves this by building a probabilistic surrogate model on \(f\)  based on the set of evaluated points \(\lbrace (\mathbf {s}_i, y_i)\rbrace _{i=1}^{t}\) . At each iteration, a new point is selected and evaluated based on the surrogate model which is then updated to include the new point \((\mathbf {s}_{t+1}, y_{t+1})\) .
BO defines an utility metric called the acquisition function, which gives a score to each \(\mathbf {s} \in \)  by balancing the predicted value and the uncertainty of the prediction for \(\mathbf {s}\) .
The maximization of the acquisition function guides the sequential decision making process and the exploration versus exploitation trade-off: the highest score identifies the next point \(\mathbf {s}_{t+1}\)  to evaluate.
BO is a statistically efficient black-box optimization approach when considering the number of necessary function evaluations [2]}. It is, thus, especially well-suited to solve problems where we can only perform a limited number of function evaluations, such as the ones found in robotics.
