Solving (REF ).
This is a continuous black-box optimization problem with
variables \(\theta ^c\)  and \(\widetilde{\theta }^d\) . Since the algorithms \(\mathbf {z}{}^{(t)}\)  are
fixed in (REF ), \(\widetilde{f}\)  only depends on the
HPs of the chosen algorithms – the active variable set \(S = \lbrace (\theta _{ij}^c, \widetilde{\theta }_{ij}^d): z_{ij}{}^{(t)} = 1\rbrace \) . This splits (REF ) even further into (i) \(\min _{\widetilde{\theta }_{ij}^d \in \widetilde{\mathcal {D}}_{ij}} \Vert  \widetilde{\theta }_{ij}^d -\mathbf {b}_{ij} \Vert _2^2\)  with \(z_{ij} = 0\)  (the inactive set) requiring
a Euclidean projection of \(\mathbf {b}_{ij}\)  onto \(\widetilde{\mathcal {D}}_{ij}\) ,
and (ii) a black-box optimization with a small active continuous variable
set\(S\) .
For the CASH problems we consider in our empirical evalutations, \(| \theta | = | \theta _{ij}^c | + | \widetilde{\theta }_{ij}^d | \approx 100 \)  while the
largest possible active set \(S\)  is less than 15 and typically less than
10.
Solvers such as BO [1]}, direct
search [2]}, or trust-region based derivative-free
optimization [3]} work fairly well.
