Bayesian Optimization. We also investigate the effectiveness of Bayesian Optimization (BO) in yielding global, Pareto optimal solutions.
Given the cost of running several growth sequences is high, we investigate if BO can yield useful network size-performance tradeoffs, even under low budget scenarios.
We use the Optuna framework [1]}, which uses the NSGA-II algorithm [2]} for multi-objective Bayesian Optimization.
To restrict the large optimization space of our benchmarks for a low budget BO setting, we assume a fixed task arrival order, keep a shared growth coefficient across tasks, and sample coefficient \(\delta \)  from a discrete uniform distribution \(\mathcal {U}\lbrace 0,1\rbrace \)  with steps of \(0.05\) .
We perform 20 grow-sequence trials, each trial taking about 11 minutes on a single NVIDIA Tesla K80 GPU.
