Bilevel Optimization Theory:  In the past few years, there have been a flurry of new algorithms to solve the single-objective bilevel optimization problem (REF ). [1]} and [2]} propose BSA and stocBiO algorithms respectively that solve (REF ) in a double-loop manner, meaning that for each outer step, multiple inner SGD steps are taken to better approximate \(y^*(x)\) . These works consider the setting where the outer problem is nonconvex and the inner problem is strongly convex. One drawback to double loop algorithms is that selecting step-sizes and termination criterion for inner loop steps is tricky. Motivated by this, [3]}, [4]}, and [5]} propose STABLE, TTSA, and ALSET, respectively, as single-loop algorithms that solve (REF ).
