This bilevel program formulation is general and subsumes the problems of test set model generalization, Stackelberg competition [1]}, meta-learning [2]} and few-shot learning [3]}.
The quality of the optimization result, \({z^\star }\) , of the objective \(f_L\)  is quantified via the upper objective, \(f_U\) . In the example of a statistical model, \(f_L\)  represents the loss on the train dataset and \(f_U\)  the loss on the test dataset. Solving the bilevel program requires selecting parameters \(p\)  which produce such \({z^\star }\)  that together lead to the minimal upper level loss \(f_U\) .When \({z^\star }\)  itself has an interpretation as “parameters”, e.g., in model learning, \(p\)  may be referred to as “hyperparameters”; we will refer to \(p\)  as parameters throughout the remainder of this domain-agnostic work.
