Unlike previous works, L2A deploys selective prediction models, which are models that can choose to abstain from making a prediction.
This abstention option improves the reliability of ML algorithms in settings where abstaining is better than giving a misleading prediction [1]}, [2]}, [3]}, [4]}, [5]}.
Assuming there is some fixed cost \(\delta > 0\)  associated with abstaining, we show that the average risk of L2A never exceeds \(\delta + \epsilon \)  for some non-inferiority margin \(\epsilon > 0\) .
This implies that L2A can safely search over unsafe approval strategies as long as the abstention option is available.
We analyze the operating characteristics of L2A in both simulation studies and real-world analyses.
In the presence of severe distributional shifts, L2A learns to approve modifications cautiously and abstain more; When the data are (relatively) stationary, it learns to approve modifications quickly and abstain less.
