In this paper, we study how to design pACPs that control the online error-rate in the presence of distributional shifts.
We define the online error-rate as the cumulative risk.
This quantity has been extensively studied in the online learning literature and can also be viewed as a generalization of the utility function in the adaptive clinical trial literature [1]}, [2]}, [3]} to non-stationary settings.
To give the model developers flexibility in constructing algorithmic modifications, the developer is allowed to be adaptive, i.e. they may change how modifications are generated based on historical data.
Because ML algorithms may induce distributional shifts in unanticipated and undesirable ways, we allow the distributional shifts to be adaptive as well.
For example, ML algorithms can affect clinical decision-making as human experts develop trust or distrust in these systems [4]}, [5]}.
In addition, ML algorithms can induce significant feedback loops in the environments they operate in, and initially fair algorithms can become increasingly unfair [6]}, [7]}, [8]}, [9]}.
