@inproceedings{
mishler2023active,
title={Active Learning with Missing Not At Random Outcomes},
author={Alan Mishler and Mohsen Ghassemi and Alec Koppel and Sumitra Ganesh},
booktitle={NeurIPS 2023 Workshop on Adaptive Experimental Design and Active Learning in the Real World},
year={2023},
url={https://openreview.net/forum?id=CzdSCFOG1n}
}
When outcomes in training data are missing not at random (MNAR), predictors that are trained on that data can be arbitrarily biased. In some cases, however, batches of missing outcomes can be recovered at some cost, giving rise to a pool-based active learning setting. Previous active learning approaches implicitly treat all labeled data as having come from the same distribution, whereas in the MNAR setting, the training data and the initial unlabeled pool have different distributions. We propose MNAR-Aware Active Learning (MAAL), an active learning procedure that takes this into account and takes advantage of information that the missingness indicator carries about the outcome. We additionally consider acquisition functions that are attuned to the MNAR setting. Experiments on a large set of classification benchmark datasets demonstrate the benefits of our proposed approach over standard active and passive learning approaches.
Active Learning with Missing Not At Random Outcomes
Alan Mishler Mohsen Ghassemi Alec Koppel Sumitra Ganesh