@inproceedings{
ochiai2023active,
title={Active Testing of Binary Classification Model Using Level Set Estimation},
author={Takuma Ochiai and Keiichiro Seno and Kota Matsui and Satoshi Hara},
booktitle={NeurIPS 2023 Workshop on Adaptive Experimental Design and Active Learning in the Real World},
year={2023},
url={https://openreview.net/forum?id=gkChsof0Rg}
}
In this study, we propose a method for estimating the test loss in binary classification model with minimal labeling of the test data. The central idea of the proposed method is to reduce the problem of test loss estimation to the problem of level set estimation for the loss function. This reduction allows us to achieve sequential test loss estimation through iterative labeling using active learning methods for level set estimation.  
Through multiple dataset experiments, we confirmed that the proposed method is effective for evaluating binary classification models and allows for test loss estimation with fewer labeled samples compared to existing methods.
Active Testing of Binary Classification Model Using Level Set Estimation
Takuma Ochiai Keiichiro Seno Kota Matsui Satoshi Hara