On the reusability of samples in active learning
An interesting but not extensively studied question in active learning is
that of sample reusability: to what extent can samples selected for one learner
be reused by another? This paper explains why sample reusability is of
practical interest, why reusability can be a problem, how reusability could be
improved by importance-weighted active learning, and which obstacles to
universal reusability remain. With theoretical arguments and practical
demonstrations, this paper argues that universal reusability is impossible.
Because every active learning strategy must undersample some areas of the
sample space, learners that depend on the samples in those areas will learn
more from a random sample selection. This paper describes several experiments
with importance-weighted active learning that show the impact of the
reusability problem in practice. The experiments confirmed that universal
reusability does not exist, although in some cases -- on some datasets and with
some pairs of classifiers -- there is sample reusability. Finally, this paper
explores the conditions that could guarantee the reusability between two
classifiers.