We propose a new way of interpreting active learning methods by analyzing the information lost' upon sampling a random variable. We use some recent analytical developments of these losses to formally prove that facility location methods reduce these losses under mild assumptions, and to derive a new data dependent bound on information losses that can be used to evaluate other active learning methods. We show that this new bound is extremely tight to experiment, and further show that the bound has a decent predictive power for classification accuracy.
On the unsupervised active learning side, two works in particular are relevant to our work. The first @cite_3 provides an objective function for choosing points in a linear (or kernelized linear) regression problem. The goal of the objective function is to minimize the error variances on the unlabelled points in the linear regression setting. While its derivation is not as general to the one provided in this work - relating only to linear or kernel-linear regression problems - its objective function ends up being a variant of the one that we derive. In this sense, our work strengthens this method by showing that a variant of its objective targets information losses directly.
@cite_3
2114188922
This paper considers the problem of selecting the most informative experiments x to get measurements y for learning a regression model y = f(x). We propose a novel and simple concept for active learning, transductive experimental design, that explores available unmeasured experiments (i.e., unlabeled data) and has a better scalability in comparison with classic experimental design methods. Our in-depth analysis shows that the new method tends to favor experiments that are on the one side hard-to-predict and on the other side representative for the rest of the experiments. Efficient optimization of the new design problem is achieved through alternating optimization and sequential greedy search. Extensive experimental results on synthetic problems and three real-world tasks, including questionnaire design for preference learning, active learning for text categorization, and spatial sensor placement, highlight the advantages of the proposed approaches.