Actively Learning Deep Neural Networks with Uncertainty Sampling Based on Sum-Product Networks
Active learning is popular approach for reducing the amount of data in
training deep neural network model. Its success hinges on the choice of an
effective acquisition function, which ranks not yet labeled data points
according to their expected informativeness. In uncertainty sampling, the
uncertainty that the current model has about a point's class label is the main
criterion for this type of ranking. This paper proposes a new approach to
uncertainty sampling in training a Convolutional Neural Network (CNN). The main
idea is to use feature representation extracted extracted by the CNN as data
for training a Sum-Product Network (SPN). Since SPNs are typically used for
estimating the distribution of a dataset, they are well suited to the task of
estimating class probabilities that can be used directly by standard
acquisition functions such as max entropy and variational ratio. Moreover, we
enhance these acquisition functions by weights calculated with the help of the
SPN model; these weights make the acquisition function more sensitive to the
diversity of conceivable class labels for data points. The effectiveness of our
method is demonstrated in an experimental study on the MNIST, Fashion-MNIST and
CIFAR-10 datasets, where we compare it to the state-of-the-art methods MC
Dropout and Bayesian Batch.