Learning Debiased Classifier with Biased Committee
Neural networks are prone to be biased towards spurious correlations between
classes and latent attributes exhibited in a major portion of training data,
which ruins their generalization capability. This paper proposes a new method
for training debiased classifiers with no spurious attribute label. The key
idea of the method is to employ a committee of classifiers as an auxiliary
module that identifies bias-conflicting data, i.e., data without spurious
correlations, and assigns large weights to them when training the main
classifier. The committee is learned as a bootstrapped ensemble so that a
majority of its classifiers are biased as well as being diverse, and
intentionally fail to predict classes of bias-conflicting data accordingly. The
consensus within the committee on prediction difficulty thus provides a
reliable cue for identifying and weighting bias-conflicting data. Moreover, the
committee is also trained with knowledge transferred from the main classifier
so that it gradually becomes debiased along with the main classifier and
emphasizes more difficult data as training progresses. On five real-world
datasets, our method outperforms existing methods using no spurious attribute
label like ours and even surpasses those relying on bias labels occasionally.