Integrated Weak Learning
We introduce Integrated Weak Learning, a principled framework that integrates
weak supervision into the training process of machine learning models. Our
approach jointly trains the end-model and a label model that aggregates
multiple sources of weak supervision. We introduce a label model that can learn
to aggregate weak supervision sources differently for different datapoints and
takes into consideration the performance of the end-model during training. We
show that our approach outperforms existing weak learning techniques across a
set of 6 benchmark classification datasets. When both a small amount of labeled
data and weak supervision are present the increase in performance is both
consistent and large, reliably getting a 2-5 point test F1 score gain over
non-integrated methods.