A Boosting Algorithm for Positive-Unlabeled Learning
Positive-unlabeled (PU) learning deals with binary classification problems
when only positive (P) and unlabeled (U) data are available. A lot of PU
methods based on linear models and neural networks have been proposed; however,
there still lacks study on how the theoretically sound boosting-style
algorithms could work with P and U data. Considering that in some scenarios
when neural networks cannot perform as good as boosting algorithms even with
fully-supervised data, we propose a novel boosting algorithm for PU learning:
Ada-PU, which compares against neural networks. Ada-PU follows the general
procedure of AdaBoost while two different distributions of P data are
maintained and updated. After a weak classifier is learned on the newly updated
distribution, the corresponding combining weight for the final ensemble is
estimated using only PU data. We demonstrated that with a smaller set of base
classifiers, the proposed method is guaranteed to keep the theoretical
properties of boosting algorithms. In experiments, we showed that Ada-PU
outperforms neural networks on benchmark PU datasets. We also study a
real-world dataset UNSW-NB15 in cyber security and demonstrated that Ada-PU has
superior performance for malicious activity detection.