SHELS: Exclusive Feature Sets for Novelty Detection and Continual Learning Without Class Boundaries
While deep neural networks (DNNs) have achieved impressive classification
performance in closed-world learning scenarios, they typically fail to
generalize to unseen categories in dynamic open-world environments, in which
the number of concepts is unbounded. In contrast, human and animal learners
have the ability to incrementally update their knowledge by recognizing and
adapting to novel observations. In particular, humans characterize concepts via
exclusive (unique) sets of essential features, which are used for both
recognizing known classes and identifying novelty. Inspired by natural
learners, we introduce a Sparse High-level-Exclusive, Low-level-Shared feature
representation (SHELS) that simultaneously encourages learning exclusive sets
of high-level features and essential, shared low-level features. The
exclusivity of the high-level features enables the DNN to automatically detect
out-of-distribution (OOD) data, while the efficient use of capacity via sparse
low-level features permits accommodating new knowledge. The resulting approach
uses OOD detection to perform class-incremental continual learning without
known class boundaries. We show that using SHELS for novelty detection results
in statistically significant improvements over state-of-the-art OOD detection
approaches over a variety of benchmark datasets. Further, we demonstrate that
the SHELS model mitigates catastrophic forgetting in a class-incremental
learning setting,enabling a combined novelty detection and accommodation
framework that supports learning in open-world settings