Label and Distribution-discriminative Dual Representation Learning for Out-of-Distribution Detection
To classify in-distribution samples, deep neural networks learn
label-discriminative representations, which, however, are not necessarily
distribution-discriminative according to the information bottleneck. Therefore,
trained networks could assign unexpected high-confidence predictions to
out-of-distribution samples drawn from distributions differing from that of
in-distribution samples. Specifically, networks extract the strongly
label-related information from in-distribution samples to learn the
label-discriminative representations but discard the weakly label-related
information. Accordingly, networks treat out-of-distribution samples with
minimum label-sensitive information as in-distribution samples. According to
the different informativeness properties of in- and out-of-distribution
samples, a Dual Representation Learning (DRL) method learns
distribution-discriminative representations that are weakly related to the
labeling of in-distribution samples and combines label- and
distribution-discriminative representations to detect out-of-distribution
samples. For a label-discriminative representation, DRL constructs the
complementary distribution-discriminative representation by an implicit
constraint, i.e., integrating diverse intermediate representations where an
intermediate representation less similar to the label-discriminative
representation owns a higher weight. Experiments show that DRL outperforms the
state-of-the-art methods for out-of-distribution detection.