Few-Shot Non-Parametric Learning with Deep Latent Variable Model
Most real-world problems that machine learning algorithms are expected to
solve face the situation with 1) unknown data distribution; 2) little
domain-specific knowledge; and 3) datasets with limited annotation. We propose
Non-Parametric learning by Compression with Latent Variables (NPC-LV), a
learning framework for any dataset with abundant unlabeled data but very few
labeled ones. By only training a generative model in an unsupervised way, the
framework utilizes the data distribution to build a compressor. Using a
compressor-based distance metric derived from Kolmogorov complexity, together
with few labeled data, NPC-LV classifies without further training. We show that
NPC-LV outperforms supervised methods on all three datasets on image
classification in low data regime and even outperform semi-supervised learning
methods on CIFAR-10. We demonstrate how and when negative evidence lowerbound
(nELBO) can be used as an approximate compressed length for classification. By
revealing the correlation between compression rate and classification accuracy,
we illustrate that under NPC-LV, the improvement of generative models can
enhance downstream classification accuracy.