A geometric framework for outlier detection in high-dimensional data
Outlier or anomaly detection is an important task in data analysis. We
discuss the problem from a geometrical perspective and provide a framework that
exploits the metric structure of a data set. Our approach rests on the manifold
assumption, i.e., that the observed, nominally high-dimensional data lie on a
much lower dimensional manifold and that this intrinsic structure can be
inferred with manifold learning methods. We show that exploiting this structure
significantly improves the detection of outlying observations in
high-dimensional data. We also suggest a novel, mathematically precise, and
widely applicable distinction between distributional and structural outliers
based on the geometry and topology of the data manifold that clarifies
conceptual ambiguities prevalent throughout the literature. Our experiments
focus on functional data as one class of structured high-dimensional data, but
the framework we propose is completely general and we include image and graph
data applications. Our results show that the outlier structure of
high-dimensional and non-tabular data can be detected and visualized using
manifold learning methods and quantified using standard outlier scoring methods
applied to the manifold embedding vectors.