In many scientific disciplines structures in high-dimensional data have to be found, e.g., in stellar spectra, in genome data, or in face recognition tasks. In this work we present a novel approach to non-linear dimensionality reduction. It is based on fitting K-nearest neighbor regression to the unsupervised regression framework for learning of low-dimensional manifolds. Similar to related approaches that are mostly based on kernel methods, unsupervised K-nearest neighbor (UNN) regression optimizes latent variables w.r.t. the data space reconstruction error employing the K-nearest neighbor heuristic. The problem of optimizing latent neighborhoods is difficult to solve, but the UNN formulation allows the design of efficient strategies that iteratively embed latent points to fixed neighborhood topologies. UNN is well appropriate for sorting of high-dimensional data. The iterative variants are analyzed experimentally.
In the following, we give a short introduction to K-nearest neighbor regression that is basis of the UNN approach. The problem in regression is to predict output values @math to given input values @math based on sets of @math input-output examples @math . The goal is to learn a function @math known as regression function. We assume that a data set consisting of observed pairs @math is given. For a novel pattern @math , KNN regression computes the mean of the function values of its K-nearest neighbors: with set @math containing the indices of the @math -nearest neighbors of @math . The idea of KNN is based on the assumption of locality in data space: In local neighborhoods of @math patterns are expected to have similar output values @math (or class labels) to @math . Consequently, for an unknown @math the label must be similar to the labels of the closest patterns, which is modeled by the average of the output value of the @math nearest samples. KNN has been proven well in various applications, e.g., in detection of quasars in interstellar data sets @cite_6 .
@cite_6
2150709118
We present a classification-based approach to identify quasi-stellar radio sources (quasars) in the Sloan Digital Sky Survey and evaluate its performance on a manually labeled training set. While reasonable results can already be obtained via approaches working only on photometric data, our experiments indicate that simple but problem-specific features extracted from spectroscopic data can significantly improve the classification performance. Since our approach works orthogonal to existing classification schemes used for building the spectroscopic catalogs, our classification results are well suited for a mutual assessment of the approaches' accuracies.