Manifold learning is a hot research topic in the field of computer science. A crucial issue with current manifold learning methods is that they lack a natural quantitative measure to assess the quality of learned embeddings, which greatly limits their applications to real-world problems. In this paper, a new embedding quality assessment method for manifold learning, named as Normalization Independent Embedding Quality Assessment (NIEQA), is proposed. Compared with current assessment methods which are limited to isometric embeddings, the NIEQA method has a much larger application range due to two features. First, it is based on a new measure which can effectively evaluate how well local neighborhood geometry is preserved under normalization, hence it can be applied to both isometric and normalized embeddings. Second, it can provide both local and global evaluations to output an overall assessment. Therefore, NIEQA can serve as a natural tool in model selection and evaluation tasks for manifold learning. Experimental results on benchmark data sets validate the effectiveness of the proposed method.
Venna and Kaski @cite_42 proposed an assessment method which consists of two measures, one for trustworthiness and one for continuity, based on the change of indices of neighbor samples in @math and @math according to pairwise Euclidean distances, respectively. Aguirre al proposed an alternative approach for quantifying the embedding quality, by evaluating the possible overlaps in the low-dimensional embedding. Their assessment is used for automatic choice of the number of nearest neighbors for LLE @cite_39 and also exploited in @cite_7 to evaluate the embedding quality of LLE with optimal regularization parameter. Akkucuk and Carroll @cite_8 independently developed the Agreement Rate (AR) metric which shares the same form to @math . Based on AR, they suggested another useful assessment method called corrected agreement rate, by randomly reorganize the indices of data in @math . Also with AR, France and Carroll @cite_22 proposed a method using the RAND index to evaluate dimensionality reduction methods.
@cite_22 @cite_7 @cite_8 @cite_42 @cite_39
1688729699 1979182421 2040608731 2043508111
We develop a metric i¾?, based upon the RAND index, for the comparison and evaluation of dimensionality reduction techniques. This metric is designed to test the preservation of neighborhood structure in derived lower dimensional configurations. We use a customer information data set to show how i¾?can be used to compare dimensionality reduction methods, tune method parameters, and choose solutions when methods have a local optimum problem. We show that i¾?is highly negatively correlated with an alienation coefficient K that is designed to test the recovery of relative distances. In general a method with a good value of i¾?also has a good value of K. However the monotonic regression used by Nonmetric MDS produces solutions with good values of i¾?, but poor values of K. Locally linear embedding (LLE) is a recent unsupervised learning algorithm for non-linear dimensionality reduction of high dimensional data. One advantage of this algorithm is that just two parameters are needed to be set by user: the number of nearest neighbors and a regularization parameter. The choice of the regularization parameter plays an important role in the embedding results. In this paper, an automated method for choosing this parameter is proposed. Besides, in order to objectively qualify the performance of the embedding results, a new measure of embedding quality is suggested. Our approach is experimentally verified on 9 artificial data sets and 2 real world data sets. Numerical results are compared against two methods previously found in the state of art. Dimensionality reduction techniques are used for representing higher dimensional data by a more parsimonious and meaningful lower dimensional structure. In this paper we will study two such approaches, namely Carroll’s Parametric Mapping (abbreviated PARAMAP) (Shepard and Carroll, 1966) and Tenenbaum’s Isometric Mapping (abbreviated Isomap) (Tenenbaum, de Silva, and Langford, 2000). The former relies on iterative minimization of a cost function while the latter applies classical MDS after a preprocessing step involving the use of a shortest path algorithm to define approximate geodesic distances. We will develop a measure of congruence based on preservation of local structure between the input data and the mapped low dimensional embedding, and compare the different approaches on various sets of data, including points located on the surface of a sphere, some data called the "Swiss Roll data", and truncated spheres. In a visualization task, every nonlinear projection method needs to make a compromise between trustworthiness and continuity. In a trustworthy projection the visualized proximities hold in the original data as well, whereas a continuous projection visualizes all proximities of the original data. We show experimentally that one of the multidimensional scaling methods, curvilinear components analysis, is good at maximizing trustworthiness. We then extend it to focus on local proximities both in the input and output space, and to explicitly make a user-tunable parameterized compromise between trustworthiness and continuity. The new method compares favorably to alternative nonlinear projection methods.