These traditional dimensionality reduction techniques are widely used. Recently, many new techniques have been proposed [1]}, [2]}. Kernel PCA is the reformulation of traditional linear PCA in a high-dimensional space that is constructed using a kernel function [3]}. When applied PCA and kernel PCA to a nonlinear system, [4]} found PCA was inefficient and problematic. [5]} propose a defect prediction framework that combines kernel (PCA) and Weighted Extreme Learning Machine to extract representative data features and learn an effective defect prediction model.
Nonlinear dimensionality reduction techniques are notably able to avoid overcrowding of the representation, wherein distinct clusters are represented on an overlapping area [6]}. High-dimensional data can be transformed into low-dimensional codes by training a multilayer neural network with a small central layer. [7]} presented that the deep autoencoder networks worked better than PCA if the initial weights were close to a good solution.
[8]} proposed the isometric feature mapping (Isomap) technique. Their method based on classical MDS but seeks to retain the intrinsic geometry of data sets, as captured in the geodesic manifold distances between all pairs of data points. It achieves the goal of nonlinear dimensionality reduction by using easily measured local metric information to learn the underlying global geometry of a data set. Non-parametric dimensionality reduction techniques such as t-distributed stochastic neighbor embedding (t-SNE) is a powerful tool for high-dimensional data [9]}. [10]} presented kernel t-SNE as an extension of t-SNE to a parametric framework, which enables explicit out-of-sample extensions. It is demonstrated that this technique yields satisfactory results for large data sets. [11]} proposed Uniform Manifold Approximation and Projection (UMAP), constructing from a theoretical framework based in Riemannian geometry and algebraic topology. Compared with t-SNE, UMAP arguably preserves more of the global structure. UMAP can be used as a general purpose dimension reduction technique, since it does not have computational restrictions on embedding dimension.
