DPDR: A novel machine learning method for the Decision Process for Dimensionality Reduction
This paper discusses the critical decision process of extracting or selecting
the features in a supervised learning context. It is often confusing to find a
suitable method to reduce dimensionality. There are pros and cons to deciding
between a feature selection and feature extraction according to the data's
nature and the user's preferences. Indeed, the user may want to emphasize the
results toward integrity or interpretability and a specific data resolution.
This paper proposes a new method to choose the best dimensionality reduction
method in a supervised learning context. It also helps to drop or reconstruct
the features until a target resolution is reached. This target resolution can
be user-defined, or it can be automatically defined by the method. The method
applies a regression or a classification, evaluates the results, and gives a
diagnosis about the best dimensionality reduction process in this specific
supervised learning context. The main algorithms used are the Random Forest
algorithms (RF), the Principal Component Analysis (PCA) algorithm, and the
multilayer perceptron (MLP) neural network algorithm. Six use cases are
presented, and every one is based on some well-known technique to generate
synthetic data. This research discusses each choice that can be made in the
process, aiming to clarify the issues about the entire decision process of
selecting or extracting the features.