Existing semi-supervised and supervised feature selection methods can be
classified into wrapper, filter and embedded three classes
[1]}. Wrapper methods [2]}
create multiple models with different subsets of input features and
select the model containing the features that yield the best
performance. One example is recursive feature elimination (RFE)
[3]}. This process can be computationally expensive. Filter
methods involve evaluating the relationship between input and target
variables using statistics and selecting those variables that have the
strongest relation with the target ones. One example is the analysis of
variance (ANOVA) [4]}. This approach is computationally
efficient with robust performance. Embedded methods perform feature
selection in the process of training and are usually specific to a
single learner. One example is “feature importance" (FI) obtained from
the training process of the XGBoost classifier/regressor
[5]}, [6]}, which is also known as
“feature selection from model" (SFM). Recently, the discriminant
feature test (DFT) and the relevant feature test (RFT) were proposed in
[7]} for the classification and the regression
problems, respectively. They belong to the filter class.
<FIGURE>