Machine learning algorithm can be used for variable importance analysis, such as Random Forest (RF) [1]} and Neural Network (NN) [2]}. RF has referred as a useful tool to handle the feature selection issue even with high-dimensional datasets [1]}.
[4]} explored RF algorithm in presence of correlated predictors. Their results motivated the use of the recursive feature elimination (REF) algorithm using permutation importance measure as a ranking criterion. [1]} compared the result of three popular datasets (Bank Marketing, Car Evaluation Database, Human Activity Recognition Using Smartphones) with and without important feature selection by RF method varImp(), Boruta, and REF. Experimental results demonstrated that RF achieved a better performance in all experiment groups. These experiments proved important feature selection is essential for classification problem. Furthermore, the best feature selection method is varImp() compared to Boruta and REF. varImp() is included in the classification and regression training (Caret) package. This package contains effective tools for data splitting, pre-processing, feature selection, model tuning with resampling, variable importance estimation, as well as other functionality. NN have received great attention in recent years. However, they are labeled as a "black box" since they provide little explanatory insight into the contributions of input variables in the prediction process [2]}. [2]} compared nine methodologies for quantifying variable importance in artificial NN using simulated data. Their results show Connection Weight Approach (CWA) provided the best accuracy. Furthermore, CWA was the only method identifying the ranked importance of all predictors. The other methods either only identified the first few important variables in the network or no variables at all. [8]} presented a deep Baysian Rectified Linear Unit network (ReLU) for high-dimensional regression and variable selection. The result showed their method outperforms existing classic and NN-based variable selection methods.
