A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Feature selection <s> A feature subset selection algorithm based on branch and bound techniques is developed to select the best subset of m features from an n-feature set. Existing procedures for feature subset selection, such as sequential selection and dynamic programming, do not guarantee optimality of the selected feature subset. Exhaustive search, on the other hand, is generally computationally unfeasible. The present algorithm is very efficient and it selects the best subset without exhaustive search. Computational aspects of the algorithm are discussed. Results of several experiments demonstrate the very substantial computational savings realized. For example, the best 12-feature set from a 24-feature set was selected with the computational effort of evaluating only 6000 subsets. Exhaustive search would require the evaluation of 2 704 156 subsets. <s> BIB001 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Feature selection <s> Machine learning algorithms automatically extract knowledge from machine readable information. Unfortunately, their success is usually dependant on the quality of the data that they operate on. If the data is inadequate, or contains extraneous and irrelevant information, machine learning algorithms may produce less accurate and less understandable results, or may fail to discover anything of use at all. Feature subset selectors are algorithms that attempt to identify and remove as much irrelevant and redundant information as possible prior to learning. Feature subset selection can result in enhanced performance, a reduced hypothesis search space, and, in some cases, reduced storage requirement. This paper describes a new feature selection algorithm that uses a correlation based heuristic to determine the “goodness” of feature subsets, and evaluates its effectiveness with three common machine learning algorithms. Experiments using a number of standard machine learning data sets are presented. Feature subset selection gave significant improvement for all three algorithms. <s> BIB002 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Feature selection <s> In this paper, we examine a method for feature subset selection based on Information Theory. Initially, a framework for defining the theoretically optimal, but computationally intractable, method for feature subset selection is presented. We show that our goal should be to eliminate a feature if it gives us little or no additional information beyond that subsumed by the remaining features. In particular, this will be the case for both irrelevant and redundant features. We then give an efficient algorithm for feature selection which computes an approximation to the optimal feature selection criterion. The conditions under which the approximate algorithm is successful are examined. Empirical results are given on a number of data sets, showing that the algorithm effectively handles datasets with a very large number of features. <s> BIB003 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Feature selection <s> Abstract In the feature subset selection problem, a learning algorithm is faced with the problem of selecting a relevant subset of features upon which to focus its attention, while ignoring the rest. To achieve the best possible performance with a particular learning algorithm on a particular training set, a feature subset selection method should consider how the algorithm and the training set interact. We explore the relation between optimal feature subset selection and relevance. Our wrapper method searches for an optimal feature subset tailored to a particular algorithm and a domain. We study the strengths and weaknesses of the wrapper approach and show a series of improved designs. We compare the wrapper approach to induction without feature subset selection and to Relief, a filter approach to feature subset selection. Significant improvement in accuracy is achieved for some datasets for the two families of induction algorithms used: decision trees and Naive-Bayes. <s> BIB004 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Feature selection <s> Algorithms for feature selection fall into two broad categories: wrappers that use the learning algorithm itself to evaluate the usefulness of features and filters that evaluate features according to heuristics based on general characteristics of the data. For application to large databases, filters have proven to be more practical than wrappers because they are much faster. However, most existing filter algorithms only work with discrete classification problems. This paper describes a fast, correlation-based filter algorithm that can be applied to continuous and discrete problems. The algorithm often outperforms the well-known ReliefF attribute estimator when used as a preprocessing step for naive Bayes, instance-based learning, decision trees, locally weighted regression, and model trees. It performs more feature selection than ReliefF does—reducing the data dimensionality by fifty percent in most cases. Also, decision and model trees built from the preprocessed data are often significantly smaller. <s> BIB005 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Feature selection <s> In view of the substantial number of existing feature selection algorithms, the need arises to count on criteria that enables to adequately decide which algorithm to use in certain situations. This work assesses the performance of several fundamental algorithms found in the literature in a controlled scenario. A scoring measure ranks the algorithms by taking into account the amount of relevance, irrelevance and redundance on sample data sets. This measure computes the degree of matching between the output given by the algorithm and the known optimal solution. Sample size effects are also studied. <s> BIB006 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Feature selection <s> In this article, we describe an unsupervised feature selection algorithm suitable for data sets, large in both dimension and size. The method is based on measuring similarity between features whereby redundancy therein is removed. This does not need any search and, therefore, is fast. A new feature similarity measure, called maximum information compression index, is introduced. The algorithm is generic in nature and has the capability of multiscale representation of data sets. The superiority of the algorithm, in terms of speed and performance, is established extensively over various real-life data sets of different sizes and dimensions. It is also demonstrated how redundancy and information loss in feature selection can be quantified with an entropy measure. <s> BIB007 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Feature selection <s> This paper enhances and analyses the power of local weighted similarity measures. The paper proposes a new entropy-based local weighting algorithm (EBL) to be used in similarity assessment to improve the performance of the CBR retrieval task. We describe a comparative analysis of the performance of unweighted similarity measures, global weighted similarity measures, and local weighting similarity measures. The testing has been done using several similarity measures, and some data sets from the UCI Machine Learning Data-base Repository and other environmental databases. Main result is that using EBL, and a weight sensitive similarity measure could improve similarity assessment in case retrieval. <s> BIB008 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Feature selection <s> We study an approach to text categorization that combines distributional clustering of words and a Support Vector Machine (SVM) classifier. This word-cluster representation is computed using the recently introduced Information Bottleneck method, which generates a compact and efficient representation of documents. When combined with the classification power of the SVM, this method yields high performance in text categorization. This novel combination of SVM with word-cluster representation is compared with SVM-based categorization using the simpler bag-of-words (BOW) representation. The comparison is performed over three known datasets. On one of these datasets (the 20 Newsgroups) the method based on word clusters significantly outperforms the word-based representation in terms of categorization accuracy or representation efficiency. On the two other sets (Reuters-21578 and WebKB) the word-based representation slightly outperforms the word-cluster representation. We investigate the potential reasons for this behavior and relate it to structural differences between the datasets. <s> BIB009 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Feature selection <s> In supervised learning variable selection is used to find a subset of the available inputs that accurately predict the output. This paper shows that some of the variables that variable selection discards can beneficially be used as extra outputs for inductive transfer. Using discarded input variables as extra outputs forces the model to learn mappings from the variables that were selected as inputs to these extra outputs. Inductive transfer makes what is learned by these mappings available to the model that is being trained on the main output, often resulting in improved performance on that main output. We present three synthetic problems (two regression problems and one classification problem) where performance improves if some variables discarded by variable selection are used as extra outputs. We then apply variable selection to two real problems (DNA splice-junction and pneumonia risk prediction) and demonstrate the same effect: using some of the discarded input variables as extra outputs yields somewhat better performance on both of these problems than can be achieved by variable selection alone. This new approach enhances the benefit of variable selection by allowing the learner to benefit from variables that would otherwise have been discarded by variable selection, but without suffering the loss in performance that occurs when these variables are used as inputs. <s> BIB010 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Feature selection <s> In this paper, we identify two issues involved in developing an automated feature subset selection algorithm for unlabeled data: the need for finding the number of clusters in conjunction with feature selection, and the need for normalizing the bias of feature selection criteria with respect to dimension. We explore the feature selection problem and these issues through FSSEM (Feature Subset Selection using Expectation-Maximization (EM) clustering) and through two different performance criteria for evaluating candidate feature subsets: scatter separability and maximum likelihood. We present proofs on the dimensionality biases of these feature criteria, and present a cross-projection normalization scheme that can be applied to any criterion to ameliorate these biases. Our experiments show the need for feature selection, the need for addressing these two issues, and the effectiveness of our proposed solutions. <s> BIB011 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Feature selection <s> The major hypothesis that we will be prove in this paper is that unsupervised learning techniques of feature weighting are not significantly worse than supervised methods, as is commonly believed in the machine learning community. ::: ::: This paper tests the power of unsupervised feature weighting techniques for predictive tasks within several domains. The paper analyses several unsupervised and supervised feature weighting techniques, and proposes new unsupervised feature weighting techniques. Two unsupervised entropy-based weighting algorithms are proposed and tested against all other techniques. The techniques are evaluated in terms of predictive accuracy on unseen instances, measured by a ten-fold cross-validation process. The testing has been done using thirty-four data sets from the UCI Machine Learning Database Repository and other sources. Unsupervised weighting methods assign weights to attributes without any knowledge about class labels, so this task is considerably more difficult. It has commonly been assumed that unsupervised methods would have a substantially worse performance than supervised ones, as they do not use any domain knowledge to bias the process. The major result of the study is that unsupervised methods really are not so bad. Moreover, one of the new unsupervised learning method proposals has shown a promising behaviour when faced against domains with many irrelevant features, reaching similar performance as some of the supervised methods. <s> BIB012 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Feature selection <s> Feature selection is an important problem for pattern classification systems. We study how to select good features according to the maximal statistical dependency criterion based on mutual information. Because of the difficulty in directly implementing the maximal dependency condition, we first derive an equivalent form, called minimal-redundancy-maximal-relevance criterion (mRMR), for first-order incremental feature selection. Then, we present a two-stage feature selection algorithm by combining mRMR and other more sophisticated feature selectors (e.g., wrappers). This allows us to select a compact set of superior features at very low cost. We perform extensive experimental comparison of our algorithm and other methods using three different classifiers (naive Bayes, support vector machine, and linear discriminate analysis) and four different data sets (handwritten digits, arrhythmia, NCI cancer cell lines, and lymphoma tissues). The results confirm that mRMR leads to promising improvement on feature selection and classification accuracy. <s> BIB013 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Feature selection <s> A major challenge for genomewide disease association studies is the high cost of genotyping large number of single nucleotide polymorphisms (SNP). The correlations between SNPs, however, make it possible to select a parsimonious set of informative SNPst known as "tagging" SNPs, able to capture most variation in a population. Considerable research interest has recently focused on the development of methods for finding such SNPs. In this paper, we present an efficient method for finding tagging SNPs. The method does not involve computation-intensive search for SNP subsets but discards redundant SNPs using a feature selection algorithm. In contrast to most existing methods, the method presented here does not limit itself to using only correlations between SNPs in local groups. By using correlations that occur across different chromosomal regions, the method can reduce the number of globally redundant SNPs. Experimental results show that the number of tagging SNPs selected by our method is smaller than by using block-based methods. <s> BIB014 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Feature selection <s> PREFACE Introduction and Background Less Is More Huan Liu and Hiroshi Motoda Background and Basics Supervised, Unsupervised, and Semi-Supervised Feature Selection Key Contributions and Organization of the Book Looking Ahead Unsupervised Feature Selection Jennifer G. Dy Introduction Clustering Feature Selection Feature Selection for Unlabeled Data Local Approaches Summary Randomized Feature Selection David J. Stracuzzi Introduction Types of Randomizations Randomized Complexity Classes Applying Randomization to Feature Selection The Role of Heuristics Examples of Randomized Selection Algorithms Issues in Randomization Summary Causal Feature Selection Isabelle Guyon, Constantin Aliferis, and Andre Elisseeff Introduction Classical "Non-Causal" Feature Selection The Concept of Causality Feature Relevance in Bayesian Networks Causal Discovery Algorithms Examples of Applications Summary, Conclusions, and Open Problems Extending Feature Selection Active Learning of Feature Relevance Emanuele Olivetti, Sriharsha Veeramachaneni, and Paolo Avesani Introduction Active Sampling for Feature Relevance Estimation Derivation of the Sampling Benefit Function Implementation of the Active Sampling Algorithm Experiments Conclusions and Future Work A Study of Feature Extraction Techniques Based on Decision Border Estimate Claudia Diamantini and Domenico Potena Introduction Feature Extraction Based on Decision Boundary Generalities about Labeled Vector Quantizers Feature Extraction Based on Vector Quantizers Experiments Conclusions Ensemble-Based Variable Selection Using Independent Probes Eugene Tuv, Alexander Borisov, and Kari Torkkola Introduction Tree Ensemble Methods in Feature Ranking The Algorithm: Ensemble-Based Ranking against Independent Probes Experiments Discussion Efficient Incremental-Ranked Feature Selection in Massive Data Roberto Ruiz, Jesus S. Aguilar-Ruiz, and Jose C. Riquelme Introduction Related Work Preliminary Concepts Incremental Performance over Ranking Experimental Results Conclusions Weighting and Local Methods Non-Myopic Feature Quality Evaluation with (R)ReliefF Igor Kononenko and Marko Robnik Sikonja Introduction From Impurity to Relief ReliefF for Classification and RReliefF for Regression Extensions Interpretation Implementation Issues Applications Conclusion Weighting Method for Feature Selection in k-Means Joshua Zhexue Huang, Jun Xu, Michael Ng, and Yunming Ye Introduction Feature Weighting in k-Means W-k-Means Clustering Algorithm Feature Selection Subspace Clustering with k-Means Text Clustering Related Work Discussions Local Feature Selection for Classification Carlotta Domeniconi and Dimitrios Gunopulos Introduction The Curse of Dimensionality Adaptive Metric Techniques Large Margin nearest Neighbor Classifiers Experimental Comparisons Conclusions Feature Weighting through Local Learning Yijun Sun Introduction Mathematical Interpretation of Relief Iterative Relief Algorithm Extension to Multiclass Problems Online Learning Computational Complexity Experiments Conclusion Text Classification and Clustering Feature Selection for Text Classification George Forman Introduction Text Feature Generators Feature Filtering for Classification Practical and Scalable Computation A Case Study Conclusion and Future Work A Bayesian Feature Selection Score Based on Naive Bayes Models Susana Eyheramendy and David Madigan Introduction Feature Selection Scores Classification Algorithms Experimental Settings and Results Conclusion Pairwise Constraints-Guided Dimensionality Reduction Wei Tang and Shi Zhong Introduction Pairwise Constraints-Guided Feature Projection Pairwise Constraints-Guided Co-Clustering Experimental Studies Conclusion and Future Work Aggressive Feature Selection by Feature Ranking Masoud Makrehchi and Mohamed S. Kamel Introduction Feature Selection by Feature Ranking Proposed Approach to Reducing Term Redundancy Experimental Results Summary Feature Selection in Bioinformatics Feature Selection for Genomic Data Analysis Lei Yu Introduction Redundancy-Based Feature Selection Empirical Study Summary A Feature Generation Algorithm with Applications to Biological Sequence Classification Rezarta Islamaj Dogan, Lise Getoor, and W. John Wilbur Introduction Splice-Site Prediction Feature Generation Algorithm Experiments and Discussion Conclusions An Ensemble Method for Identifying Robust Features for Biomarker Discovery Diana Chan, Susan M. Bridges, and Shane C. Burgess Introduction Biomarker Discovery from Proteome Profiles Challenges of Biomarker Identification Ensemble Method for Feature Selection Feature Selection Ensemble Results and Discussion Conclusion Model Building and Feature Selection with Genomic Data Hui Zou and Trevor Hastie Introduction Ridge Regression, Lasso, and Bridge Drawbacks of the Lasso The Elastic Net The Elastic-Net Penalized SVM Sparse Eigen-Genes Summary INDEX <s> BIB015 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Feature selection <s> The use of feature selection can improve accuracy, efficiency, applicability and understandability of a learning process. For this reason, many methods of automatic feature selection have been developed. Some of these methods are based on the search of the features that allows the data set to be considered consistent. In a search problem we usually evaluate the search states, in the case of feature selection we measure the possible feature sets. This paper reviews the state of the art of consistency based feature selection methods, identifying the measures used for feature sets. An in-deep study of these measures is conducted, including the definition of a new measure necessary for completeness. After that, we perform an empirical evaluation of the measures comparing them with the highly reputed wrapper approach. Consistency measures achieve similar results to those of the wrapper approach with much better efficiency. <s> BIB016 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Feature selection <s> Feature selection plays an important role in data mining and pattern recognition, especially for large scale data. During past years, various metrics have been proposed to measure the relevance between different features. Since mutual information is nonlinear and can effectively represent the dependencies of features, it is one of widely used measurements in feature selection. Just owing to these, many promising feature selection algorithms based on mutual information with different parameters have been developed. In this paper, at first a general criterion function about mutual information in feature selector is introduced, which can bring most information measurements in previous algorithms together. In traditional selectors, mutual information is estimated on the whole sampling space. This, however, cannot exactly represent the relevance among features. To cope with this problem, the second purpose of this paper is to propose a new feature selection algorithm based on dynamic mutual information, which is only estimated on unlabeled instances. To verify the effectiveness of our method, several experiments are carried out on sixteen UCI datasets using four typical classifiers. The experimental results indicate that our algorithm achieved better results than other methods in most cases. <s> BIB017 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Feature selection <s> Steganalysis is used to detect hidden content in innocuous images. Many successful steganalysis algorithms use ::: a large number of features relative to the size of the training set and suffer from a "curse of dimensionality": ::: large number of feature values relative to training data size. High dimensionality of the feature space can reduce ::: classification accuracy, obscure important features for classification, and increase computational complexity. This ::: paper presents a filter-type feature selection algorithm that selects reduced feature sets using the Mahalanobis ::: distance measure, and develops classifiers from the sets. The experiment is applied to a well-known JPEG ::: steganalyzer, and shows that using our approach, reduced-feature steganalyzers can be obtained that perform as ::: well as the original steganalyzer. The steganalyzer is that of Pevn´y et al. (SPIE, 2007) that combines DCT-based ::: feature values and calibrated Markov features. Five embedding algorithms are used. Our results demonstrate ::: that as few as 10-60 features at various levels of embedding can be used to create a classifier that gives comparable ::: results to the full suite of 274 features. <s> BIB018 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Feature selection <s> With the advent of high dimensionality, adequate identification of relevant features of the data has become indispensable in real-world scenarios. In this context, the importance of feature selection is beyond doubt and different methods have been developed. However, with such a vast body of algorithms available, choosing the adequate feature selection method is not an easy-to-solve question and it is necessary to check their effectiveness on different situations. Nevertheless, the assessment of relevant features is difficult in real datasets and so an interesting option is to use artificial data. In this paper, several synthetic datasets are employed for this purpose, aiming at reviewing the performance of feature selection methods in the presence of a crescent number or irrelevant features, noise in the data, redundancy and interaction between attributes, as well as a small ratio between number of samples and number of features. Seven filters, two embedded methods, and two wrappers are applied over eleven synthetic datasets, tested by four classifiers, so as to be able to choose a robust method, paving the way for its application to real datasets. <s> BIB019 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Feature selection <s> We present a unifying framework for information theoretic feature selection, bringing almost two decades of research on heuristic filter criteria under a single theoretical interpretation. This is in response to the question: "what are the implicit statistical assumptions of feature selection criteria based on mutual information?". To answer this, we adopt a different strategy than is usual in the feature selection literature--instead of trying to define a criterion, we derive one, directly from a clearly specified objective function: the conditional likelihood of the training labels. While many hand-designed heuristic criteria try to optimize a definition of feature 'relevancy' and 'redundancy', our approach leads to a probabilistic framework which naturally incorporates these concepts. As a result we can unify the numerous criteria published over the last two decades, and show them to be low-order approximations to the exact (but intractable) optimisation problem. The primary contribution is to show that common heuristics for information based feature selection (including Markov Blanket algorithms as a special case) are approximate iterative maximisers of the conditional likelihood. A large empirical study provides strong evidence to favour certain classes of criteria, in particular those that balance the relative size of the relevancy/redundancy terms. Overall we conclude that the JMI criterion (Yang and Moody, 1999; Meyer et al., 2008) provides the best tradeoff in terms of accuracy, stability, and flexibility with small data samples. <s> BIB020 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Feature selection <s> A plenitude of feature selection (FS) methods is available in the literature, most of them rising as a need to analyze data of very high dimension, usually hundreds or thousands of variables. Such data sets are now available in various application areas like combinatorial chemistry, text mining, multivariate imaging, or bioinformatics. As a general accepted rule, these methods are grouped in filters, wrappers, and embedded methods. More recently, a new group of methods has been added in the general framework of FS: ensemble techniques. The focus in this survey is on filter feature selection methods for informative feature discovery in gene expression microarray (GEM) analysis, which is also known as differentially expressed genes (DEGs) discovery, gene prioritization, or biomarker discovery. We present them in a unified framework, using standardized notations in order to reveal their technical details and to highlight their common characteristics as well as their particularities. <s> BIB021 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Feature selection <s> Plenty of feature selection methods are available in literature due to the availability of data with hundreds of variables leading to data with very high dimension. Feature selection methods provides us a way of reducing computation time, improving prediction performance, and a better understanding of the data in machine learning or pattern recognition applications. In this paper we provide an overview of some of the methods present in literature. The objective is to provide a generic introduction to variable elimination which can be applied to a wide array of machine learning problems. We focus on Filter, Wrapper and Embedded methods. We also apply some of the feature selection techniques on standard datasets to demonstrate the applicability of feature selection techniques. <s> BIB022 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Feature selection <s> Filter-based feature selection has become crucial in many classification settings, especially object recognition, recently faced with feature learning strategies that originate thousands of cues. In this paper, we propose a feature selection method exploiting the convergence properties of power series of matrices, and introducing the concept of infinite feature selection (Inf-FS). Considering a selection of features as a path among feature distributions and letting these paths tend to an infinite number permits the investigation of the importance (relevance and redundancy) of a feature when injected into an arbitrary set of cues. Ranking the importance individuates candidate features, which turn out to be effective from a classification point of view, as proved by a thoroughly experimental section. The Inf-FS has been tested on thirteen diverse benchmarks, comparing against filters, embedded methods, and wrappers, in all the cases we achieve top performances, notably on the classification tasks of PASCAL VOC 2007-2012. <s> BIB023
Feature selection is a specialization of Feature Weighting, where all the weights get binary values: 0 or 1. If the weight of a variable is 0, that means it is discarded, if the weight is 1, the variable is used for the analysis. For a survey of common feature selection techniques, see BIB019 BIB022 BIB015 BIB006  . Feature selectors are algorithms that attempt to identify and remove as much irrelevant and redundant information as possible prior to learning or knowledge discovery. It is important to remark that most of the feature selection methods perform with respect to a certain response variable that has to be modeled, although some research has been done in the nonsupervised scenario BIB011 BIB007 BIB012 BIB008 . Feature selection can result in enhanced performance, a reduced hypothesis search space, and, in some cases, reduced storage requirement. Usually, analyzing the feature subset selection provides better results than analyzing the complete set of variables BIB002 . Feature selection can be performed by applying a cutting threshold over the results of a feature weighting process. However, automated techniques for identifying and removing unhelpful or redundant variables are extensively used. Some can use statistical criteria to rank the variables BIB020 BIB005 , others are mutualinformation criteria based BIB017 BIB013 or more general concepts like instance consistency  BIB016  . Feature selection methods usually take one of the three forms BIB002 : • Filter Feature Selection Methods: Direct examination of the relevance of candidate variables, independently of the machine learning model to be used for the data mining, usually a predictive method. Usually we can distinguish two types of filter methods BIB009 BIB010 BIB018 BIB003 BIB021 BIB023 : * Feature ranking methods: they rank features by a metric and eliminates all features bellow an adequate score. Often these methods do not consider potential interactions among features. * Subset selection methods: they search for the optimal subset of features. Some of them can take into account the interaction among features. * Mixed approaches: they can sequentially interleave some feature ranking operations with some subset selection operations, to try to capture all the benefits from both approaches. • Wrapper Feature Selection methods: Searching the best combination of variables in terms of model performance and feedback. They utilize the ML model of interest as a black box to score subsets of feature according to their predictive power BIB014 : * Brute-force methods: they explore all the possible subsets of combinations of features and get the optimal one BIB004 BIB001 .