Out of several heuristic wrapper selection methods, the Recursive Feature Elimination (RFE) [1]} algorithm has received considerable interest. It has been successfully used with various classifiers on biological, chemical, gene-expression, financial, and text data [1]}, [3]}, [4]}, [5]}. RFE is a hybrid wrapper-filter algorithm that uses information about feature importance exposed by the wrapped classifier to recursively eliminate the least promising attributes. An important property of this algorithm is that it can be combined with cross-validation to automatically select the optimal number of features to remove. Moreover, to speed up feature selection the user can decide to remove features in larger steps, rather than one attribute at a time. However, larger steps infer larger gaps between consecutively analyzed feature subset sizes. This, in turn, causes RFE to choose too many or too few attributes compared to removing one feature at a time, which negatively affects the number of selected features, and potentially the final learning performance.
