<jupyter_start><jupyter_text>Recursive Feature Elimination (RFE)This procedure is used to eliminate predictors (or dependent variables) that will not have influence on the outcome of a classifier. It works by recursively removing features from a model and estimating the model accuracy with the remaining features.In order to illustrate how it works, I will use the function `make_classification` from sklearn.datasets in order to create a binary classification problem. In this case I will create a dataset with 4 predictors, two of which will not be informative and will be discarded by RFE<jupyter_code>from sklearn.datasets import make_classification
X, y = make_classification(n_samples=5000, n_features=4, n_informative=2,
                           n_redundant=2, n_repeated=0, n_classes=2,
                           n_clusters_per_class=1,
                           random_state=0)
from sklearn.feature_selection import RFE
from sklearn.linear_model import LogisticRegression

# I will create the RFE model for the logistic regression classifier
logreg = LogisticRegression()

# In this case, we will select 2 features
rfe = RFE(logreg, 2)
rfe = rfe.fit(X, y)
# print summaries for the selection of attributes
print(rfe.support_)
print(rfe.ranking_)<jupyter_output>[ True  True False False]
[1 1 3 2]
{'fraction_non_alphanumeric': 0.051908396946564885, 'fraction_numerical': 0.013740458015267175, 'mean_word_length': 4.161417322834645, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '195580', 'license': 'Apache-2.0', 'n_tokens_mistral': 380, 'n_tokens_neox': 354, 'n_words': 153, 'path': 'MACHINE_LEARNING/Recursive Feature Elimination (RFE).ipynb', 'repo_name': 'elowy01/PYTHON_NOTES', 'size': '1310'}