<jupyter_start><jupyter_text>Example using random forestIn this example we use a pool of classifiers generated using the Random Forest method rather than Bagging. We alsoshow how to change the size of the region of competence, used to estimate the local competence of the base classifiers.This demonstrates that the library accepts any kind of base classifiers as long as they implement the predict andpredict proba functions. Moreover, any ensemble generation method such as Boosting or Rotation Trees can be usedto generate a pool containing diverse base classifiers. We also included the performance of the RandomForest classifieras a baseline comparison.<jupyter_code>from sklearn.datasets import load_breast_cancer
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split

# Example of a dcs techniques
from deslib.dcs.ola import OLA
from deslib.dcs.mcb import MCB
# Example of a des techniques
from deslib.des.knora_e import KNORAE
from deslib.des.des_p import DESP
from deslib.des.knora_u import KNORAU
from deslib.des.meta_des import METADES<jupyter_output><empty_output><jupyter_text>Loading a classification dataset and preparing the data<jupyter_code># Generate a classification dataset
data = load_breast_cancer()
X = data.data
y = data.target
# split the data into training and test data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25)

#Training a random forest classifier using the whole training data
RF = RandomForestClassifier()
RF.fit(X_train, y_train)

X_train, X_dsel, y_train, y_dsel = train_test_split(X, y, test_size=0.50)<jupyter_output><empty_output><jupyter_text>Training a random forest to be used as the pool of classifiersWe set the maximum depth of each decision tree to 5 so that it can estimate posterior probabilities.  Otherwise, sklearn generates decision tree in which each leaf contains only samples from a single class, attributing always probability 1 to the predicted class and 0 to all others. Some DS techinques such as the META-DES, requires good probabilities estimates to achieve a better performance.<jupyter_code>pool_classifiers = RandomForestClassifier(max_depth=5)
pool_classifiers.fit(X_train, y_train)<jupyter_output><empty_output><jupyter_text>Initializing DS techniques<jupyter_code># Initialize a DS technique. Here we specify the size of the region of competence (5 neighbors)
knorau = KNORAU(pool_classifiers)
kne = KNORAE(pool_classifiers, k=5)
desp = DESP(pool_classifiers, k=5)
ola = OLA(pool_classifiers, k=5)
mcb = MCB(pool_classifiers, k=5)
meta = METADES(pool_classifiers, k=5)
# Fit the DS techniques
knorau.fit(X_dsel, y_dsel)
kne.fit(X_dsel, y_dsel)
desp.fit(X_dsel, y_dsel)
meta.fit(X_dsel, y_dsel)
ola.fit(X_dsel, y_dsel)
mcb.fit(X_dsel, y_dsel)<jupyter_output><empty_output><jupyter_text>Calculate classification accuracy of each technique<jupyter_code># Calculate classification accuracy of each technique
print('Classification accuracy RF: ', RF.score(X_test, y_test))
print('Evaluating DS techniques:')
print('Classification accuracy KNORAU: ', knorau.score(X_test, y_test))
print('Classification accuracy KNORA-Eliminate: ', kne.score(X_test, y_test))
print('Classification accuracy DESP: ', desp.score(X_test, y_test))
print('Classification accuracy OLA: ', ola.score(X_test, y_test))
print('Classification accuracy MCB: ', mcb.score(X_test, y_test))
print('Classification accuracy META-DES: ', meta.score(X_test, y_test))<jupyter_output>Classification accuracy RF:  0.9090909090909091
Evaluating DS techniques:
Classification accuracy KNORAU:  0.9790209790209791
Classification accuracy KNORA-Eliminate:  0.986013986013986
Classification accuracy DESP:  0.9790209790209791
Classification accuracy OLA:  0.972027972027972
Classification accuracy MCB:  0.9790209790209791
Classification accuracy META-DES:  0.9790209790209791
{'fraction_non_alphanumeric': 0.06917914300464637, 'fraction_numerical': 0.03433144037170883, 'mean_word_length': 6.908163265306122, 'pattern_counts': {'":': 0, '<': 20, '<?xml version=': 0, '>': 20, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '398622', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1313, 'n_tokens_neox': 1204, 'n_words': 400, 'path': 'examples/Notebooks_examples/Example_random_forest.ipynb', 'repo_name': 'mathematixy/DESlib', 'size': '3874'}