<jupyter_start><jupyter_text>Bagging  Voting Classifier- same training set,- ≠ algorithms. Bagging- one algorithm,- ≠ subsets of the training set.- Bagging: Bootstrap Aggregation.- Uses a technique known as the bootsrap.- Reduces variance of individual models in the ensemble.  Bootstrap ![image.png](attachment:image.png)  Treino ![image.png](attachment:image.png)  Bagging: Classification & Regression- Classification:    - Aggregates predictions by majority voting.    - BaggingClassifier in scikit-learn.- Regression:    - Aggregates predictions through averaging.    - BaggingRegressor in scikit-learn.<jupyter_code>from sklearn.ensemble import BaggingClassifier
from sklearn.tree import DecisionTreeClassifier
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split
from sklearn import datasets

X, y = datasets.load_breast_cancer(return_X_y=True)

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, stratify=y, random_state=42)
dt = DecisionTreeClassifier(max_depth=4, min_samples_leaf=0.16, random_state=42)

bc = BaggingClassifier(base_estimator=dt, n_estimators=300, n_jobs=-1)
bc.fit(X_train, y_train)

y_pred = bc.predict(X_test)

accuracy = accuracy_score(y_test, y_pred)
print('Accuracy of Bagging Classifier: {:.3f}'.format(accuracy))<jupyter_output>Accuracy of Bagging Classifier: 0.930<jupyter_text>Bagging- some instances may be sampled several times for one model,- other instances may not be sampled at all.  Out Of Bag (OOB) instances- On average, for each model, 63% of the training instances are sampled.- The remaining 37% constitute the OOB instances. ![image.png](attachment:image.png)<jupyter_code>from sklearn.ensemble import BaggingClassifier
from sklearn.tree import DecisionTreeClassifier
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size= 0.3, stratify= y, random_state=42)
dt = DecisionTreeClassifier(max_depth=4, min_samples_leaf=0.16, random_state=42)

bc = BaggingClassifier(base_estimator=dt, n_estimators=300, oob_score=True, n_jobs=-1)
bc.fit(X_train, y_train)
y_pred = bc.predict(X_test)
test_accuracy = accuracy_score(y_test, y_pred)
oob_accuracy = bc.oob_score_

print('Test set accuracy: {:.3f}'.format(test_accuracy))
print('OOB accuracy: {:.3f}'.format(oob_accuracy))<jupyter_output>Test set accuracy: 0.930
OOB accuracy: 0.925
{'fraction_non_alphanumeric': 0.09514087382605145, 'fraction_numerical': 0.019191506737443853, 'mean_word_length': 6.608695652173913, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '265597', 'license': 'MIT', 'n_tokens_mistral': 858, 'n_tokens_neox': 806, 'n_words': 214, 'path': '2019/08-Modelos_de_Ensemble/Bagging.ipynb', 'repo_name': 'InsightLab/imersao-ciencia-de-dados-2019', 'size': '2449'}