<jupyter_start><jupyter_text>OverviewWe discussed ensemble methods in the previous notebook. We mentioned how weak learners can become a strong predictor after averaging their outputs. This is actually known as Boosting. Adaboost places an emphasis on missclassified examples by using weights. The weights increase the liklihood of the missclassified examples showing in the training data for the next model. Eventually, the models are used in an ensemble to help predict the new class.The most popular boosting techniques are:- AdaBoost- Gradient BoostingIn this notebook we will discuss Adaboost.   Contents Key Points- Adaboost combines weak learners to make classifications.- Each predictor is influenced by the mistakes made by the previous predictors-  Some predictors will get more say than others  AdaBoostIn scikit-learn, SAMME (Stagewise Additive Modeling using a mutliclass exponential loss function) is used. This is the same as adaboost when only two classes are used. Samme.R uses class probabilities and generally performs much better. This technique can be used for multiclass problems too.<jupyter_code>from sklearn.datasets import load_iris
from sklearn.ensemble import AdaBoostClassifier
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import train_test_split
iris = load_iris()
X = iris["data"]
y = iris["target"]

X_train, X_test, y_train, y_test = train_test_split(X, y,random_state = 42)
y_train
ada_clf = AdaBoostClassifier(
    DecisionTreeClassifier(max_depth=1), n_estimators = 500, 
    algorithm = "SAMME.R", learning_rate = 0.5)

ada_clf.fit(X_train,y_train)
from sklearn.model_selection import cross_val_score
cross_val_score(ada_clf,X_train,y_train, cv=3, scoring ="accuracy")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.05149971703452179, 'fraction_numerical': 0.0050933786078098476, 'mean_word_length': 5.90625, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '574732', 'license': 'MIT', 'n_tokens_mistral': 516, 'n_tokens_neox': 478, 'n_words': 205, 'path': 'Techniques/Ensemble Methods/Code/Boosting - AdaBoost.ipynb', 'repo_name': 'VirajVaitha123/Machine-Learning', 'size': '1767'}