<jupyter_start><jupyter_text>Bagging and Voting感觉上，assemble就是倒来倒去。三个臭皮匠，顶得上一个诸葛亮。只是三个皮匠的组合方式不同而已。  文中掷骰子的例子，其实更加科学一点。就是想办法*增加*数据集的**大小**。因为数据集越大，那么随机的影响就越小。- Voting就是N个predictor预测一组数据，把预测的数据，按照一个规则进行vote    - hard: 出现最多    - soft：概率最高- bagging： 就是用一个predictor，把数据分成N份，来训练N个模型。然后取众数。然后N份有放回操作    - oob(out of bag): 在sklearn里面的这个，可以理解为自然形成的一个测试集        - [资料一](https://stackoverflow.com/questions/33631125/scikit-learn-parameters-oob-score-oob-score-oob-prediction)        - [资料二](https://www.jianshu.com/p/9420ebfd05bd)- pasting：形成数据N的时候，资料并不放回。<jupyter_code>from sklearn.datasets import make_moons
from sklearn.ensemble import RandomForestClassifier
from sklearn.ensemble import VotingClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.svm import SVC

log_clf = LogisticRegression()
rnd_clf = RandomForestClassifier()
svm_clf = SVC()

voting_clf = VotingClassifier(
    estimators=[('lr',log_clf),('rf',rnd_clf),('svc',svm_clf)],
    voting= 'hard'
)

X,y = make_moons(500,noise=0.4,random_state=18)
X_train,X_test,y_train,y_test = train_test_split(X,y,test_size=0.2,random_state=18)

voting_clf.fit(X_train,y_train)
from sklearn.metrics import accuracy_score

for clf in (log_clf,rnd_clf,svm_clf,voting_clf):
    clf.fit(X_train,y_train)
    y_pred = clf.predict(X_test)
    print(clf.__class__.__name__,accuracy_score(y_test,y_pred))
from sklearn.ensemble import BaggingClassifier
from sklearn.tree import DecisionTreeClassifier

bag_clf = BaggingClassifier(
    DecisionTreeClassifier(),n_estimators=500,
    max_samples=100,bootstrap=True,n_jobs=1,oob_score=True
)

bag_clf.fit(X_train,y_train)
y_pred - bag_clf.predict(X_test)
print("bag accuracy is ",accuracy_score(y_test,y_pred))
import numpy as np
import matplotlib.pyplot as plt

def plot_predictions(ax, clf_plot, x_value, y_value,axes):
    x0s = np.linspace(axes[0],axes[1],100)
    x1s = np.linspace(axes[2],axes[3],100)
    x0,x1= np.meshgrid(x0s,x1s)
    x_plot = np.c_[x0.ravel(),x1.ravel()]
    y_pred_plot = clf_plot.predict(x_plot).reshape(x0.shape)
    ax.contourf(x0,x1,y_pred_plot,cmap=plt.cm.brg, alpha=0.2)
    ax.plot(x_value[:, 0][y_value==0], x_value[:, 1][y_value==0], "bs")
    ax.plot(x_value[:, 0][y_value==1], x_value[:, 1][y_value==1], "g^")

fig, axs = plt.subplots(1,2,figsize=(12, 5))
axis_size = [-2.5,3,-1.5,2.0]
plot_predictions(axs[0],bag_clf,X,y,axis_size)
axs[0].set_title(r"$bag classifier$",)
plt.show()
bag_clf.oob_score_<jupyter_output><empty_output><jupyter_text>Random Forests<jupyter_code>from sklearn.ensemble import  RandomForestClassifier

rnd_clf = RandomForestClassifier(n_estimators=500,max_leaf_nodes=16,n_jobs=-1)
rnd_clf.fit(X_train,y_train)
y_pred_rf = rnd_clf.predict(X_test)
print("random forest accuracy", accuracy_score(y_test,y_pred_rf))
rnd_clf.feature_importances_<jupyter_output><empty_output><jupyter_text>Boosting Ada boosting首先，是一个线性的过程。找到那些**难**的sample。然后让下一棒，注意这些难的**sample**<jupyter_code>from sklearn.ensemble import AdaBoostClassifier

ada_clf = AdaBoostClassifier(
    DecisionTreeClassifier(max_depth=1),n_estimators=200,
    algorithm="SAMME.R",learning_rate= 0.5
)
ada_clf.fit(X_train,y_train)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1346749226006192, 'fraction_numerical': 0.027554179566563468, 'mean_word_length': 7.7560975609756095, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 2, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '80627', 'license': 'Apache-2.0', 'n_tokens_mistral': 1541, 'n_tokens_neox': 1545, 'n_words': 158, 'path': 'myNoteBook/chapter7.ipynb', 'repo_name': 'chandlersong/handson-ml', 'size': '3230'}