<jupyter_start><jupyter_text>随即森林调参实战<jupyter_code>data = load_breast_cancer()
data
data.data.shape
data.target
rfc = RandomForestClassifier(n_estimators=100, random_state=90)
score_pre = cross_val_score(rfc, data.data, data.target, cv=10).mean()
score_pre<jupyter_output><empty_output><jupyter_text>探索n_estimators的最好区间<jupyter_code>scorel = []
for i in range(0,200,10):
    rfc = RandomForestClassifier(n_estimators=i+1,
                                n_jobs=-1,
                                random_state=90)
    score = cross_val_score(rfc, data.data, data.target, cv=10).mean()
    scorel.append(score)
print(max(scorel), (scorel.index(max(scorel))*10)+1)
plt.figure(figsize=[20,5])
plt.plot(range(1,201,10),scorel)
plt.show()<jupyter_output>0.9666353383458647 81<jupyter_text>更细化的探索<jupyter_code>scorel = []
for i in range(65,95):
    rfc = RandomForestClassifier(n_estimators=i,
                                n_jobs=-1,
                                random_state=90)
    score = cross_val_score(rfc, data.data, data.target, cv=10).mean()
    scorel.append(score)
print(max(scorel), ([*range(65,95)][scorel.index(max(scorel))]))
plt.figure(figsize=[20,5])
plt.plot(range(65,95),scorel)
plt.show()<jupyter_output>0.9666353383458647 73<jupyter_text>网格搜索进行调参  首先调整max_depth<jupyter_code>param_grid = {'max_depth':np.arange(1,20,1)}

rfc = RandomForestClassifier(n_estimators=73,
                            random_state=90)
GS = GridSearchCV(rfc, param_grid, cv=10)
GS.fit(data.data, data.target)
GS.best_params_
GS.best_score_<jupyter_output><empty_output><jupyter_text>调整maxdepth，模型没有上升的空间，基本上已经保持在泛化误差的最低点或者稍微偏左，现在将泛化误差向右推看模型效果是否会更好。 调整max_features<jupyter_code>param_grid = {'max_features':np.arange(5,30,1)}

rfc = RandomForestClassifier(n_estimators=73,
                             max_depth=8,
                            random_state=90)
GS = GridSearchCV(rfc, param_grid, cv=10)
GS.fit(data.data, data.target)
GS.best_params_
GS.best_score_<jupyter_output><empty_output><jupyter_text>模型效果又有一个提升，现在模型已经基本在最低点了。最终的参数结果<jupyter_code>rfc = RandomForestClassifier(n_estimators=73,
                            max_depth=8,
                            max_features=22,
                            random_state=90)
score = cross_val_score(rfc, data.data, data.target, cv=10).mean()
score<jupyter_output><empty_output><jupyter_text>AdaBoostClassifier调参<jupyter_code>from sklearn.ensemble import AdaBoostClassifier
abc = AdaBoostClassifier(n_estimators=50, random_state=90)
score_pre = cross_val_score(abc, data.data, data.target, cv=10).mean()
score_pre
scorel = []
for i in range(0,200,10):
    rfc = AdaBoostClassifier(n_estimators=i+1,
                                random_state=90)
    score = cross_val_score(rfc, data.data, data.target, cv=10).mean()
    scorel.append(score)
print(max(scorel), (scorel.index(max(scorel))*10)+1)
plt.figure(figsize=[20,5])
plt.plot(range(1,201,10),scorel)
plt.show()
scorel = []
for i in range(171,191):
    rfc = AdaBoostClassifier(n_estimators=i,
                                random_state=90)
    score = cross_val_score(rfc, data.data, data.target, cv=10).mean()
    scorel.append(score)
print(max(scorel), ([*range(171,191)][scorel.index(max(scorel))]))
plt.figure(figsize=[20,5])
plt.plot(range(171,191),scorel)
plt.show()
from sklearn.tree import DecisionTreeClassifier

dc = DecisionTreeClassifier(max_depth=2)
abc = AdaBoostClassifier(n_estimators=175,
                        base_estimator=dc,
                        random_state=90)
score = cross_val_score(abc, data.data, data.target, cv=10).mean()
score<jupyter_output><empty_output><jupyter_text>max_depth设置为2可以看见模型效果反倒下降一些<jupyter_code>param_grid = {'learning_rate':np.arange(1,10,1),
              'n_estimators':np.arange(171,191)
             }

rfc = AdaBoostClassifier(random_state=90)
GS = GridSearchCV(rfc, param_grid, cv=10)
GS.fit(data.data, data.target)
GS.best_params_
GS.best_score_<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13209939148073022, 'fraction_numerical': 0.04969574036511156, 'mean_word_length': 4.150130548302872, 'pattern_counts': {'":': 0, '<': 31, '<?xml version=': 0, '>': 31, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '357241', 'license': 'MIT', 'n_tokens_mistral': 1713, 'n_tokens_neox': 1624, 'n_words': 169, 'path': 'code/randomForest/breastCancerClassification.ipynb', 'repo_name': 'Knowledge-Precipitation-Tribe/Machine-Learning', 'size': '3944'}