<jupyter_start><jupyter_text>Random Forest<jupyter_code>##-----------------
## Copyright in private
## Modify History :
##      2018 - 9 - 25
## Purpose:
##     # 1.标准随机森林- Random Forest
##     # 2. 极其随机0 - 随机森林 Extra Random Forest 
##
##
import numpy as np
import matplotlib.pyplot as plt
from sklearn import datasets

X,y = datasets.make_moons(n_samples = 1000,random_state = 100,noise = 0.20)

from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test = train_test_split(X,y,random_state = 100)<jupyter_output><empty_output><jupyter_text>1 Random Forest<jupyter_code>## random forest
from sklearn.ensemble import RandomForestClassifier

# oob_score = True 记住放回取样过程中，没有被取到的样本
# bootstrap = True 放回取样方法
rf_clf  = RandomForestClassifier(n_estimators = 200,bootstrap = True,oob_score = True,random_state = 100, n_jobs = -1 ) # n_jobs = -1 调用双核
rf_clf.fit(X,y)
# 拿没有取到样本作为测试数据
rf_clf.oob_score_
# new RF
# max_leaf_nodes = 12 更决策树一样
rf_clf_RF  = RandomForestClassifier(n_estimators = 200,max_leaf_nodes = 12,bootstrap = True,oob_score = True,random_state = 100, n_jobs = -1 ) 
rf_clf_RF.fit(X,y)
rf_clf_RF.oob_score_
# print 
print("随机森林具备决策树和bagging相同的参数")<jupyter_output>随机森林具备决策树和bagging相同的参数<jupyter_text>2.Extra Random Forest - 极其随机森林，提高了模型的随机性，整体集成学习效果更好<jupyter_code>from sklearn.ensemble import ExtraTreesClassifier
extra_tree_clf = ExtraTreesClassifier(n_estimators = 120,bootstrap = True,oob_score = True,
                                      max_features = 2,random_state = 100)
extra_tree_clf.fit(X,y)
extra_tree_clf.oob_score_<jupyter_output><empty_output><jupyter_text>3.AdaBoosting（模型之间相互争强） - 另外一种集成学习方法<jupyter_code>## 由原始的数据集做训练，不断更新数据差异大的点之间的权重，在拟合模型。
## 不断更新，每个子模型都在弥补上一次拟合的模型差错
#from sklearn.model_selection import train_test_split

#X_train,X_test,y_train,y_test = train_test_split(X,y,random_state = 100)
from sklearn.ensemble import AdaBoostClassifier 
from sklearn.tree import DecisionTreeClassifier 

adaboosting_clf = AdaBoostClassifier(DecisionTreeClassifier(max_depth = 2), n_estimators = 200)
adaboosting_clf.fit(X_train,y_train)
adaboosting_clf.score(X_test,y_test)<jupyter_output><empty_output><jupyter_text>4.GrdientBoosting（针对上次模型拟合的误差再次训练新的模型） - 另外一种集成学习方法<jupyter_code>###Note:
## 训练一个模型m1，产生错位e1
## 针对错误e2 训练模型m2，产生错误e2
## 针对错误e2,训练模型m3.产生错误e3
## 针对错误e3,训练模型m4,产生e4 
## 最终的训练结果是： m1 + m2 + m3 + m4..... + 所有模型的和。。。。每个模型都已弥补一些误差。。
from sklearn.ensemble import GradientBoostingClassifier

# GradientBoostingClassifier 是以决策树模型为基础的
gb_clf = GradientBoostingClassifier(max_depth = 12, n_estimators = 100)
gb_clf.fit(X_train,y_train)
gb_clf.score(X_train,y_train)
## test score 
gb_clf.score(X_test,y_test)<jupyter_output><empty_output><jupyter_text>上述算法同样适应于 回归问题<jupyter_code>##Stacking
## 由几层 是个超参数
## 将数据集分成 几个子数据，每个子数据分配格每个层模型
## 参数多，容易出现过拟合现象<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10907165548888105, 'fraction_numerical': 0.028238616307800918, 'mean_word_length': 6.701086956521739, 'pattern_counts': {'":': 0, '<': 24, '<?xml version=': 0, '>': 24, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '634077', 'license': 'MIT', 'n_tokens_mistral': 1440, 'n_tokens_neox': 1494, 'n_words': 165, 'path': '6_Ensemble_Learing_Random_Forest_.ipynb', 'repo_name': 'Yazooliu/Ai_Lab_', 'size': '2833'}