<jupyter_start><jupyter_text>irisのデータをXGBoostで解析してみる。<jupyter_code>import xgboost as xgb
from sklearn import datasets
from sklearn import model_selection
from sklearn.metrics import confusion_matrix, mean_squared_error
import sklearn.preprocessing as sp
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>データセットの読み込み<jupyter_code>iris = datasets.load_iris()
iris_df = pd.DataFrame(iris.data)
iris_df = iris_df.rename(columns={
    0: 'sepal_length',
    1: 'sepal_width',
    2: 'petal_length',
    3: 'petal_width'})
iris_df['target'] = iris.target

for i, name in enumerate(iris.target_names):
    iris_df['target'] = iris_df['target'].where(iris_df['target'] != i, name)

le = sp.LabelEncoder()
le.fit(iris_df.target.unique())
iris_df.target = le.fit_transform(iris_df.target)
    
ohe = sp.OneHotEncoder()
enced = ohe.fit_transform(iris_df.target.values.reshape(1, -1).transpose())
temp = pd.DataFrame(index=iris_df.target.index, columns="target-" + le.classes_, data=enced.toarray())
iris_df = pd.concat([iris_df, temp], axis=1)
del iris_df['target']

iris_df.head()<jupyter_output><empty_output><jupyter_text>データセットを学習と評価用に分ける<jupyter_code>train_df, test_df = model_selection.train_test_split(iris_df, test_size=0.3)
train_df_y = train_df[['sepal_length']]
train_df_x = train_df.copy().drop('sepal_length', axis=1)
test_df_y = test_df[['sepal_length']]
test_df_x = test_df.copy().drop('sepal_length', axis=1)<jupyter_output><empty_output><jupyter_text>ハイパーパラメータ探索しつつ学習する<jupyter_code>clf = xgb.XGBRegressor()

# ハイパーパラメータ探索
clf_cv = model_selection.GridSearchCV(clf, {'max_depth': [2,4,6], 'n_estimators': [50,100,200]}, verbose=1)
clf_cv.fit(train_df_x, [i[0] for i in train_df_y.values])
print(clf_cv.best_params_, clf_cv.best_score_)

# 改めて最適パラメータで学習
clf = xgb.XGBRegressor(**clf_cv.best_params_)
clf.fit(train_df_x, [i[0] for i in train_df_y.values])<jupyter_output>Fitting 3 folds for each of 9 candidates, totalling 27 fits
{'max_depth': 2, 'n_estimators': 100} 0.823792509678<jupyter_text>評価する<jupyter_code>mean_pred = [train_df_y.mean() for i in range(len(test_df_y))]
rmse_base = np.sqrt(mean_squared_error(test_df_y, mean_pred))
print("学習データの平均を予測としたやつをBaseLineとする\nBaseLineのrmse: " + str(rmse_base))

pred = clf.predict(test_df_x)
rmse = np.sqrt(mean_squared_error(test_df_y, pred))
print("予測したやつのrmse: " + str(rmse))<jupyter_output>学習データの平均を予測としたやつをBaseLineとする
BaseLineのrmse: 0.825791175637
予測したやつのrmse: 0.357222034136<jupyter_text>予測と実測の可視化<jupyter_code>plt.ylabel("Predict")
plt.xlabel("Actual")
plt.scatter(test_df_y, pred)
plt.plot([4.5, 8], [4.5, 8], c='r')
plt.show()<jupyter_output><empty_output><jupyter_text>予測と実測の可視化(BaseLine)<jupyter_code>plt.ylabel("Predict")
plt.xlabel("Actual")
plt.scatter(test_df_y, mean_pred)
plt.plot([4.5, 8], [4.5, 8], c='r')
plt.show()<jupyter_output><empty_output><jupyter_text>変数の重要度を可視化<jupyter_code>xgb.plot_importance(clf)
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1459658520254436, 'fraction_numerical': 0.028121861399397388, 'mean_word_length': 8.732899022801304, 'pattern_counts': {'":': 0, '<': 31, '<?xml version=': 0, '>': 31, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '107070', 'license': 'MIT', 'n_tokens_mistral': 1458, 'n_tokens_neox': 1404, 'n_words': 173, 'path': '20180215_iris/iris_xgboost_regression.ipynb', 'repo_name': 'EnsekiTT/blog', 'size': '2987'}