<jupyter_start><jupyter_text>测试集合和训练集合<jupyter_code>from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test=train_test_split(X,y, test_size=0.4, random_state=666)
X_train.shape
X_test.shape<jupyter_output><empty_output><jupyter_text>KNN识别手写<jupyter_code>from sklearn.neighbors import KNeighborsClassifier
best_score, best_p, best_k=0.0, 0, 0

for k in range(2, 10):
    for p in range(1,6):
        knn_clf=KNeighborsClassifier(n_neighbors=k, weights="distance", p=p)
        knn_clf.fit(X_train, y_train)
        score = knn_clf.score(X_test, y_test)
        
        if score>best_score:
            best_score,best_p,best_k=score,p,k
print("best score:{}, best p:{}, best k:{}".format(best_score, best_p, best_k))<jupyter_output>best score:0.9860917941585535, best p:4, best k:3<jupyter_text>使用交叉验证<jupyter_code>from sklearn.model_selection import cross_val_score
knn_clf2=KNeighborsClassifier()

cross_val_score(knn_clf2, X_train, y_train)
best_score, best_p, best_k=0.0, 0, 0

for k in range(2, 10):
    for p in range(1,6):
        knn_clf=KNeighborsClassifier(n_neighbors=k, weights="distance", p=p)
        knn_clf.fit(X_train, y_train)
        scores=cross_val_score(knn_clf, X_train, y_train)
        score = np.mean(scores)
        
        if score>best_score:
            best_score,best_p,best_k=score,p,k
print("best score:{}, best p:{}, best k:{}".format(best_score, best_p, best_k))<jupyter_output>best score:0.9823599874006478, best p:2, best k:2<jupyter_text>可以看出，交叉验证和train_test_split的结果不一样。并且交叉验证的score会低一些。 这里，我们只是想获得对应的参数k和p，并以此构建最佳模型<jupyter_code>best_knn_clf=KNeighborsClassifier(weights="distance", n_neighbors=2, p=2)
best_knn_clf.fit(X_train, y_train)
best_knn_clf.score(X_test, y_test)<jupyter_output><empty_output><jupyter_text>可以看出：以3交叉验证，得到模型参数k=2，p=2的模型，在测试数据上的R2分数是0.98  回顾网格搜索<jupyter_code>from sklearn.model_selection import GridSearchCV
param_grid=[{
    'weights':['distance'],
    'n_neighbors':[i for i in range(2,10)],
    'p':[i for i in range(1,6)]
}]
knn_clf=KNeighborsClassifier()
grid=GridSearchCV(knn_clf, param_grid, verbose=1)
grid.fit(X_train, y_train)<jupyter_output>Fitting 3 folds for each of 40 candidates, totalling 120 fits<jupyter_text>上述的意思是：40个模型模型组合参数，每个模型，都进行3交叉验证。因此，是120次fit<jupyter_code>grid.best_score_
grid.best_params_
best_knn_clf=grid.best_estimator_

#best_knn_clf.fit(X_train, y_train)
best_knn_clf.score(X_test, y_test)
cross_val_score(knn_clf, X_train, y_train, cv=5)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1212, 'fraction_numerical': 0.0364, 'mean_word_length': 6.889589905362776, 'pattern_counts': {'":': 0, '<': 22, '<?xml version=': 0, '>': 24, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '132897', 'license': 'Apache-2.0', 'n_tokens_mistral': 1212, 'n_tokens_neox': 1184, 'n_words': 149, 'path': 'imooc/Polynomial/cross_validation.ipynb', 'repo_name': 'imgoodman/play-with-machine-learning', 'size': '2500'}