<jupyter_start><jupyter_text>These tests measure the accuracy of models on training data, models that perform well on training data(KNN1) may not be as accurate when used on out of sample data.<jupyter_code>from sklearn.datasets import load_iris
from sklearn.neighbors import KNeighborsClassifier
from sklearn.linear_model import LogisticRegression
from sklearn import metrics
iris = load_iris()
X = iris.data
y = iris.target
knn1 = KNeighborsClassifier(n_neighbors = 1)
knn1.fit(X,y)
y1_knn = knn1.predict(X)
print(metrics.accuracy_score(y, y1_knn))<jupyter_output>1.0<jupyter_text>KNN1 memorizes the dataset so it always achieves 100% accuracy. It does this by choosing the nearest node and since there is only one to pick from it always picks the correct one.<jupyter_code>knn5 = KNeighborsClassifier(n_neighbors=5)
knn5.fit(X,y)
y5_knn = knn5.predict(X)
print(metrics.accuracy_score(y, y5_knn))
logReg = LogisticRegression()
logReg.fit(X, y)
y_prediction = logReg.predict(X)
print(metrics.accuracy_score(y, y_prediction))<jupyter_output>0.96<jupyter_text>So a better solution is to perform a Train/test split- Split the dataset into training set and testing set- train on training set and test on testing set<jupyter_code>from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.4, random_state = 4)
print(X_train.shape)
print(X_test.shape)
print(y_train.shape)
print(y_test.shape)<jupyter_output>(90, 4)
(60, 4)
(90,)
(60,)<jupyter_text>Logistic Regression accuracy test<jupyter_code>logReg2 = LogisticRegression()
logReg2.fit(X_train, y_train)
y_prediction2 = logReg2.predict(X_test)
print(metrics.accuracy_score(y_test, y_prediction2))
KNN5 = KNeighborsClassifier(n_neighbors = 5)
KNN5.fit(X_train, y_train)
y5_KNN = KNN5.predict(X_test)
print(metrics.accuracy_score(y_test, y5_KNN))
KNN1 = KNeighborsClassifier(n_neighbors = 1)
KNN1.fit(X_train, y_train)
y1_KNN = KNN1.predict(X_test)
print(metrics.accuracy_score(y_test, y1_KNN))<jupyter_output>0.95<jupyter_text>KNN 5 is the best out of the three models for out of sample dataCould there be a KNN value which can be better than Knn 5's accuracy<jupyter_code>i_range = range(1, 26)
scores = []
for i in i_range:
    KNNi = KNeighborsClassifier(n_neighbors = i)
    KNNi.fit(X_train, y_train)
    y_pred = KNNi.predict(X_test)
    scores.append(metrics.accuracy_score(y_test, y_pred))
import matplotlib.pyplot as plt
%matplotlib inline
plt.plot(i_range, scores)
plt.xlabel("# of KNN neighbors")
plt.ylabel("Accuracy")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09529366005445351, 'fraction_numerical': 0.023337222870478413, 'mean_word_length': 6.938271604938271, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '186925', 'license': 'MIT', 'n_tokens_mistral': 1026, 'n_tokens_neox': 984, 'n_words': 236, 'path': 'iris-accracyModels.ipynb', 'repo_name': 'vgutta/ML', 'size': '2571'}