<jupyter_start><jupyter_text>K Nearest Neighbors Project Welcome to the KNN Project! This will be a simple project very similar to the lecture, except you'll be given another data set. Go ahead and just follow the directions below. Import Libraries**Import pandas,seaborn, and the usual libraries.**<jupyter_code>import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
%matplotlib inline<jupyter_output><empty_output><jupyter_text>Get the Data** Read the 'KNN_Project_Data csv file into a dataframe **<jupyter_code>df = pd.read_csv("KNN_Project_Data")<jupyter_output><empty_output><jupyter_text>**Check the head of the dataframe.**<jupyter_code>df.head()<jupyter_output><empty_output><jupyter_text>EDASince this data is artificial, we'll just do a large pairplot with seaborn.**Use seaborn on the dataframe to create a pairplot with the hue indicated by the TARGET CLASS column.**<jupyter_code>sns.pairplot(df,hue="TARGET CLASS",palette="coolwarm")<jupyter_output><empty_output><jupyter_text>Standardize the VariablesTime to standardize the variables.** Import StandardScaler from Scikit learn.**<jupyter_code>from sklearn.preprocessing import  StandardScaler<jupyter_output><empty_output><jupyter_text>** Create a StandardScaler() object called scaler.**<jupyter_code>scaler = StandardScaler()<jupyter_output><empty_output><jupyter_text>** Fit scaler to the features.**<jupyter_code>scaler.fit(df.drop("TARGET CLASS",axis=1))<jupyter_output><empty_output><jupyter_text>**Use the .transform() method to transform the features to a scaled version.**<jupyter_code>scaled_features = scaler.transform(df.drop("TARGET CLASS",axis = 1))<jupyter_output><empty_output><jupyter_text>**Convert the scaled features to a dataframe and check the head of this dataframe to make sure the scaling worked.**<jupyter_code>df_feat = pd.DataFrame(scaled_features,columns=df.columns[:-1])
df_feat.head()<jupyter_output><empty_output><jupyter_text>Train Test Split**Use train_test_split to split your data into a training set and a testing set.**<jupyter_code>from sklearn.model_selection import train_test_split
X = df_feat
y = df['TARGET CLASS']
X_train, X_test, y_train, y_test = train_test_split(df_feat,df["TARGET CLASS"],test_size=0.30,random_state = 101)<jupyter_output><empty_output><jupyter_text>Using KNN**Import KNeighborsClassifier from scikit learn.**<jupyter_code>from sklearn.neighbors import  KNeighborsClassifier<jupyter_output><empty_output><jupyter_text>**Create a KNN model instance with n_neighbors=1**<jupyter_code>knn = KNeighborsClassifier(n_neighbors=1)<jupyter_output><empty_output><jupyter_text>**Fit this KNN model to the training data.**<jupyter_code>knn.fit(X_train,y_train)<jupyter_output><empty_output><jupyter_text>Predictions and EvaluationsLet's evaluate our KNN model! **Use the predict method to predict values using your KNN model and X_test.**<jupyter_code>pred = knn.predict(X_test)<jupyter_output><empty_output><jupyter_text>** Create a confusion matrix and classification report.**<jupyter_code>from sklearn.metrics import classification_report, confusion_matrix
print(confusion_matrix(y_test,pred))
print(classification_report(y_test,pred))<jupyter_output><empty_output><jupyter_text>Choosing a K ValueLet's go ahead and use the elbow method to pick a good K Value!** Create a for loop that trains various KNN models with different k values, then keep track of the error_rate for each of these models with a list. Refer to the lecture if you are confused on this step.**<jupyter_code>error_rate = []

# Will take some time
for i in range(1,40):
    
    knn = KNeighborsClassifier(n_neighbors=i)
    knn.fit(X_train,y_train)
    pred_i = knn.predict(X_test)
    error_rate.append(np.mean(pred_i != y_test))<jupyter_output><empty_output><jupyter_text>**Now create the following plot using the information from your for loop.**<jupyter_code>plt.figure(figsize=(10,6))
plt.plot(range(1,40),error_rate,color='blue', linestyle='dashed', marker='o',
         markerfacecolor='red', markersize=10)
plt.title('Error Rate vs. K Value')
plt.xlabel('K')
plt.ylabel('Error Rate')
plt.grid(True)<jupyter_output><empty_output><jupyter_text>Retrain with new K Value**Retrain your model with the best K value (up to you to decide what you want) and re-do the classification report and the confusion matrix.**<jupyter_code># NOW WITH K=16
knn = KNeighborsClassifier(n_neighbors=16)

knn.fit(X_train,y_train)
pred = knn.predict(X_test)

print('WITH K=16')
print('\n')
print(confusion_matrix(y_test,pred))
print('\n')
print(classification_report(y_test,pred))
print("Calculated Theoritical Error : \n")
print((confusion_matrix(y_test,pred)[0][0]+confusion_matrix(y_test,pred)[1][1])/300)
# NOW WITH K=30
knn = KNeighborsClassifier(n_neighbors=30)

knn.fit(X_train,y_train)
pred = knn.predict(X_test)

print('WITH K=30')
print('\n')
print(confusion_matrix(y_test,pred))
print('\n')
print(classification_report(y_test,pred))
print("Calculated Theoritical Error : \n")
print((confusion_matrix(y_test,pred)[0][0]+confusion_matrix(y_test,pred)[1][1])/300)
# NOW WITH K=24
knn = KNeighborsClassifier(n_neighbors=24)

knn.fit(X_train,y_train)
pred = knn.predict(X_test)

print('WITH K=24')
print('\n')
print(confusion_matrix(y_test,pred))
print('\n')
print(classification_report(y_test,pred))
print("Calculated Theoritical Error : \n")
print((confusion_matrix(y_test,pred)[0][0]+confusion_matrix(y_test,pred)[1][1])/300)
# NOW WITH K=39
knn = KNeighborsClassifier(n_neighbors=39)

knn.fit(X_train,y_train)
pred = knn.predict(X_test)

print('WITH K=39')
print('\n')
print(confusion_matrix(y_test,pred))
print('\n')
print(classification_report(y_test,pred))
print("Calculated Theoritical Error : \n")
print((confusion_matrix(y_test,pred)[0][0]+confusion_matrix(y_test,pred)[1][1])/300)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12130025662959795, 'fraction_numerical': 0.012660393498716851, 'mean_word_length': 8.72712146422629, 'pattern_counts': {'":': 0, '<': 73, '<?xml version=': 0, '>': 73, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '18', 'dir': 'jupyter-structured-clean-dedup', 'id': '75923', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 2132, 'n_tokens_neox': 2038, 'n_words': 451, 'path': '18. K Nearest Neighbors/.ipynb_checkpoints/18.3 KNN Project-checkpoint.ipynb', 'repo_name': 'CommunityOfCoders/ML_Workshop_Teachers', 'size': '5845'}