<jupyter_start><jupyter_text>KNN (K Nearerst Neighbor) WHAT IS KNN?KNN is a rather simple algorithm that is typically used as a classifier (which is the use we'll focus on here) but it may also be utilized in regression type problems. KNN is a *lazy*, *instance* based learner that takes some observation and tries to assign it a category based upon its proximity to "k" (number of observations) neighboring data points. Essentially the algorithm looks at a data points 'K' nearest neighbors and and then assigns the classification of the the majority class near that data point.Lets go ahead an visualize this so we can start to get a more intuitive understanding of the mechanisms at play here.Data used in this example: [Iris Data](https://archive.ics.uci.edu/ml/datasets/iris)<jupyter_code># import our libraries
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
#Now that we have our libraries we can pull down a basic data set to experiment with

#Iris dataset, a typical toy data set
#Reading the data into a pandas dataframe
df = pd.read_csv('https://raw.githubusercontent.com/Jaunson/Scripts/master/iris.csv')

#Column names for data to be inserted
col_names = ['Sepal Length - cm', 'Sepal Width - cm', 'Petal Length - cm', 'Petal Width - cm', 'Class']
#Mystery Data Points adding these for our KNN demo
mysterydata1 = pd.DataFrame(data=[[7,2.8,3.75,1.2,'?']],columns=col_names) 
mysterydata2 = pd.DataFrame(data=[[4.75,3.25,3.75,1.2,'?']],columns=col_names)

#Appending our points onto the data set (similar to a SQL UNION)
df = df.append([mysterydata1,mysterydata2])
df = df.drop('Unnamed: 0',axis=1)

df.info() #pull down basic metadata to make sure things look ok
df.describe()
df.columns<jupyter_output><empty_output><jupyter_text>VISUALIZING KNNOk now that we have gone a head and imported our libraries and pull our data in, lets start to get a sense of how KNN works by visualizing our data points on a scatter plot. We'll start out by taking two dimensions (Sepal Length & Sepal Width) and visualizes where each of our observations fall on the scatter, we'll also color the observations by their name (Class).<jupyter_code>#take a look at if there is a weighted distance KNN

fig=plt.figure(figsize=(10, 8), dpi= 80, facecolor='w', edgecolor='k')
original=sns.scatterplot(x=df['Sepal Width - cm'],y=df['Sepal Length - cm'],hue=df['Class'])<jupyter_output><empty_output><jupyter_text>Great so we can see that we have 3 distinct classes on our scatterplot with ~152 observations, and 2 of our data points are unclassified. HOW IS KNN GOING TO HELP US IDENTIFY THESE MYSTERY VALUES?![<NAME>](https://i.imgflip.com/2xur2n.jpg)- How do we decide who is  a neighbor?We first determine a k-value- We want an odd number, this will keep us from having ties between categories- Low values (1's & 2's) can skew the selection made for a given observation- Not sure where to start? - try making 'k' = $\sqrt{n}$, n = number of observations    The KNN algorithm will then calculates the euclidean distance between observations and find the 'k' neighbors closest.The majority class within 'k' neighboring points will then be used to classify our unknown observation.[Overview of KNN - By StatQuest!](https://youtu.be/HVXime0nQeI)<jupyter_code>import sklearn

# preprocess our data to work with the model

# split our mystery points from our data
known = df.query("Class!='?'").reset_index(drop=True)
mystery = df.query("Class=='?'").reset_index(drop=True)

# Split each set into its features and target (for our simple example remember we are only using the Sepal features)
# Sklearn works with Numpy arrays so we'll also do a quick conversion

known_features = known.iloc[:,0:2].values # .values leaves us with each vector as a np array 
known_target = known.iloc[:,4].values

mystery_features = mystery.iloc[:,0:2].values
mystery_target = mystery.iloc[:,4].values

k = 3 # our k value, after running through exercise adjust this value to see what changes


knn = sklearn.neighbors.KNeighborsClassifier(n_neighbors=k)

#Fit our model
knn.fit(known_features, known_target) # < build our model with known information
prediction=pd.Series(knn.predict(mystery_features)) # < use model to predict mystery information
prediction<jupyter_output><empty_output><jupyter_text>NOW VISUALIZE<jupyter_code>mystery['Class'] = prediction
pred_df = known.append(mystery).reset_index(drop=True)
pred_df

fig, ax = plt.subplots(1,2, figsize=(16,8), dpi= 80, facecolor='w', edgecolor='k')

ax[0].set_title('Prediction')
ax[1].set_title('Original')

sns.scatterplot(x=pred_df['Sepal Width - cm'],y=pred_df['Sepal Length - cm']
                ,hue=pred_df['Class'],ax=ax[0])

sns.scatterplot(x=df['Sepal Width - cm'],y=df['Sepal Length - cm']
                ,hue=df['Class'],ax=ax[1])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10159651669085631, 'fraction_numerical': 0.012025710138917687, 'mean_word_length': 5.240620957309185, 'pattern_counts': {'":': 0, '<': 20, '<?xml version=': 0, '>': 18, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '377648', 'license': 'MIT', 'n_tokens_mistral': 1553, 'n_tokens_neox': 1474, 'n_words': 577, 'path': 'Classifier/KNN.ipynb', 'repo_name': 'Jaunson/Analysis-DS_Techniques', 'size': '4830'}