<jupyter_start><jupyter_text>Supervised Learning and K Nearest Neighbors Exercises![knn.png](Assets/knn.png)  Learning Objectives:- Explain supervised learning and how it can be applied to regression and classification problems- Apply K-Nearest Neighbor (KNN) algorithm for classification- Apply Intel® Extension for Scikit-learn* to leverage underlying compute capabilities of hardware scikit-learn* Frameworks provide structure that Data Scientists use to build code. Frameworks are more than just libraries, because in addition to callable code, frameworks influence how code is written. A main virtue of using an optimized framework is that code runs faster. Code that runs faster is just generally more convenient but when we begin looking at applied data science and AI models, we can see more material benefits. Here you will see how optimization, particularly hyperparameter optimization can benefit more than just speed. These exercises will demonstrate how to apply **the Intel® Extension for Scikit-learn*,** a seamless way to speed up your Scikit-learn application. The acceleration is achieved through the use of the Intel® oneAPI Data Analytics Library (oneDAL). Patching is the term used to extend scikit-learn with Intel optimizations and makes it a well-suited machine learning framework for dealing with real-life problems. To get optimized versions of many Scikit-learn algorithms using a patch() approach consisting of adding these lines of code Prior to importing sklearn: - **from sklearnex import patch_sklearn**- **patch_sklearn()** This exercise relies on installation of  Intel® Extension for Scikit-learn*If you have not already done so, follow the instructions from Week 1 for instructions  IntroductionWe will be using customer churn data from the telecom industry for the first week's exercises. The data file is called `Orange_Telecom_Churn_Data.csv`. We will load this data together, do some preprocessing, and use K-nearest neighbors to predict customer churn based on account characteristics.<jupyter_code>from __future__ import print_function
import os
data_path = ['data']

from sklearnex import patch_sklearn
patch_sklearn()  # patch should be called PRIOR to importing Scikit-learn algorithms

from sklearn.preprocessing import LabelBinarizer
from sklearn.preprocessing import MinMaxScaler
from sklearn.neighbors import KNeighborsClassifier<jupyter_output>Intel(R) Extension for Scikit-learn* enabled (https://github.com/intel/scikit-learn-intelex)<jupyter_text>Question 1* Begin by importing the data. Examine the columns and data.* Notice that the data contains a state, area code, and phone number. Do you think these are good features to use when building a machine learning model? Why or why not? We will not be using them, so they can be dropped from the data.<jupyter_code>import pandas as pd

# Import the data using the file path
filepath = os.sep.join(data_path + ['Orange_Telecom_Churn_Data.csv'])
data = pd.read_csv(filepath)
data.head(1).T
# Remove extraneous columns
data.drop(['state', 'area_code', 'phone_number'], axis=1, inplace=True)
data.columns<jupyter_output><empty_output><jupyter_text>Question 2* Notice that some of the columns are categorical data and some are floats. These features will need to be numerically encoded using one of the methods from the lecture.* Finally, remember from the lecture that K-nearest neighbors requires scaled data. Scale the data using one of the scaling methods discussed in the lecture.<jupyter_code>lb = LabelBinarizer()

for col in ['intl_plan', 'voice_mail_plan', 'churned']:
    data[col] = lb.fit_transform(data[col])
# Mute the sklearn warning
import warnings
warnings.filterwarnings('ignore', module='sklearn')

msc = MinMaxScaler()

data = pd.DataFrame(msc.fit_transform(data),  # this is an np.array, not a dataframe.
                    columns=data.columns)<jupyter_output><empty_output><jupyter_text>Question 3* Separate the feature columns (everything except `churned`) from the label (`churned`). This will create two tables.* Fit a K-nearest neighbors model with a value of `k=3` to this data and predict the outcome on the same data.<jupyter_code># Get a list of all the columns that don't contain the label
x_cols = [x for x in data.columns if x != 'churned']

# Split the data into two dataframes
X_data = data[x_cols]
y_data = data['churned']

# # alternatively:
# X_data = data.copy()
# y_data = X_data.pop('churned')
knn = KNeighborsClassifier(n_neighbors=3)

knn = knn.fit(X_data, y_data)

y_pred = knn.predict(X_data)<jupyter_output><empty_output><jupyter_text>Question 4Ways to measure error haven't been discussed in class yet, but accuracy is an easy one to understand--it is simply the percent of labels that were correctly predicted (either true or false). * Write a function to calculate accuracy using the actual and predicted labels.* Using the function, calculate the accuracy of this K-nearest neighbors model on the data.<jupyter_code># Function to calculate the % of values that were correctly predicted

def accuracy(real, predict):
    return sum(y_data == y_pred) / float(real.shape[0])
print(accuracy(y_data, y_pred))<jupyter_output>0.9422<jupyter_text>Question 5* Fit the K-nearest neighbors model again with `n_neighbors=3` but this time use distance for the weights. Calculate the accuracy using the function you created above. * Fit another K-nearest neighbors model. This time use uniform weights but set the power parameter for the Minkowski distance metric to be 1 (`p=1`) i.e. Manhattan Distance.When weighted distances are used for part 1 of this question, a value of 1.0 should be returned for the accuracy. Why do you think this is? *Hint:* we are predicting on the data and with KNN the model *is* the data. We will learn how to avoid this pitfall in the next lecture.<jupyter_code>knn = KNeighborsClassifier(n_neighbors=5, weights='distance')

knn = knn.fit(X_data, y_data)

y_pred = knn.predict(X_data)

print(accuracy(y_data, y_pred))
knn = KNeighborsClassifier(n_neighbors=5, p=2)

knn = knn.fit(X_data, y_data)

y_pred = knn.predict(X_data)

print(accuracy(y_data, y_pred))<jupyter_output>0.9284<jupyter_text>Question 6* Fit a K-nearest neighbors model using values of `k` (`n_neighbors`) ranging from 1 to 20. Use uniform weights (the default). The coefficient for the Minkowski distance (`p`) can be set to either 1 or 2--just be consistent. Store the accuracy and the value of `k` used from each of these fits in a list or dictionary.* Plot (or view the table of) the `accuracy` vs `k`. What do you notice happens when `k=1`? Why do you think this is? *Hint:* it's for the same reason discussed above.<jupyter_code># Fit the K-nearest neighbors model with different values of k
# Store the accuracy measurement for each k

score_list = list()

for k in range(1, 21):
    
    knn = KNeighborsClassifier(n_neighbors=k)
    knn = knn.fit(X_data, y_data)
    
    y_pred = knn.predict(X_data)
    score = accuracy(y_data, y_pred)
    
    score_list.append((k, score))
    
score_df = pd.DataFrame(score_list, columns=['k', 'accuracy'])
# Import libraries to make the plot

import matplotlib.pyplot as plt
import seaborn as sns

%matplotlib inline
sns.set_context('talk')
sns.set_style('ticks')
sns.set_palette('dark')

ax = score_df.set_index('k').plot()

ax.set(xlabel='k', ylabel='accuracy')
ax.set_xticks(range(1, 21));<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07601944369430191, 'fraction_numerical': 0.005806103159600324, 'mean_word_length': 5.2401010951979785, 'pattern_counts': {'":': 0, '<': 26, '<?xml version=': 0, '>': 26, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '643205', 'license': 'MIT', 'n_tokens_mistral': 2204, 'n_tokens_neox': 2120, 'n_words': 919, 'path': 'AI-and-Analytics/Jupyter/Introduction_to_Machine_Learning/02-Supervised_Learning_and_K_Nearest_Neighbors/Supervised_Learning_and_K_Nearest_Neighbors_Exercises-ANSWERS.ipynb', 'repo_name': 'krisrak/oneAPI-samples', 'size': '7406'}