<jupyter_start><jupyter_text>K-Nearest Neighbors Classification in R: Exercise   Exercise 1: Balance ScaleIn this exercise, we will be using the balance_scale.txt data set. In this data set, each data point is classified as having the balance scale tip to the right, tip to the left, or be balanced. The variables of the data set are 'left weight', 'left distance', 'right weight', and 'right distance'. The class is determined by the greater of (right-distance x right-weight) and (left-distance x left-weight). However, if these values are equal, the data point is balanced.The objective of this exercise is to use the k-nearest neighbors algorithm to determine the classification of test data and to estimate using knn regression, the difference between the product of right-distance and right-weight, and the product of left-distance and left-weight.  a). Install the packageWe first install and load the libraries that will be used in this exercise.<jupyter_code># Install the packages 'class' and 'kknn' and load their libraries, which will be needed for their k-nearest neighbors algorithms
install.packages("class")
install.packages("kknn")
library(class)
library(kknn)<jupyter_output><empty_output><jupyter_text>b). Download the data set, load the data and view its structureWe can use the download.file command to download the data set, "balance_scale.txt", load the data using the read.csv command, and view its structure using the str command. We will also look at the first few lines of the data using the head command.<jupyter_code># Load the data and view its structure
balance_scale <- read.csv("https://ibm.box.com/shared/static/684jzm7e6fbbssg87yc2v4dy53dgkdew.txt", sep = ",")
str(balance_scale)

# View the first few rows of the data using the head function
# Note: The raw data does not contain any column names
head(balance_scale)<jupyter_output><empty_output><jupyter_text>c). Clean the dataNow we will clean the data by adding column names, as well new columns for the right and left products and their differences.<jupyter_code>## Your Answer Code ##<jupyter_output><empty_output><jupyter_text>Click here for the solution``` Add column namescolnames(balance_scale) <- c("Class_Name","Left_Weight", "Left_Distance", "Right_Weight", "Right_Distance")head(balance_scale) Note: We do not need to standardize the data in this instance since the numerical data values lie on the same scale. Calculate the products and differencesRight_Product <- balance_scale[,4]*balance_scale[,5]Left_Product <- balance_scale[,2]*balance_scale[,3]Differences <- Right_Product-Left_Product Add columns for Right_Product, Left_Product and Differencesbalance_scale$Right_Product <- Right_Productbalance_scale$Left_Product <- Left_Productbalance_scale$Differences <- Differences    ```  d). Separate the data into training and test groupsNext we will split the data into two groups which will be used for training and testing respectively. To do this, we will create a vector of equal length to our data, with approximately 70% of the values in the vector being 1's and the remaining values being 2's. The order in which the values appear will be random and depending on the indices of the vector which correspond to the value of 1 will be used to select our training data, while the indices corresponding to 2 will be used to select our test data.<jupyter_code>## Your Answer Code ##<jupyter_output><empty_output><jupyter_text>Click here for the solution``` Use the sample function to create a vector of indices that will be usedset.seed(1234)ind <- sample(2, nrow(balance_scale), replace=TRUE, prob=c(0.7, 0.3)) Create the training and test data from the dataset using indbscale.train <- balance_scale[ind==1, 6:8]bscale.test <- balance_scale[ind==2, 6:8] Create the target vectors for the training and test data from the dataset using indbscale.trainLabels <- balance_scale[ind==1, 1]bscale.testLabels <- balance_scale[ind==2, 1]    ```  e). Implementation of k-nearest neighbors for classificationNow we will use the knn algorithm from the 'class' package to perform our classification task. This will be done using the knn command to determine the class name (R,L,B) of our test data based on our training data and our choice of k.<jupyter_code>## Your Answer Code ##<jupyter_output><empty_output><jupyter_text>Click here for the solution``` Use the knn command to make predictions on the Class_Name of the test dataknn_class <- knn(train = bscale.train, test = bscale.test, cl = bscale.trainLabels, k=3) Find the number of incorrectly classified pointscorrect <- which(knn_class == bscale.testLabels, arr.ind = TRUE)incorrect <- which(knn_class != bscale.testLabels, arr.ind = TRUE)cat("Number of incorrectly classified points:",length(incorrect),"\n") Find the proportion of correctly classified pointsproportion_correct <- length(correct)/length(bscale.testLabels)cat("Proportion of correctly classified points", proportion_correct,"\n")    ```  f). K-nearest neighbors regression using the kknn commandThe idea of using regression is by far predicting the continious response variable, instead of class variables, here the method is similar to localized linear regressions a.k.a LOESS, technique used, where $k$ is equivalent to the spread of kernel function around the point. [Reference](http://127.0.0.1:44104/help/library/kknn/doc/paper399.pdf) is good starting point to know about this methodology<jupyter_code>## Your Answer Code ##<jupyter_output><empty_output><jupyter_text>Click here for the solution``` Run the knn regression using the kknn commandknn_reg <- kknn(formula = Differences ~ ., train=bscale.train, test=bscale.test, k=3) Find the number of incorrectly classified pointsincorrect_reg <- which(knn_reg$fitted.values != bscale.test$Differences, arr.ind = TRUE)cat("Number of incorrectly classified points:", length(incorrect_reg), "\n"); Find the proportion of correctly classified pointscorrect_reg <- which(knn_reg$fitted.values == bscale.test$Differences, arr.ind = TRUE)cat("Proportion of correctly classified points", length(correct_reg)/length(bscale.test$Differences), "\n") Display the first few rows of the regression estimates of the differences and their true valueshead(cbind(knn_reg$fitted.values,bscale.test$Differences))    ``` We can also determine the optimal value of k using the train.kknn command.<jupyter_code>## Your Answer Code ##<jupyter_output><empty_output><jupyter_text>Click here for the solution```best_reg <- train.kknn(formula = Differences ~ ., data=bscale.train, kmax=8)best_reg$best.parameters  ``` The results indicate that the best value for k, in terms of knn regression, is k=2. This value makes sense given that the k-nearest neighbors regression algorithm estimates the outcome using a weighted average of the k nearest neighbors, weighted by the inverse of their distance.<jupyter_code># Run the knn regression again using the kknn command with k=2
knn_reg2 <- kknn(formula = Differences ~ ., train=bscale.train, test=bscale.test, k=2)

# Find the number of incorrectly classified points
incorrect_reg2 <- which(knn_reg2$fitted.values != bscale.test$Differences, arr.ind = TRUE)
cat("Number of incorrectly classified points:", length(incorrect_reg2),"\n")
# Find the proportion of correctly classified points
correct_reg2 <- which(knn_reg2$fitted.values == bscale.test$Differences, arr.ind = TRUE)
cat("Proportion of correctly classified points",length(correct_reg2)/length(bscale.test$Differences),"\n")

# Display the first few rows of the new regression estimates of the differences and their true values
head(cbind(knn_reg2$fitted.values,bscale.test$Differences))<jupyter_output><empty_output><jupyter_text>As can be seen, the estimates of the regression using k = 2 are closer to the true values with almost double the accuracy.  g). K-nearest neighbors classification using the kknn command We can also use the kknn command for classification:<jupyter_code>## Your Answer Code ##<jupyter_output><empty_output><jupyter_text>Click here for the solution``` The kknn function can also be used for classificationknn_class2 <- kknn(formula = Class_Name ~ ., train=subset(balance_scale, select=c(Class_Name,Right_Product,Left_Product,Differences))[ind==1,], test=subset(balance_scale, select=c(Class_Name,Right_Product,Left_Product,Differences))[ind==2,], k=3) Find the number of incorrectly classified pointsincorrect_class2 <- which(knn_class2$fitted.values != bscale.testLabels, arr.ind = TRUE)cat("Number of incorrectly classified points:",length(incorrect_class2),"\n")    ``` Now let's try finding the optimal value of k for classification.<jupyter_code>best_class <- train.kknn(formula = Class_Name ~ ., data=subset(balance_scale, select=c(Class_Name,Right_Product,Left_Product,Differences))[ind==1,], kmax=8)
best_class$best.parameters
# Using k=1
knn_class3 <- kknn(formula = Class_Name ~ ., train=subset(balance_scale, select=c(Class_Name,Right_Product,Left_Product,Differences))[ind==1,], test=subset(balance_scale, select=c(Class_Name,Right_Product,Left_Product,Differences))[ind==2,], k=1)
# Find the number of incorrectly classified points
incorrect_class3 <- which(knn_class3$fitted.values != bscale.testLabels, arr.ind = TRUE)
cat("Number of incorrectly classified points:", length(incorrect_class3),"\n")<jupyter_output><empty_output><jupyter_text>The results of using k=1 are apparently similar to those of using k=3.     Exercise 2: Second Language Immersion Schools in Canada (Classification)French immersion programs are designed for students who are not native French speakers or students who do not speak French at home. These programs provide instruction in the classroom entirely in French until the end of grade 3, although certain specialist subjects may be instructed in English. We are provided with a data set of Canadian schools offering French immersion programs ("Second_Language_Immersion_Schools_in_Canada.csv"), which contains information regarding these schools such as their names, provinces and coordinates.Suppose we would like to determine the province in which a certain school or other place of interest is located in Canada, given that we have the coordinates (latitude and longitude) of this place. We can do this by using the k-nearest neighbor algorithm.  a). Download the data set, load the data and view its structureWe can use the download.file command to download the data set, "Second_Language_Immersion_Schools_in_Canada.csv", load the data using the read.csv command, and view its structure using the str command.<jupyter_code># Load the data and view the structure
schools <- read.csv("https://ibm.box.com/shared/static/uummw8ijp41gn3nfkuipi78xnalkss4c.csv", sep = ",")
str(schools)<jupyter_output><empty_output><jupyter_text>b). Clean the dataFor the task we would like to do we only need a few variables of the data. These variables are school name (just for viewing), province, latitude and longitude. We will also change the variable names to cleaner names where possible.<jupyter_code># Create a subset of the data containing the variables province, latitude, and longitude
schools.sub <- subset(schools, select=c(name, province.name..english, latitude, longitude))
head(schools.sub)
# Change the column names to cleaner names
colnames(schools.sub) <- c("name", "province", "latitude", "longitude")
head(schools.sub)<jupyter_output><empty_output><jupyter_text>c). Visualize the data on a mapWe will now install and load packages that will enable us to visualize our data by displayiing it on a map. This should make it easier to see clearly the predictions made by the knn algorithm.<jupyter_code># Installing leaflet maps
install.packages("leaflet")
library(leaflet)
# Packages used to display the maps in this notebook
library(htmlwidgets)
library(IRdisplay)<jupyter_output><empty_output><jupyter_text>Next we will set up a default parameters for the setView attribute of the map we are creating.<jupyter_code># Establish the limits of our default visualization
lower_lon = -140
upper_lon = -50
lower_lat = 40
upper_lat = 65
# Establish the center of our default visualization
center_lon = (lower_lon + upper_lon)/2
center_lat = (lower_lat + upper_lat)/2
# Set the zoom of our default visualization
zoom = 4<jupyter_output><empty_output><jupyter_text>Now we create and display our map<jupyter_code># Create a leaflet map
schools_map <- leaflet(schools.sub) %>%
  setView(center_lon,center_lat, zoom)%>% 
  addProviderTiles("OpenStreetMap.BlackAndWhite")%>% # set the map that we want to use as background
  addCircleMarkers(lng = schools.sub$longitude, 
                   lat = schools.sub$latitude, 
                   popup = schools.sub$name, # pop-ups will show the name of school if you click on a data point
                   fillColor = "Black", # colors of the markers will be black
                   fillOpacity = 1, # the shapes will have maximum opacity
                   radius = 4, # radius determine the size of each shape
                   stroke = F) # no stroke will be drawn in each data point

saveWidget(schools_map, file="schools_map.html", selfcontained = F) #saving the leaflet map in html
display_html(paste("<iframe src=' ", 'schools_map.html', " ' width='100%' height='400'","/>")) #display the map !<jupyter_output><empty_output><jupyter_text>d). Implementation of k-nearest neighborsNow we will use the knn algorithm from the 'class' package to perform our classification task. This will be done by first separating our data into training and test groups, selecting our k parameter value, and then using them to make predictions of the provinces of our test data using the knn command.<jupyter_code>## Your Answer Code ##<jupyter_output><empty_output><jupyter_text>Click here for the solution``` Use the sample function to create a vector of indices that will be used to split our data into training and test dataset.seed(1234)ind <- sample(2, nrow(schools.sub), replace=TRUE, prob=c(0.7, 0.3))  This creates a vector of equal length to our data, with approximately 70% of the values being 1 and the remaining values are 2   Create training and test sets containing only the geographical locations schools.train <- schools.sub[ind==1,3:4]schools.test <- schools.sub[ind==2,3:4] Note: Normalizing or standardizing the geographical attributes of the data will not be helpful since values are already on the same scale  Create the target vectors for the training and test data from the original dataset using indschools.trainLabels <- schools.sub[ind==1,2]schools.testLabels <- schools.sub[ind==2,2] Run the knn algorithm to classify test data points into the different provincesprov_pred <- knn(train = schools.train, test = schools.test, cl = schools.trainLabels, k=3) Find the number of incorrectly classified pointscorrect_provinces <- which(prov_pred == schools.testLabels, arr.ind = TRUE)incorrect_provinces <- which(prov_pred != schools.testLabels, arr.ind = TRUE)cat("Number of incorrectly classified points:", length(incorrect_provinces),"\n") Find the proportion of correctly classified pointsproportion_of_correct_provinces <- length(correct_provinces)/length(schools.testLabels)cat("Proportion of correctly classified points", proportion_of_correct_provinces,"\n")    ```  e). Plot the classification predictions of k-nearest neighborsNow we will link the knn predictions of the provinces with the coordinates of the test data, and display them on a map.<jupyter_code># Link the knn predictions with the test data
schools.test$prov <- prov_pred
# Plot the test data with predicted provinces
color <- colorFactor(topo.colors(10), schools.test$prov)
schools_map2 <- leaflet(schools.test) %>%
  setView(center_lon,center_lat, zoom)%>% 
  addProviderTiles("OpenStreetMap.BlackAndWhite")%>% # set the map that we want to use as background
  addCircleMarkers(lng = schools.test$longitude, 
                   lat = schools.test$latitude, 
                   popup = schools.test$prov, # pop-ups will show the predicted province if you click on a data point
                   fillColor = ~color(prov),
                   fillOpacity = 1, # the shapes will have maximum opacity
                   radius = 4, # radius determine the size of each shape
                   stroke = F) # no stroke will be drawn in each data point

saveWidget(schools_map2, file="schools_map2.html", selfcontained = F) #saving the leaflet map in html
display_html(paste("<iframe src=' ", 'schools_map2.html', " ' width='100%' height='400'","/>")) #display the map !<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08404377531894311, 'fraction_numerical': 0.009432251042989299, 'mean_word_length': 5.471048513302034, 'pattern_counts': {'":': 0, '<': 116, '<?xml version=': 0, '>': 77, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 1, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '17', 'dir': 'jupyter-structured-clean-dedup', 'id': '336711', 'license': 'Apache-2.0', 'n_tokens_mistral': 4720, 'n_tokens_neox': 4462, 'n_words': 1835, 'path': 'notebooks/machine-learning/ML0151EN-Exercise-K-NearestNeighbors.ipynb', 'repo_name': 'IBMDeveloperUK/r-workshops', 'size': '16539'}