<jupyter_start><jupyter_text>Lab 03 - "Supervised Machine Learning: Naive Bayes" AssignmentsEMBA 58/59 - W8/3 - "AI Coding for Executives", University of St. Gallen In the last lab, we saw an application of **supervised machine learning** by using the **Gaussian Naive-Bayes (GNB) classifier** to classify features derived from real-world **Iris flowers**. You learned how to train a model and to evaluate and interpret its results. In this lab, we aim to leverage that knowledge by applying it to a set of self-coding assignments. But before we do so let's start with a motivational video by OpenAI:<jupyter_code>from IPython.display import YouTubeVideo
# OpenAI: "Solving Rubik's Cube with a Robot Hand"
YouTubeVideo('x4O8pojMF0w', width=1000, height=500)<jupyter_output><empty_output><jupyter_text>As always, pls. don't hesitate to ask all your questions either during the lab, post them in our CANVAS (StudyNet) forum (https://learning.unisg.ch), or send us an email (using the course email).  1. Assignment Objectives: Similar today's lab session, after today's self-coding assignments you should be able to:> 1. Know how to setup a **notebook or "pipeline"** that solves a simple supervised classification task.> 2. Recognize the **data elements** needed to train and evaluate a supervised machine learning classifier. > 3. Understand how a generative Gaussian **Naive-Bayes (NB)** classifier can be trained and evaluated.> 4. Know how to use Python's sklearn library to **train** and **evaluate** arbitrary classifiers.> 5. Understand how to **evaluate** and **interpret** the classification results.  2. Setup of the Jupyter Notebook Environment Similarly to the previous labs, we need to import a couple of Python libraries that allow for data analysis and data visualization. In this lab will use the `Pandas`, `Numpy`, `Scikit-Learn`, `Matplotlib` and the `Seaborn` library. Let's import the libraries by the execution of the statements below:<jupyter_code># import the numpy, scipy and pandas data science library
import pandas as pd
import numpy as np

# import sklearn data and data pre-processing libraries
from sklearn import datasets
from sklearn.model_selection import train_test_split

# import sklearn naive.bayes classifier library
from sklearn.naive_bayes import GaussianNB

# import sklearn classification evaluation library
from sklearn import metrics
from sklearn.metrics import confusion_matrix 

# import matplotlib data visualization library
import matplotlib.pyplot as plt
import seaborn as sns<jupyter_output><empty_output><jupyter_text>Enable inline Jupyter notebook plotting:<jupyter_code>%matplotlib inline<jupyter_output><empty_output><jupyter_text>Use the `Seaborn`plotting style in all subsequent visualizations:<jupyter_code>plt.style.use('seaborn')<jupyter_output><empty_output><jupyter_text>3. Gaussian Naive-Bayes (NB) Classification Assignments  3.1 Iris Dataset Download and Pre-Processing The **Iris Dataset** is a classic and straightforward dataset often used as a "Hello World" example in multi-class classification. This data set consists of measurements taken from three different types of iris flowers (referred to as **Classes**),  namely the Iris Setosa, the Iris Versicolour and the Iris Virginica, and their respective measured petal and sepal length (referred to as **Features**). (Source: http://www.lac.inpe.br/~rafael.santos/Docs/R/CAP394/WholeStory-Iris.html) In total, the dataset consists of **150 samples** (50 samples taken per class) as well as their corresponding **4 different measurements** taken for each sample. Please, find below the list of the individual measurements:>- `Sepal length (cm)`>- `Sepal width (cm)`>- `Petal length (cm)`>- `Petal width (cm)`Further details of the dataset can be obtained from the following puplication: *<NAME>. "The use of multiple measurements in taxonomic problems" Annual Eugenics, 7, Part II, 179-188 (1936); also in "Contributions to Mathematical Statistics" (John Wiley, NY, 1950)."*Let's load the dataset and conduct a preliminary data assessment:<jupyter_code>iris = datasets.load_iris()<jupyter_output><empty_output><jupyter_text>Print and inspect the names of the four features contained in the dataset:<jupyter_code>iris.feature_names<jupyter_output><empty_output><jupyter_text>Determine and print the feature dimensionality of the dataset:<jupyter_code>iris.data.shape<jupyter_output><empty_output><jupyter_text>Determine and print the class label dimensionality of the dataset:<jupyter_code>iris.target.shape<jupyter_output><empty_output><jupyter_text>Print and inspect the names of the three classes contained in the dataset:<jupyter_code>iris.target_names<jupyter_output><empty_output><jupyter_text>Let's briefly envision how the feature information of the dataset is collected and presented in the data:  Let's inspect the top five feature rows of the Iris Dataset:<jupyter_code>pd.DataFrame(iris.data, columns=iris.feature_names).head(5)<jupyter_output><empty_output><jupyter_text>Let's also inspect the top five class labels of the Iris Dataset:<jupyter_code>pd.DataFrame(iris.target, columns=["class"]).head(5)<jupyter_output><empty_output><jupyter_text>Let's now conduct a more in depth data assessment. Therefore, we plot the feature distributions of the Iris dataset according to their respective class memberships as well as the features pairwise relationships. Pls. note that we use Python's **Seaborn** library to create such a plot referred to as **Pairplot**. The Seaborn library is a powerful data visualization library based on the Matplotlib. It provides a great interface for drawing informative statstical graphics (https://seaborn.pydata.org).<jupyter_code># init the plot
plt.figure(figsize=(10, 10))

# load the dataset also available in seaborn
iris_plot = sns.load_dataset("iris")

# plot a pairplot of the distinct feature distributions
sns.pairplot(iris_plot, diag_kind='hist', hue='species');<jupyter_output><empty_output><jupyter_text>3.2 Gaussian Naive Bayes Model Training and Evaluation  We recommend you to try the following exercises as part of the self-coding session:**Exercise 1: Train and evaluate the prediction accuracy of different train- vs. eval-data ratios.** > Change the ratio of the Iris training data vs. test data split to **30%/70%** (currently 70%/30%), fit a Gaussian Naive Bayes model using the `Scikit-Learn` (https://scikit-learn.org) library and compute the classification accuracy of the trained model on the held-out test data.<jupyter_code># ***************************************************
# INSERT YOUR SOLUTION/CODE HERE
# ***************************************************

# set evaluation fraction to 70%
eval_fraction_ex1 = 0.7

# set random seed
seed = 42

# 70% training and 30% evaluation
x_train_ex1, x_eval_ex1, y_train_ex1, y_eval_ex1 = train_test_split(iris.data, iris.target, test_size=eval_fraction_ex1, random_state=seed, stratify=None)

# init the Gaussian Naive Bayes classifier
gnb_ex1 = GaussianNB(priors=None)

# train the Gaussian Naive Bayes classifier
gnb_ex1.fit(x_train_ex1, y_train_ex1)

# predict the labels of the evaluation data set
y_pred_ex1 = gnb_ex1.predict(x_eval_ex1)

# determine and print the accuracy score
print("Classification Accuracy: ", metrics.accuracy_score(y_eval_ex1, y_pred_ex1))

# determine and print the number of mislabeled data points
print("Mislabeled points out of a total {} points: {}".format(x_eval_ex1.shape[0], np.sum(y_eval_ex1 != y_pred_ex1)))<jupyter_output>Classification Accuracy:  0.9428571428571428
Mislabeled points out of a total 105 points: 6<jupyter_text>**Exercise 2: Train and evaluate the prediction accuracy of different train- vs. eval-data ratios.**> Now, repeat the experiment a second time using a **10%/90%** Iris train vs. test data split fraction. Next, fit again a Gaussian Naive Bayes model using the `Scikit-Learn` library and also compute the classification accuracy of the trained model on the held-out test data. What can be observed when comparing the results of exercise 1 and 2 in terms of classification accuracy?<jupyter_code># ***************************************************
# INSERT YOUR SOLUTION/CODE HERE
# ***************************************************

# set evaluation fraction to 10%
eval_fraction_ex2 = 0.1

# set random seed
seed = 42

# 70% training and 30% evaluation
x_train_ex2, x_eval_ex2, y_train_ex2, y_eval_ex2 = train_test_split(iris.data, iris.target, test_size=eval_fraction_ex2, random_state=seed, stratify=None)

# init the Gaussian Naive Bayes classifier
gnb_ex2 = GaussianNB(priors=None)

# train the Gaussian Naive Bayes classifier
gnb_ex2.fit(x_train_ex2, y_train_ex2)

# predict the labels of the evaluation data set
y_pred_ex2 = gnb.predict(x_eval_ex2)

# determine and print the accuracy score
print("Classification Accuracy: ", metrics.accuracy_score(y_eval_ex2, y_pred_ex2))

# determine and print the number of mislabeled data points
print("Mislabeled points out of a total {} points: {}".format(x_eval_ex2.shape[0], np.sum(y_eval_ex2 != y_pred_ex2)))<jupyter_output>Classification Accuracy:  1.0
Mislabeled points out of a total 15 points: 0<jupyter_text>**Exercise 3: Calculate the true-positive as well as the false-positive rate of the Iris versicolor vs. virginica.**> Compute and visualize the confusion matrices of (1) the experiment exhibiting a 30%/70% ratio of training data vs. evaluation data and (2) the experiment exhibiting a 10%/90% ratio of training data vs. evaluation data. Similar to the guided lab session you might want to use the `confusion_matrix` function that comes with the `Scikit-Learn` library. (Source: https://en.wikipedia.org/wiki/Confusion_matrix)<jupyter_code># ***************************************************
# INSERT YOUR SOLUTION/CODE HERE
# ***************************************************

# determine the prediction confusion matrix
mat_ex1 = confusion_matrix(y_eval_ex1, y_pred_ex1)

# init the plot
plt.figure(figsize=(5, 5))

# plot confusion matrix heatmap
sns.heatmap(mat_ex1.T, square=True, annot=True, fmt='d', cbar=False, cmap='BuGn_r', xticklabels=iris.target_names, yticklabels=iris.target_names)

# add plot axis labels
plt.xlabel('[true label]')
plt.ylabel('[predicted label]')

# add plot title
plt.title('Gaussian Naive Bayes Confusion Matrix');
# ***************************************************
# INSERT YOUR SOLUTION/CODE HERE
# ***************************************************

# determine the prediction confusion matrix
mat_ex2 = confusion_matrix(y_eval_ex2, y_pred_ex2)

# init the plot
plt.figure(figsize=(5, 5))

# plot confusion matrix heatmap
sns.heatmap(mat_ex2.T, square=True, annot=True, fmt='d', cbar=False, cmap='BuGn_r', xticklabels=iris.target_names, yticklabels=iris.target_names)

# add plot axis labels
plt.xlabel('[true label]')
plt.ylabel('[predicted label]')

# add plot title
plt.title('Gaussian Naive Bayes Confusion Matrix');<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11756658153958409, 'fraction_numerical': 0.01659978110178767, 'mean_word_length': 6.319759679572764, 'pattern_counts': {'":': 0, '<': 60, '<?xml version=': 0, '>': 72, 'https://': 4, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '74427', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 3407, 'n_tokens_neox': 3156, 'n_words': 1200, 'path': 'resources/lab_03/lab_03a_solution.ipynb', 'repo_name': 'HSG-AIML/LabAI-Coding', 'size': '10969'}