<jupyter_start><jupyter_text>Naive Bayesian ClassifiersIn this notebook, we'll be focusing on **_Naive Bayesian Classifiers_**, or "Naive Bayes" for short.  This is an algorithm that uses **_Bayes' Theorem_** to make a classification based on probability.  In case you're unfamiliar with Bayes' Theorem, let's look at the formula:Don't worry if you've seen this mathematical notation before. In plain English, that formula reads:"The probability of A given B equals the probability of B given A, times the probability of A, divided by the probability of B".  Let's run through an example case here and see if we can demystify this equation a little bit more. Scenario: Spam DetectionWe have a dataset of emails, and we're trying to build a classifier that can predict if an email is spam or not by examining the words based in the emails.  Each email in our training set has been labeled as "spam" or "ham" (a real email, not spam).  We've counted each word used in every email, and found the following:**_65% of the emails in the dataset are "Spam"._****_"Spam"_** emails contain the word _"deal"_ 80% of the time, and _"win"_ 40% of the time.  **_35% of the emails in the datasert are "Ham"._****_"Ham"_** emails contain the word _"deal"_ 17% of the time, and _"win"_ 6% of the time.  The next email we try to predict contains the both words "deal" and "win". Given the information above, we can plug these numbers into Bayes' Theorem and predict the likelihood that this is email Spam. P(Spam|deal, win) = (P(win, deal|Spam) * P(Spam)) / P(deal, win)This can be further broken down into: P(Spam|deal) \* P(Spam|win) = P(deal|Spam) \* P(Spam) \*  P(win|Spam) \* P(Spam) / P(deal|Spam) + P(deal|!Spam) \* P(win|Spam) + P(win|!Spam)In the equation above, "P(deal|!Spam)" can be read as "the percentage that 'deal' occurs in 'Ham' emails".  On the next step, we'll start defining the probabilities for everything in that equation so we can plug them in:1. P(deal|Spam) = .81. P(win|Spam) = .41. P(Spam) = .651. P(deal|!Spam) = .171. P(win|!Spam) = .061. P(!Spam) = .35Let's replace some of these terms with the probabilities listed above and see how it works out:(.8 \* .65 \* .4 \* .65) / .8 \* .65 + .35 \* .17 \* .4 \* .65 + .35 \* .6 = **0.922595** Based on the math from Bayes' Theorem, we can predict probability that a new email containing both "deal" and "win" is "Spam" is approximately **92.2%**!Using Naive Bayes in the Real WorldIn the above example, we did the math by hand.  That isn't very practical in the real world.  Luckily, `sklearn` contains some awesome implementations of Naive Bayesian Classifiers (and regressors!).  For this assignment, we're going to use a `GaussianNB()` object.  There are a few different kinds of Naive Bayesian Classifiers, but for this one we'll stick to one that assumes our data follows a Gaussian (normal) distribution.  Let's Get Started!<jupyter_code>import numpy as np
np.random.seed(0)
from sklearn.naive_bayes import GaussianNB
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.metrics import accuracy_score, f1_score


# Use load_iris() to load the data into the iris variable, and then assign iris.data and iris.target to the appropriate
# variables
iris = None
data = None
labels = None

# Use train_test_split to split the data into X_train, X_test, y_train, and y_test variables


# Create a GaussianNB() object and fit it using the training data
clf = None

# Use the fitted model to create predictions for the X_test data.
preds = None


# Run it all and see how you did!
print(preds)
print(y_test)
print("accuracy score: {}".format(accuracy_score(y_test, preds)))
print("f1 score: {}".format(f1_score(y_test, preds)))<jupyter_output>None
[0 2 1 2 1 1 0 1 1 0 2 1 0 2 1 2 0 0 2 1 0 2 2 0 2 2 0 1 0 2 1 1 2 0 0 2 2
 2]<jupyter_text>Some CaveatsYou may have wondered why this particular model is a called a **_Naive_** Bayesian Classifier.  In this scenario, the word "Naive" simply means that the model makes the "naive" assumption that all features are independent of one another.  This leads us to the main caveat of this model--if you have feature columns that are highly correlated, this model may not work as well as we'd like.  **_If you're going to use Naive Bayes, make sure you check for highly correlated features beforehand!_**Where to Go From HereFor the latter part of this assignment, you're going to use the famous Pima Indians Diabetes Dataset to build a Naive Bayesian Classifier that predicts whether or not an individual has diabetes.  You'll find the `pima_indians_diabetes.csv` file inside the `datasets` folder.  To build this classifier successfully, you'll want to follow the best practices for loading in and preprocessing a data set that you've learned in class:1. Importing the data1. Exploring the data1. "Cleaning" the data1. Splitting the data into training and testing sets (or using KFold Cross val--more on this below)1. Fitting the model1. Validating the model (checking predictions on the test set)Be sure to consider the following questions as you solve this problem:* How will you deal with null values?* For this model, does scaling the data improve your results? (HINT: test your assumption!)On top of cleaning and preprocessing this data set, you'll also use **_Cross Validation_** to get a better measure of the accuracy of your model.  We did not use K Fold Cross Validation in the above model on purpose--instead, you'll need to work your way through `sklearn`'s [model_selection documentation](http://scikit-learn.org/stable/modules/classes.htmlmodule-sklearn.model_selection) to figure out how to effectively make use of cross-validation.  (**_Hint:_** There are several ways to implement cross validation using sklearn.  In the `model_selection` section of the documentation, pay special attention to the `KFold` object, as well as the methods available under the _Model Selection_ subsection.)Good luck!<jupyter_code># path to file: "datasets/pima_indians_diabetes.csv". The first row of the .csv contains the column names.
# Note that in the "Outcome" column, 0 denotes someone that does NOT have diabetes, and 1 denotes someone that does.  
import pandas as pd<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08003826530612244, 'fraction_numerical': 0.016900510204081634, 'mean_word_length': 4.571047957371226, 'pattern_counts': {'":': 0, '<': 8, '<?xml version=': 0, '>': 8, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '110478', 'license': 'MIT', 'n_tokens_mistral': 1915, 'n_tokens_neox': 1744, 'n_words': 940, 'path': 'site/public/courses/DS-2.1/Assignments/03_Naive_Bayesian_Classifiers.ipynb', 'repo_name': 'KitsuneNoctus/makeschool', 'size': '6272'}