<jupyter_start><jupyter_text>Class 4-1: Supervised learning<jupyter_code>import pandas as pd
from sklearn import tree
import matplotlib.pyplot as plt
from pandas.plotting import scatter_matrix
from sklearn.model_selection import train_test_split
from sklearn import metrics
%matplotlib inline<jupyter_output><empty_output><jupyter_text>Today we're going to look at GSS data asking people whether the government is spending too much or too little on various problems. We're going to try to predict one of the answers from all of the others.  Exploring the data<jupyter_code># load it

# take a look

# Our usual GSS cleanup


# For a decision tree to work we can't have any missing values. Drop all rows with any NAs<jupyter_output><empty_output><jupyter_text>The simplest way to see if some variables are going to be predictable from others is to check out the overall correlations.  Building a decision-treeWe're going to use a machine learning technique called a decision tree to try to predict the answer on one question from the answers on all the others.<jupyter_code># By changing this line and re-running the workbook, we can see how predictiable different variables are or aren't.<jupyter_output><empty_output><jupyter_text>Actually applying the decision tree is very much like doing a regresion.<jupyter_code># Fit a decision tree to the test data<jupyter_output><empty_output><jupyter_text>Ok, now we need to evaluate the result, by seeing how well it can predict our question from all the others.<jupyter_code># Get the predictions

# What percentage did it predict right?

# What mistakes did it make?<jupyter_output><empty_output><jupyter_text>So what is this tree thing? We can actually draw it To get this working, you will need to install `GraphViz`. On Windows, download it [here](https://graphviz.gitlab.io/download/). On Mac, try `brew install graphviz`<jupyter_code>from sklearn.tree import export_graphviz
import graphviz<jupyter_output><empty_output><jupyter_text>A more realistic testThe test of a classifier is not whether it can reproduce the output training data -- it's how well it does on data it's never seen before. So let's split our data into training and test sets, and try this again.<jupyter_code># Split off a random third of this data to use for testing.

# The x values don't contain the column we want to predict. The y value is only that column.

# Fit a decision tree to the training data only

# What percentage did it predict right on the training data?

# What percentage did it predict right on the test data?

# What mistakes did it make?<jupyter_output><empty_output><jupyter_text>Random Forest: Improving accuracy with a better classifierSometimes improving accuracy is very difficult. You might just not have a very predictable problem! But we can often do better than just a simple decision tree, by using many trees. This is a "random forest," and for our purposes it's just a drop-in replacement for the classifier.<jupyter_code>from sklearn.ensemble import RandomForestClassifier
# What percentage did it predict right on the test data?

# What mistakes did it make?<jupyter_output><empty_output><jupyter_text>Binary classification metricsIn this section we'll train a basic classifier on our favorite Titantic data, and then calculate some accuracy metrics for a binary predictor where there are only two outcomes: true and false. We'll be using these metrics over and over again in our upcoming discussions of algorithmic accountability.For your reference, Wikipedia has an [amazing chart](https://en.wikipedia.org/wiki/Confusion_matrix) of the various things you can calculate from a binary confusion matrix.<jupyter_code># load titanic.csv once more

# recode the pclass and gender variables so they are numeric

# Set aside a third of the data for testing.

# Set up x and y variables and train a decision tree on the pclass and sex features, to predict survived

# Create ti_test and ti_test values, and use the classifier to predict yti_test_pred

# Print out the confusion matrix for the classifier. Use the DataFrame trick we saw in class to label the axes.

# Ok! What are the number of true positives, true negatives, false positives, and false negatives?
TP = 
TN = 
FP = 
FN = 
# Using only the varaibles TP,TN,FP,FN, calculate and print the overall accuracy

# Check that we have the calculation right by comparing to metrics.accuracy_score

# Using only the varaibles TP,TN,FP,FN, calculate and print the false positive rate and the false negative rate

# Now compute and print the precision aka positive predictive value, again from these four variables.<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.04814098431119708, 'fraction_numerical': 0.0004298302170642596, 'mean_word_length': 5.1888297872340425, 'pattern_counts': {'":': 0, '<': 37, '<?xml version=': 0, '>': 37, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '476507', 'license': 'CC-BY-3.0', 'n_tokens_mistral': 1258, 'n_tokens_neox': 1210, 'n_words': 648, 'path': 'week-4/week-4-1-supervised-learning-class-empty.ipynb', 'repo_name': 'adrianblanco/lede-algorithms', 'size': '4653'}