<jupyter_start><jupyter_text>How to Make Classifications using Decision Trees A Team Project to:* Discover how to use Machine Learning software to build Decision Trees * Experiment with the effects of changinging parameters(hyper parameters) for Machine Learning* Evaluate the quality of automatically generated decision trees<jupyter_code>#@title # Assigning Roles and Responsibilities within each group

#@markdown ---
#@markdown ### Enter Instructor Name:
Instructor_Name = "" #@param {type:"string"}
#@markdown 1. Introduces activities. Assigns roles to participants.
#@markdown 2. Responds for help or clarification request.
#@markdown 3. Collects the Jupiter notebooks from Recorder and Evaluator
#@markdown ---

#@markdown ### Enter Facilitator Name:
Facilitator_Name = "" #@param {type:"string"} 
#@markdown ### Enter Backup Facilitator Name:
Backup_Facilitator_Name = "" #@param {type:"string"} 
#@markdown 1.	Reads aloud each question and ask for volunteers to answer. If there is no volunteer then he/she starts the discussion and asks one participant after another for comments, solutions, answers, or clarifications.  When majority participants agree then she/he asks Recorder to record the answer. Also coordinates discussion about the code execution and the output like any other question.   
#@markdown 2.	Involves each participant equally in the discussions.    
#@markdown 3.	Turn the coordinating role to Evaluator after finishing each activity.  
#@markdown ---

#@markdown ### Enter Recorder Name:
Recorder_Name = "" #@param {type:"string"}
#@markdown ### Enter Backup Recorder Name:
Backup_Recorder_Name = "" #@param {type:"string"}  
#@markdown 1.	Coordinates Zoom screen access. Displays his/her screen when asking questions. Gives access to screen sharing as requested    
#@markdown 2.	Records all answers  for each question  inside the Jupiter Notebook   
#@markdown 3. Use "Run all" in menu "Runtime" and then "Save" Jupiter Workbook with all answers and results.
#@markdown 4. Submit Jupiter notebook with all answers and results of the running code.
#@markdown ---

#@markdown ### Enter Evaluator Name:
Evaluator_Name = "" #@param {type:"string"}
#@markdown ### Enter Backup Evaluator Name:
Backup_Evaluator_Name = "" #@param {type:"string"} 
#@markdown 1.	Keeps track of time for each designated Activity.   
#@markdown 2.	After each activity leads the discussion about material and collects feedback in the form of the table below.   
#@markdown 3. Submit Jupiter notebook with all comments and results of discussion at the end of each activity.  
#@markdown ---

#@markdown ### Enter Participant names
Participant_4_Name = "" #@param {type:"string"} 
Participant_5_Name = "" #@param {type:"string"}
Participant_6_Name = "" #@param {type:"string"}
Participant_7_Name = "" #@param {type:"string"}
Participant_8_Name = "" #@param {type:"string"}

#@markdown 1.	Participates actively in team work to answer all questions.
#@markdown 2.	Executes the code and shares the comments.
#@markdown ---

print("You have chosen {} as the Instructor".format(Instructor_Name))
#@title Download files to work on
import urllib
import os
DOWNLOAD_ROOT = "https://raw.githubusercontent.com/sbhattac/ai-workshop/master/DecisionTree/"
for filename in ("vehicles_binary_sample.csv", "vehicles_binary_test.csv", "vehicles_binary_train.csv", "vehicles_digital_test.csv", "vehicles_digital_train.csv", "vehicles_logical_sample.csv"):
    print("Downloading", filename)
    url = DOWNLOAD_ROOT + filename
    urllib.request.urlretrieve(url, filename)
#@title Library Imports
import pandas as pd  #imports the "pandas" library
import matplotlib.pyplot as plt
from sklearn.tree import DecisionTreeClassifier, export_graphviz, plot_tree
from sklearn.metrics import accuracy_score
import numpy as np
from IPython.display import SVG, display
from graphviz import Source
from ipywidgets import interactive, fixed
from sklearn.metrics import confusion_matrix

from sklearn import tree
#@title Helper Functions
def getColumnIndex(table, col):
    headers = table.columns.values.tolist()
    index = headers.index(col)
    return(index)

def calculateConfusionMatrix(table, col1, col2, pos_value=1, neg_value=0, true_pos_value=1, true_neg_value=0):
    #Note: col2 MUST be the true values, or this will be backwards.
    truePositive= 0
    trueNegative = 0
    falsePositive = 0
    falseNegative = 0
    col1_index = getColumnIndex(table, col1)
    col2_index = getColumnIndex(table, col2)
    for i in range(len(table)):
        if table.iloc[i,col1_index] == pos_value and table.iloc[i,col2_index] == true_pos_value:
            truePositive += 1
        elif table.iloc[i,col1_index] == pos_value and table.iloc[i,col2_index] == true_neg_value:
            falsePositive += 1
        elif table.iloc[i,col1_index] == neg_value and table.iloc[i,col2_index] == true_pos_value:
            falseNegative += 1
        elif table.iloc[i,col1_index] == neg_value and table.iloc[i,col2_index] == true_neg_value:
            trueNegative += 1             
    return(truePositive, falsePositive, trueNegative, falseNegative)

def calculateImpurity2(table, col1, col2, pos_value=1, neg_value=0, true_pos_value=1, true_neg_value=0):
    # NOTE. Col2 must be the ground truth or this will be backwards
    
    truePositive, falsePositive, trueNegative, falseNegative = calculateConfusionMatrix(table, col1, col2, pos_value, neg_value, true_pos_value, true_neg_value)
    total = truePositive + trueNegative + falsePositive + falseNegative

    if (truePositive + falsePositive) > 0:
        impurityLeft = 1 - pow(truePositive/(truePositive + falsePositive),2) - pow(falsePositive/(truePositive + falsePositive),2)
    else:
        impurityLeft = 0
    if (trueNegative + falseNegative) > 0:   
        impurityRight = 1 - pow(falseNegative/(falseNegative + trueNegative),2) - pow(trueNegative/(falseNegative + trueNegative),2) 
    else:
        impurityRight = 0
    
    if total > 0:
        impurity = impurityLeft*((truePositive+falsePositive)/total) + impurityRight*((trueNegative+falseNegative)/total)
    else:
        print("you messed up somewhere")
        return()
    print ("impurityLeft:  ",impurityLeft)
    print ("impurityRight:  ",impurityRight)
    print ("total :  ", impurity)
    return (impurity)

def calculateImpurity(table, col1, col2, pos_value=1, neg_value=0, true_pos_value=1, true_neg_value=0):
    # NOTE. Col2 must be the ground truth or this will be backwards
    
    truePositive, falsePositive, trueNegative, falseNegative = calculateConfusionMatrix(table, col1, col2, pos_value, neg_value, true_pos_value, true_neg_value)
    total = truePositive + trueNegative + falsePositive + falseNegative

    if (truePositive + falsePositive) > 0:
        impurityLeft = 1 - pow(truePositive/(truePositive + falsePositive),2) - pow(falsePositive/(truePositive + falsePositive),2)
    else:
        impurityLeft = 0
    if (trueNegative + falseNegative) > 0:   
        impurityRight = 1 - pow(falseNegative/(falseNegative + trueNegative),2) - pow(trueNegative/(falseNegative + trueNegative),2) 
    else:
        impurityRight = 0
    
    if total > 0:
        impurityBefore = 1 - pow((truePositive+falsePositive)/total,2) - pow((trueNegative+falseNegative)/total,2)
        impurityAfter = impurityLeft*((truePositive+falsePositive)/total) + impurityRight*((trueNegative+falseNegative)/total)
    else:
        return()
#    print ("impurity Initial:  {:.3f}".format(impurityBefore))
    print ("impurity Local:  {:.3f}".format(impurityLeft))
    print ("impurity Visitor:  {:.3f}".format(impurityRight))
#    print ("impurity Total :  {:.3f}".format( impurityAfter))
    return (impurityRight, impurityLeft, impurityAfter)



def decisionTreeForPredictor(table_train, col, label_col):
    # load dataset
    indexList = []
    index = getColumnIndex(table_train, col)
    indexList.append(index)
    # feature matrix
    X_train = table_train.iloc[:,[index]]

    labelIndex = getColumnIndex(table_train, label_col)
    # target vector
    y_train = table_train.iloc[:,labelIndex]

    # feature labels
    labels = table_train.columns[indexList].values.tolist()

    estimator = plot_tree_graphviz_no_test(X_train, y_train, labels, split="best", depth=4, min_split=0.1, min_leaf=0.2)


def selectRowsCondition(table, columnName, operator, value):
    
    operatorList = ['>', '<', '='] 
    if operator not in operatorList:
        print("Please use one of the following operators. ['>', '<', '='] ")
        newTable = table

    else:
        if operator == '>':
            condition = (table[columnName] > value)
        elif operator == '<':
            condition = (table[columnName] < value)
        else:
            condition = (table[columnName] == value)
        newTable = table[condition]

        if newTable.empty:
            print("you dont need go any further")
    return (newTable)

def plot_tree_graphviz(xtrain, xtest, ytrain, ytest, labels, split, depth, min_split, min_leaf=0.2):

    estimator = DecisionTreeClassifier(random_state = 0, 
          criterion = "gini",
          splitter = split,
          max_depth = depth,
          min_samples_split=min_split,
          min_samples_leaf=min_leaf)
    estimator.fit(xtrain, ytrain)
    
    predictions = estimator.predict(xtest)
    score = accuracy_score(ytest, predictions)
    
    print('Accuracy score: {}'.format(score))
    
    graph = Source(tree.export_graphviz(estimator,
      out_file=None,
      feature_names=labels,
      class_names=['Local', 'Visitor'],
      filled = True))
    display(SVG(graph.pipe(format='svg')))
    return estimator

def plot_tree_graphviz_no_test(xtrain, ytrain, labels, split, depth, min_split, min_leaf=0.2):

    estimator = DecisionTreeClassifier(random_state = 0, 
          criterion = "gini",
          splitter = split,
          max_depth = depth,
          min_samples_split=min_split,
          min_samples_leaf=min_leaf)
    estimator.fit(xtrain, ytrain)
            
    graph = Source(tree.export_graphviz(estimator,
      out_file=None,
      feature_names=labels,
      class_names=['Visitor', 'Local'],
      filled = True))
    display(SVG(graph.pipe(format='svg')))
    return estimator

def plot_tree_graphviz_selectColumns(train, test, cols, split, depth, min_split, min_leaf=0.2):
  
  indexList = []
  for element in cols:
    index = getColumnIndex(train, element)
    indexList.append(index)

  xtrain = train.iloc[:,indexList]
  xtest = test.iloc[:, indexList]
  ytrain = train.iloc[:,3]
  ytest = test.iloc[:,3]

  labels = train.columns[indexList].values.tolist()

  estimator = DecisionTreeClassifier(random_state = 0, 
        criterion = "gini",
        splitter = split,
        max_depth = depth,
        min_samples_split=min_split,
        min_samples_leaf=min_leaf)
  estimator.fit(xtrain, ytrain)
  
  predictions = estimator.predict(xtest)
  score = accuracy_score(ytest, predictions)
  
  print('Accuracy score: {}'.format(score))
  
  graph = Source(tree.export_graphviz(estimator,
    out_file=None,
    feature_names=labels,
    class_names=['Visitor', 'Local'],
    filled = True))
  display(SVG(graph.pipe(format='svg')))
  return estimator<jupyter_output><empty_output><jupyter_text>Activity 1. Decisions based on Decision Trees  Activity 1 Practical Problem:You are a new city employee who needs to  differentiate between local and visiting vehicles  based on information about  the frequent stops,  left turns, and slow speed of each vehicle.  The previous employee, a human expert in this area, created the decision tree diagram.  Your manager has given you a  sample data set in a tabular form as the file "vehicles_sample.csv" to compare the real type of vehicle with the class predicted by the diagram.  Fig 1Consider the following diagram in Fig. 1, and answer questions 1-13:![decision tree fig 1](https://raw.githubusercontent.com/sbhattac/ai-workshop/master/DecisionTree/DTFig1.png)<jupyter_code>#@title Activity 1 Questions 1 - 13
#@markdown 1. The diagram is a decision tree (it is actually an upside-down tree). All boxes are referred to as nodes. How many nodes do we have in this diagram?
activity1_answer1 = "" #@param {type:"string"}
#@markdown 2. We refer to the single node on the top of the diagram as root. Decision trees are usually multi-level. We start counting from the top of diagram but do not include root. How many levels of decision tree do we have?
activity1_answer2 = "" #@param {type:"string"}
#@markdown 3. The nodes can be connected to other nodes on a lower level. How many such nodes are in this diagram?
activity1_answer3 = "" #@param {type:"string"}
#@markdown 4. The connections are called branches of the tree.  How many branches are in this diagram?
activity1_answer4 = "" #@param {type:"string"}
#@markdown 5. Nodes that grow two branches ("Yes" and "No") are called decision boxes. They represent a selection condition of the vehicles. What is the selection condition for the decision box in the root?
activity1_answer5 = "" #@param {type:"string"}
#@markdown 6. We refer to selection condition also as "predictor" since we use them for predictions.  How many predictors do we use in this diagram?
activity1_answer6 = "" #@param {type:"string"}
#@markdown 7. The nodes that do not grow any branches are called "leaves" and represent classes of vehicles. How many leaves do we have in the diagram?
activity1_answer7 = "" #@param {type:"string"}
#@markdown 8. Is the number of leaves in the decision tree the same as number of classes? Why?
activity1_answer8 = "" #@param {type:"string"}
#@markdown 9. Can we predict the vehicle class when the vehicle stops 5 times, and makes 4 left turns?
activity1_answer9 = "" #@param {type:"string"}
#@markdown 10. Is it possible to make a decision about the class of the vehicle if we know only that the vehicle made less than 3 stops? If yes what would be the class name?
activity1_answer10 = "" #@param {type:"string"}
#@markdown 11. Is it possible to make a decision about the class of the vehicle if we know only that the vehicle made 3 or more stops?
activity1_answer11 = "" #@param {type:"string"}
#@markdown 12. Is it possible to make a decision about the class of the vehicle when all three predictors of the vehicle have the value of "No"? If yes what would be the class name? 
activity1_answer12 = "" #@param {type:"string"}
#@markdown 13. Do we always need to check three predictors to determine the class of the vehicle?
activity1_answer13 = "" #@param {type:"string"}
#@title #Run the Code
#@markdown 14.  There is a  sample data in a tabular form as the file "vehicles_logical_sample.csv". Run the code below to see the sample of data. 
Vehicles_Logical_Sample = pd.read_csv("vehicles_logical_sample.csv")
display(Vehicles_Logical_Sample)


#@title Activity 1 Questions 15 - 20
#@markdown 15.	The vehicles are numbered starting from 0 through 4. How many vehicles are in the data sample?
activity1_answer15 = "" #@param {type:"string"}
#@markdown 16. The actual (real) class name is provided for each vehicle in the "Vehicle"  column. Why this column is also called "the label"?
activity1_answer16 = "" #@param {type:"string"}
#@markdown 17. If we were to use the decision tree in Fig. 1 to make a prediction of the vehicle class, would it match the label for vehicle number 0?
activity1_answer17 = "" #@param {type:"string"}
#@markdown 18.	Let us consider the vehicle number 3. Is the prediction of the decision tree in Fig. 1 the same as the label?
activity1_answer18 = "" #@param {type:"string"}
#@markdown 19.	Are there any other vehicles  classified incorrectly by the decision tree in Fig. 1?
activity1_answer19 = "" #@param {type:"string"}
#@markdown 20.  In the next section we will discover how to build the decision tree based on a scientific approach. Can we always create a perfect decision tree i.e. the decision tree that would correctly classify all of the vehicles? Why?
activity1_answer20 = "" #@param {type:"string"}<jupyter_output><empty_output><jupyter_text>Activity 2. Choosing Predictors for Decision Trees  Activity 2 Practical Problem:You, a city employee, would like to use a scientific method to evaluate predictors individually to see which one is the best.<jupyter_code>#@title # Run the Code

#@markdown 1. Let us analyze each predictor separately and discover how well it predicts. First let us consider the “Stops < 3” predictor. Run the code below to see only the relevant part of the sample of data.
Vehicles_Logical_Sample[['Stops < 3', 'Vehicle']]
#@title # Activity 2 Questions 2 - 3
#@markdown 2.	How many vehicles made less than 3 stops?
activity2_answer2 = "" #@param {type:"string"} 
#@markdown 3.	How many vehicles made 3 or more stops?
activity2_answer3 = "" #@param {type:"string"}<jupyter_output><empty_output><jupyter_text>Fig 2Consider the following diagram in Fig. 2![decision tree fig 2](https://raw.githubusercontent.com/sbhattac/ai-workshop/master/DecisionTree/DTFig2_0.png)<jupyter_code>#@title # Activity 2 Questions 4 - 11
#@markdown 4. Let us build a decision tree consisting of  only one predictor "Stops < 3"  as shown in Fig.2.  We added the line "samples" in Fig.2 to indicate how many vehicles are considered in each node. In the root we have 5 vehicles to split between two branches based on the prediction. What are the values of the line "samples" in the leaf nodes?
activity2_answer4 = "" #@param {type:"string"} 
#@markdown 5.	Would we correctly predict the vehicle class if we have only information about how often a vehicles stops? How many times would the prediction be correct?
activity2_answer5 = "" #@param {type:"string"} 
#@markdown 6. How many times would the prediction be incorrect?
activity2_answer6 = "" #@param {type:"string"} 
#@markdown 7.	As we discovered, we would have correct and incorrect predictions when using the single predictor "Stops < 3". Let us look only at vehicles that have “Yes” value for this feature. How many times would the prediction be correct? 
activity2_answer7 = "" #@param {type:"string"} 
#@markdown 8. When the prediction matches the true value, we call this subset “true positives”. The subset when the prediction incorrectly makes a positive prediction is called “false positives”. How many  “false positives” do we have? 
activity2_answer8 = "" #@param {type:"string"} 
#@markdown 9. Let us look only at vehicles that have “No” value for this feature. How many times would the prediction be correct? 
activity2_answer9 = "" #@param {type:"string"} 
#@markdown 10.	We call this vehicle subset “true negatives”. The negative predictions can also be incorrect. These would be called “false negatives”. How many  “false negatives” do we have? 
activity2_answer10 = "" #@param {type:"string"} 
#@markdown 11. When the single predictor has larger subsets of "true positive" and  "true negative" then the prediction is better or worse? Choose the best answer.
activity2_answer11 = "Choose answer" #@param ["Choose answer", "Better", "Worse"]<jupyter_output><empty_output><jupyter_text>As a reminder, the metrics of the predictions can be captured by these four terms:   $T_p$ is the number of True positves  $F_p$ is the number of False positves   $T_n$ is the number of True negatives  $F_n$ is the number of False negatives<jupyter_code>#@title Run the code

#@markdown 12. For larger data sets we use computing functions to calculate these metics. Run the code below

true_positive, false_positive, true_negative, false_negative = calculateConfusionMatrix(Vehicles_Logical_Sample, 
                                                'Stops < 3', 'Vehicle', 'Yes', 'No', 'Local', 'Visitor')
print( "Metrics for predictor 'Stops < 3'")
print ('True positives: {}'.format(true_positive))
print ('False positives: {}'.format(false_positive))
print ('True negatives: {}'.format(true_negative))
print ('False negatives: {}'.format(false_negative))
#@title # Activity 2 Questions 13 - 16
#@markdown 13. Are the computed values above the same as yours?
activity2_answer13 = "" #@param {type:"string"} 
#@markdown 14. We are using sample of data to "learn" the best decision tree. When we want to apply the decision tree to to different data sample we can only refer to probabilities of belonging to these subsets. In our  case expected probability of being "true positive" in the "Local" node will be 2/2 = 100%. What is the probability of being "false positive" in the "Local" node?
activity2_answer14 = "" #@param {type:"string"} 
#@markdown 15. What is the probability of being "true negative" in the "Visitor" node ?
activity2_answer15 = "" #@param {type:"string"} 
#@markdown 16. What is the probability of being "false negative" in the "Visitor" node "?
activity2_answer16 = "" #@param {type:"string"}<jupyter_output><empty_output><jupyter_text>You should have gotten the probablity of true positives as 1.   You should have gotten the probablity of false positives as 0.   Let us designate $p_1 $ as the probability of true positives.  Let us designate $p_2 $ as the probability of false positives. The mathematical function that captures this is positive ("Local" node) gini impurity is defined as:   $$gini_p = 1 - p_1^2 - p_2^2 $$<jupyter_code>#@title # Activity 2 Questions 17 - 18
#@markdown  17. Use this formula to compute gini impurity for the positive ("Local") branch.  
activity2_answer17 = "" #@param {type:"string"} 
#@markdown 18. The "Local" node contains vehicles perfectly classified, and the gini impurity is 0. What does that tell us about the probabilities of the positive branch?
activity2_answer18 = "" #@param {type:"string"}<jupyter_output><empty_output><jupyter_text>The mathematical function that captures this is gini impurity formula for the negative branch is given by:You should have gotten the probablity of true negatives as 0.667.   You should have gotten the probablity of false positives as 0.333.   Let us designate $p_3 $ as the probability of true positives.  Let us designate $p_4 $ as the probability of false positives. The mathematical function that captures this is negative ("Visitor" node) gini impurity is defined as:   $$gini_p = 1 - p_3^2 - p_4^2 $$<jupyter_code>#@title # Activity 2 Questions 19
#@markdown 19. Calculate the gini impurity of the negative branch ("Visitor" node). 
activity2_answer19 = "" #@param {type:"string"}


#@title Run the Code 
#@markdown 20. For larger data sets we use computing functions to get the gini impurity. Run the code below.
print ("gini impurity for predictor 'Stops < 3'" )
giniImpurity = calculateImpurity(Vehicles_Logical_Sample, 'Stops < 3', 'Vehicle', "Yes", "No", 'Visitor', 'Local')
#@title # Activity 2 Questions 21
#@markdown 21. Compare your gini impurity answers for the "Local" and "Visitor" nodes. Are they the same as the output above? 
activity2_answer21 = "" #@param {type:"string"}<jupyter_output><empty_output><jupyter_text>The total gini impurity is important since it captures overall "value" of the predictor. It can be computed by the weighted average of impurities of each branch node. $$ gini_t = gini_p * \frac{T_p + F_p}{total} + gini_n * \frac{T_n + F_n}{total}$$ where:   $gini_t$ is the total gini  $gini_p$ is the gini for the positive branch   $T_p$ is the number of True positves  $F_p$ is the number of False positves   $gini_n$ is the gini for the negative branch   $T_n$ is the number of True negatives  $F_n$ is the number of False negativestotal is the number of True positives + False positives + True negatives + False negatives.Since the value of $gini_p$ was 0, the first term on the right hand side of the  equation is also 0. Therefore, the equation can be simplified to $$ gini_t =  gini_n * \frac{T_n + F_n}{total}$$ (but only in this specific case.)<jupyter_code>#@title # Activity 2 Question 22
#@markdown 22.	Compute the total gini impurity by hand. What value did you get?
activity2_answer22 = "" #@param {type:"string"}
#@title Run the Code 

#@markdown 23. We can also write some code to calculate this total gini impurity for us. Run the code below to see the total gini impurity for the predictor 'Stops < 3'.
Vehicles_Binary_Sample = pd.read_csv("vehicles_binary_sample.csv")
pos_value = 1
neg_value = 0
true_pos_value = 'Local'
true_neg_value = 'Visitor'
col_name="Stops"
label_name = "Vehicle"

impurityLeft, impurityRight, impurityAfter = calculateImpurity(Vehicles_Binary_Sample, col_name, label_name, pos_value, neg_value, true_pos_value, true_neg_value)
print ("impurity total for the predictor 'Stops < 3': {:.3f}".format(impurityAfter))
#@title Activity 2 Question 24
#@markdown 24. Does the output from the program match your computation of the total gini impurity for the predictor 'Stops < 3'?
activity2_answer24 = "" #@param {type:"string"}

#@title Run the Code 

#@markdown 25. Now we direct our attention to a different predictor. Let us evaluate the	predictor "Left turns < 2". Run the code below to see the table.
display(Vehicles_Logical_Sample[['Left turns < 2', 'Vehicle']])

#@title # Activity 2 Questions 26 - 29
#@markdown 26.  Based on the data above above, would we correctly predict the vehicle class if we have only information about vehicles "Left turns < 2 "? How many times would  the prediction be correct?   
activity2_answer26 = "" #@param {type:"string"}
#@markdown 27.	How many times would the prediction be  incorrect?
activity2_answer27 = "" #@param {type:"string"}
#@markdown 28. Compute the metrics for the predictor 'Left turns < 2'. 
activity2_28_TruePositive = "" #@param {type:"string"}
activity2_28_FalsePositive = "" #@param {type:"string"}
activity2_28_TrueNegative = "" #@param {type:"string"}
activity2_28_FalseNegative = "" #@param {type:"string"}
#@markdown 29. Compute the gini impurities for the predictor 'Left turns < 2'. 
activity2_29_giniPositive = "" #@param {type:"string"}
activity2_29_giniNegative = "" #@param {type:"string"}
activity2_29_giniTotal = "" #@param {type:"string"}
#@title Run the Code 

#@markdown 30. Run the code below to see the total gini impurity for the predictor 'Left turns < 2'
col_name = 'Left turns'
label_name = 'Vehicle'
pos_value = 1
neg_value = 0
true_pos_value = 'Local'
true_neg_value = 'Visitor'
col_name = 'Left turns'
impurityLeft, impurityRight, impurityAfter = calculateImpurity(Vehicles_Binary_Sample, col_name, label_name, pos_value, neg_value, true_pos_value, true_neg_value)
print ("impurity total for the predictor 'Left turns < 2': {:.3f}".format(impurityAfter))
#@title # Activity 2 Questions 31
#@markdown 31. Compare your metrics and gini impurities with the computed output above. Are they the same?
activity2_answer31 = "" #@param {type:"string"}
#@title Run the Code 

#@markdown 32.  Finally, we direct our attention to the last predictor. We will compute the gini impurity for  "Speed < 20" for you, but you may check it yourself if you wish. Run the code below.
display(Vehicles_Logical_Sample[['Speed < 20', 'Vehicle']])

col_name = 'Speed'
label_name = 'Vehicle'
pos_value = 1
neg_value = 0
true_pos_value = 'Local'
true_neg_value = 'Visitor'
col_name = "Speed"
print()
impurityLeft, impurityRight, impurityAfter = calculateImpurity(Vehicles_Binary_Sample, col_name, label_name, pos_value, neg_value, true_pos_value, true_neg_value)
print ("impurity total for the predictor 'Speed < 20': {:.3f}".format(impurityAfter))

#@title # Activity 2 Questions 33
#@markdown 33.	Now we have the total gini impurity for all predictors. What predictor is the best according to gini formula? Why?
activity2_answer35 = "" #@param {type:"string"}<jupyter_output><empty_output><jupyter_text>Activity 3. Building Decision Trees  Activity 3 Practical Problem:You, a city employee, would like to use a scientific method to construct a decision tree to use for classification of vehicles.<jupyter_code>#@title Activity 3 Questions 1 - 2
#@markdown 1. Two predictors have the lowest gini impurity. One of them should be chosen and therefore should be used as the root for the decision tree (since the gini impurity numbers are the same, the choice can be arbitrary). We have chosen "Stops < 3" as the decision box at the root. Once we have created the root node we need to decide what two nodes should be connected to it: a class node  or a decision node. Let us first consider the left branch of the root.  We know that all vehicles in the left subset are perfectly classified. In this case the left branch should be a class node. Why?
activity3_answer1 = "" #@param {type:"string"}
#@markdown 2. Let us next consider the right branch of the root.   We know that some vehicles will be misclassified if we terminate the tree with the class node. So we need to connect the root to a decision node. How do we choose the predictor? Would it help if we choose "Stops < 3" again"?
activity3_answer2 = "" #@param {type:"string"}
#@title Code for Activity 3 Question 3
#@markdown 3. Before we decide what predictor to choose let us identify the vehicles that need to be processed in the right subtree. For small samples, this can be done manually, but it will be difficult for larger datasets. To solve that problem, we write code to identify these subsets. Run the code below.
print("Right subtree for 'Stops < 3' (No branch)")
rightSample = selectRowsCondition(Vehicles_Logical_Sample, 'Stops < 3', '=', 'No')
display(rightSample)

#@title Code for Activity 3 Question 4

#@markdown 4. We now have to recursively apply our "building one node" procedure for the resulting subset. Run the code below to reevaluate usefulness of each remaining predictor. Let us start with "Left turns < 2".

col_name = 'Left turns < 2'
label_name = 'Vehicle'

pos_value = 'Yes'
neg_value = 'No'
true_pos_value = 'Local'
true_neg_value = 'Visitor'

impurityLeft, impurityRight, impurityAfter = calculateImpurity(rightSample, col_name, label_name, pos_value, neg_value, true_pos_value, true_neg_value)
print ("impurity total for the predictor 'Left turns < 2': {:.3f}".format(impurityAfter))
#@title # Activity 3 Question 5
#@markdown 5. Did you get the same numbers using hand computations?
activity3_answer5 = "" #@param {type:"string"}




#@title Run the Code

#@markdown 6. Run the code below to evaluate the usefulness of "Speed < 20".
col_name = 'Speed < 20'
label_name = 'Vehicle'


pos_value = "Yes"
neg_value = "No"
true_pos_value = 'Local'
true_neg_value = 'Visitor'

impurityLeft, impurityRight, impurityAfter = calculateImpurity(rightSample, col_name, label_name, pos_value, neg_value, true_pos_value, true_neg_value)
print ("impurity total for the predictor 'Speed < 20': {:.3f}".format(impurityAfter))<jupyter_output><empty_output><jupyter_text>Activity 3 Questions 7 and Fig 37. Compare the total impurity for both "Left turns" and "Speed < 20". Since "Left turns" predictor has the lower gini impurity, we attach the corresponding decision box as a right node to the root decision box as shown in the plot below. Since the gini impurities are 0, we can attach the final class nodes below the 'Left turns < 2' decision box. The resulting diagram is show in Fig. 3. It includes all information about splits between different branches of the tree. The "value" line in the leaf nodes indicates the metrics of the prediction. Fig. 3![decision tree fig 3](https://raw.githubusercontent.com/sbhattac/ai-workshop/master/DecisionTree/DTFig3.png)<jupyter_code>#@title # Activity 3 Question 5
#@markdown 8. Compare Fig. 1 which was produced by a human expert, and Fig. 3 which was produced by a decision tree algorithm and discuss the differences. 
activity3_answer8 = "" #@param {type:"string"}<jupyter_output><empty_output><jupyter_text>Activity 4. Tuning Decision Trees for large Data Sets  Activity 4 Practical Problem:You, a city employee, would like to use a scientific method to evaluate decision trees and tune them for the best classification of vehicles using the previously defined predictors.  Activity 4 Questions 1 - 21. Let us use two large data sets in files "vehicles_binary_train.csv" and "vehicles_binary_test.csv". We will use the file with the postfix "train" for the algorithm to "learn" how to construct the decision tree, and the file with the postfix "test" to evaluate the performance of the decision tree. The accuracy of the decision tree is the fraction of exactly matched preditions to all test samples. The following is the equation to calculate the accuracy:$$ accuracy  = \frac{T_p + T_n}{total} $$where:   $T_p $ is the number of true positives  $T_n $ is the number of true negatives  $total $ is the number of true positives + false positives + true negatives + false negatives  * As we discovered in Activities 2 and 3, selecting different predictors affects the decision tree. In the interactive code below, the first dropdown will allow you to choose different combinations of predictors to see how the decision tree changes.2. In order to generate the best decision tree, we need to provide a few parameters for the machine learning algorithm. These parameters are typically refered to as hyperparameters. There are many that can be tuned, but the one that we will focus on today is "depth".* Depth restricts the tree levels to the given depth number. The tree can be smaller (if the number of predictors is not sufficiently large) but will never be larger than the depth selected.The code below is interactive. You will be able to make different choices and see the output in real-time. Run the code below to see the accuracy.<jupyter_code>#@title Run the code to see the training data
Vehicles_Binary_Train = pd.read_csv('vehicles_binary_train.csv')
print("Table of data for the training set")
display(Vehicles_Binary_Train)

#@title Run the code to see the testing data
Vehicles_Binary_Test = pd.read_csv('vehicles_binary_test.csv')
print("Table of data for the testing set")
display(Vehicles_Binary_Test)
#@title Run the Code for Activity 4 Question 1

Inter_Vehicles_Binary = interactive(plot_tree_graphviz_selectColumns,
   train = fixed(Vehicles_Binary_Train),
   test = fixed(Vehicles_Binary_Test), 
   cols = [['Stops'], ['Left turns'], ['Speed'], ['Stops', 'Left turns'], ['Stops', 'Speed'],['Left turns','Speed'], ['Stops', 'Left turns', 'Speed']],
   split = fixed("best"),
   depth=[1,2,3],
   min_split=fixed(.1),
   min_leaf=fixed(0.1))

display(Inter_Vehicles_Binary)
#@title # Activity 4 Questions 4 - 14
#@markdown 2. How many rows do we have in the train file?
activity4_answer2 = "" #@param {type:"string"}
#@markdown 3. How many rows do we have in the test file?
activity4_answer3 = "" #@param {type:"string"}
#@markdown 4. Create the decision tree for the single feature 'Stops'? What is the accuracy score?
activity4_answer4 = "" #@param {type:"string"}
#@markdown 5. Create the decision tree for the single feature 'Left turns'? What is the accuracy score?
activity4_answer5 = "" #@param {type:"string"}
#@markdown 6. Create the decision tree for the single feature 'Speed'? What is the accuracy score?
activity4_answer6 = "" #@param {type:"string"}
#@markdown 7. Is a one level tree (root plus its children) sufficient to accurately predict every vehicle?
activity4_answer7 = "" #@param {type:"string"}
#@markdown 8. Create the tree with two features "Stops" and "Left turns". Record your accuracy.
activity4_answer8 = "" #@param {type:"string"}
#@markdown 9. Create the tree with two features "Stops" and "Speed". Record your accuracy.
activity4_answer9 = "" #@param {type:"string"}
#@markdown 10. Create the tree with two features "Left turns" and "Speed". Record your accuracy.
activity4_answer10 = "" #@param {type:"string"}
#@markdown 11. Create the tree with three features "Stops", "Left turns" and "Speed". Record your accuracy.
activity4_answer11 = "" #@param {type:"string"}
#@markdown 12. In the tree with three features "Stops", "Left turns" and "Speed", what hyper-parameters did you use?
activity4_answer12 = "" #@param {type:"string"}
#@markdown 13. Experiment with the interactive plot above to get a three level tree and try get the highest accuracy. Record your accuracy here.
activity4_answer13 = "" #@param {type:"string"}
#@markdown 14. Compare the accuracies of all the trees above. Do they have the same accuracy?  If not, which tree gave the best accuracy?
activity4_answer14 = "" #@param {type:"string"}<jupyter_output><empty_output><jupyter_text>Activity 5. Tuning Decision Trees for Large Data Sets with Numeric Features  Practical Problem Activity 5:Now you need to use a large data set with numerical data to generate new predictors and to tune the parameters of the decision tree to choose the best predictors.  Activity 5 Questions 1 - 21. let us switch to another large dataset which has been split into a training set and a testing set.  We will use the training file for the algorithm to "learn" how to construct the decision tree, and the testing file to evaluate the performance of the decision tree. This file differs from the above set in that it contains numerical values for the vehicle features. The number of 'Stops' ranges between 0 and 5. The number of 'Left turns' ranges between 0 and 4. The 'Speed' will range between 0 and 60. The machine learning algorith will construct new predictors based on optimum conditions for feature values.<jupyter_code>#@title Run the code to see the training data
Vehicles_Digital_Train = pd.read_csv('vehicles_digital_train.csv')
print("Table of data for the training set")
display(Vehicles_Digital_Train)
#@title Run the code to see the testing data
Vehicles_Digital_Test = pd.read_csv('vehicles_digital_test.csv')
print("Table of data for the testing set")
display(Vehicles_Digital_Test)<jupyter_output><empty_output><jupyter_text>The code below is interactive. You will be able to make different choices and see the output in real-time. Run the code below to see the accuracy.<jupyter_code>#@title Run the Code for Activity 5 Question 1

Inter_Vehicles_Digital = interactive(plot_tree_graphviz_selectColumns,
   train = fixed(Vehicles_Digital_Train),
   test = fixed(Vehicles_Digital_Test),
   cols = [['Stops'], ['Left turns'], ['Speed'], ['Stops', 'Left turns'], ['Stops', 'Speed'],['Left turns','Speed'], ['Stops', 'Left turns', 'Speed']], 
   #crit = ["gini", "entropy"],
   split = fixed("best"),
   depth=[1,2,3,4],
   min_split=fixed(0.1),
   min_leaf=fixed(0.1))

display(Inter_Vehicles_Digital)
#@title Activity 5 Questions part 2
#@markdown 2. What is the difference between the training files in this activity, and the one in activity 4?
activity5_answer2 = "" #@param {type:"string"}
#@markdown 3. In the tree diagram, the first line of each decision node displays the predictor for splitting. Create the decision tree for the single feature 'Stops', using a depth of 1. What is the accuracy score, and the predictor?
activity5_3_accuracy = "" #@param {type:"string"}
activity5_3_predictor = "" #@param {type:"string"}
#@markdown 4. Create the decision tree for the single feature 'Left turns', using a depth of 1.  What is the accuracy score, and the predictor?
activity5_4_accuracy = "" #@param {type:"string"}
activity5_4_predictor = "" #@param {type:"string"}
#@markdown 5. Create the decision tree for the single feature 'Speed', using a depth of 1.  What is the accuracy score, and the predictor?
activity5_5_accuracy = "" #@param {type:"string"}
activity5_5_predictor = "" #@param {type:"string"}
#@markdown 6. Experiment with the interactive plot above to get a three level tree and try get the highest accuracy. Record your accuracy here.
activity5_answer6 = "" #@param {type:"string"}
#@markdown 7. Experiment with the interactive plot above to get a four level tree and try get the highest accuracy. Record your accuracy here.
activity5_answer7 = "" #@param {type:"string"}
#@markdown 8. Compare the accuracies of all the trees above. Do they have the same accuracy?  If not, which tree gave the best accuracy?
activity5_answer8 = "" #@param {type:"string"}
#@markdown 9. In the optimal tree, we are using different predictors than the ones we were using in Activity 4. The activity 4 predictors were conditioned for you by a human expert. These new predictors are based on the features "Stops", "Left turns" and "Speed", but with  conditions automatically selected by the machine learning algorithm. Which predictor set is better?
activity5_answer9 = "" #@param {type:"string"}
#@title Team Work Evaluation
#@markdown 1.	How much time was required for completion of all activities?
evaluation1 = "" #@param {type:"string"}
#@markdown 2.	Was the contribution from each participant equal?	
evaluation2 = "" #@param {type:"string"}
#@markdown 3.	How could the team work and learn more effectively?
evaluation3 = "" #@param {type:"string"} 
#@markdown 4.	How many participants thought the problems were too simple (trivial)? 
evaluation4 = "" #@param {type:"string"}
#@markdown 5.	How many participants thought the problems were at the proper level of difficulty?	
evaluation5 = "" #@param {type:"string"}
#@markdown 6.	How many participants thought the problems were too hard?
evaluation6 = "" #@param {type:"string"}
#@markdown 7.	Was help needed? Where?
evaluation7 = "" #@param {type:"string"}
#@markdown 8.	Does the team have any suggestions about how this notebook could be improved? If so, how?
evaluation8 = "" #@param {type:"string"}<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0934287856434566, 'fraction_numerical': 0.016494546158125137, 'mean_word_length': 4.734812760055479, 'pattern_counts': {'":': 0, '<': 90, '<?xml version=': 0, '>': 63, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '579529', 'license': 'Apache-2.0', 'n_tokens_mistral': 12503, 'n_tokens_neox': 11639, 'n_words': 4956, 'path': 'DecisionTree/AI_Workshop_Decision_Tree.ipynb', 'repo_name': 'sbhattac/ai-workshop', 'size': '41347'}