<jupyter_start><jupyter_text>Tree Learning – implementation and application of decision trees  Introduction This notebook gives you the opportunity to implement some key components of decision tree learning and run your algorithm on a benchmark dataset. So restrictions will be made to simplify the problem. The notebook concludes by asking you to run the decision tree learning (and tree-based method of "Random Forests") from scikit-learn for comparison.Make sure you have the Titanic dataset ("```titanic.csv```") in the directory from where you are running the notebook before you start.<jupyter_code>import numpy as np
import pandas as pd
from matplotlib import pyplot as plt
%matplotlib inline
import seaborn as sns
ds = pd.read_csv('titanic.csv')
ds.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 891 entries, 0 to 890
Data columns (total 12 columns):
PassengerId    891 non-null int64
Survived       891 non-null int64
Pclass         891 non-null int64
Name           891 non-null object
Sex            891 non-null object
Age            714 non-null float64
SibSp          891 non-null int64
Parch          891 non-null int64
Ticket         891 non-null object
Fare           891 non-null float64
Cabin          204 non-null object
Embarked       889 non-null object
dtypes: float64(2), int64(5), object(5)
memory usage: 83.6+ KB<jupyter_text>Data Preprocessing  To simplify things we will focus on the supplied dataset and start by doing some preprocessing, including feature selection, turning categorical data to numeric, and some other stuff. Spend about 10 minutes and go through this if you have any doubts. We start by inspecting the dataset.<jupyter_code>ds.head()<jupyter_output><empty_output><jupyter_text>Do we need all the features ? No.<jupyter_code>cols_to_drop = [
    'PassengerId',
    'Name',
    'Ticket',
    'Cabin',
    'Embarked',
]

df = ds.drop(cols_to_drop, axis=1)
df.head()<jupyter_output><empty_output><jupyter_text>Another simplification will be to treat all attributes as numeric. So we need to convert any that are not.<jupyter_code>def convert_sex_to_num(s):
    if s=='male':
        return 0
    elif s=='female':
        return 1
    else:
        return s

df.Sex = df.Sex.map(convert_sex_to_num)
df.head()<jupyter_output><empty_output><jupyter_text>Let's overview the preprocessed dataset now with some standard commands.<jupyter_code>data = df.dropna()
data.describe()
plt.figure()
sns.heatmap(data.corr())
input_cols = ['Pclass', 'Sex', 'Age', 'SibSp', 'Parch', 'Fare']
out_cols = ['Survived']

X = data[input_cols]
y = data[out_cols]

#X.head()
print (X.shape, y.shape)
data = data.reset_index(drop=True)<jupyter_output><empty_output><jupyter_text>Functions for your Decision Tree learning algorithm Now is your chance to go ahead and implement some of the functionality needed for the decision tree learner. Remember that the _class_ variable for which we need to learn a tree is ```Survived```.<jupyter_code>def divide_data(x_data, fkey, fval):
    x_right = pd.DataFrame([], columns=x_data.columns)
    x_left = pd.DataFrame([], columns=x_data.columns)
    
    
    for ix in range(x_data.shape[0]):
        # Retrieve the current value for the fkey column lets call it val
        try:
            #TODO
            
        except:
            #TODO
            
        
        # Check where the row needs to go
        if val > fval:
            # pass the row to right
            #TODO
        else:
            # pass the row to left
            #TODO
    
    # return the divided datasets
    #TODO

def entropy(col):
    #TODO

def information_gain(xdata, fkey, fval):
    #TODO
#Here X is your data without the Survived column. Run it after you have filled in the missing code above. 
'''
for fx in X.columns:
    print (fx) 
    print (information_gain(data, fx, data[fx].mean()))
'''
class DecisionTree:
    def __init__(self, depth=0, max_depth=5):
        self.left = None
        self.right = None
        self.fkey = None
        self.fval = None
        self.max_depth = max_depth
        self.depth = depth
        self.target = None
    
    def train(self, X_train):
        
        print (self.depth, '-'*10)
        # Get the best possible feature and division value (gains)
        #TODO
        
        # store the best feature (using min information gain)
        #TODO
        
        # divide the dataset and reset index 
        #TODO
        
        # Check the shapes and depth if it has exceeded max_depth or not in case it has make predictions 
        # TODO
        
        # branch to right
        #TODO
        
        # branch to left
        #TODO
        
        #Make your prediction 
        #TODO
        
        return
    
    def predict(self, test):
        if test[self.fkey] > self.fval:
            pass
            # go right
            #TODO
        else:
            pass
            # go left
            #TODO<jupyter_output><empty_output><jupyter_text>Divide your data: separate Training and Test sets  Train your own decision tree<jupyter_code>'''
dt = DecisionTree()
dt.train(training_data)
'''
'''
print (dt.fkey, dt.fval)
print (dt.right.fkey, dt.right.fval)
print (dt.left.fkey, dt.left.fval)

print (dt.right.right.fkey, dt.right.right.fval)
print (dt.right.left.fkey, dt.right.left.fval)


print (dt.left.right.fkey, dt.left.right.fval)
print (dt.left.left.fkey, dt.left.left.fval)
'''<jupyter_output><empty_output><jupyter_text>Make predictions for the first 10 and see if they are correct.<jupyter_code>for ix in testing_data.index[:10]:
    print (dt.predict(testing_data.loc[ix]))
testing_data.head(10)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08841624187027597, 'fraction_numerical': 0.014941114431358763, 'mean_word_length': 2.811118553248493, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 35, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '553991', 'license': 'MIT', 'n_tokens_mistral': 1837, 'n_tokens_neox': 1740, 'n_words': 590, 'path': 'Tutorial/week05/Lab5_TreeLearning_Qs.ipynb', 'repo_name': 'US579/COMP9417-19T2', 'size': '5689'}