<jupyter_start><jupyter_text>Student Detials> Title= "Mr"\> Name= "<NAME>"\> email = "<EMAIL>"\> whatsapp = "03358043653"  In this Notebook we are going to know about Decision Tree Classifier using Sci-kit algorithm  Decision Tree AlgorithmA decision tree is a flowchart-like tree structure where an internal node represents feature(or attribute), the branch represents a decision rule, and each leaf node represents the outcome. The topmost node in a decision tree is known as the root node. It learns to partition on the basis of the attribute value. It partitions the tree in recursively manner call recursive partitioning. This flowchart-like structure helps you in decision making. It's visualization like a flowchart diagram which easily mimics the human level thinking. That is why decision trees are easy to understand and interpret. ![Diagram Decsion Tree](https://res.cloudinary.com/dyd911kmh/image/upload/f_auto,q_auto:best/v1545934190/1_r5ikdb.png)  How does the Decision Tree algorithm work?The basic idea behind any decision tree algorithm is as follows:Select the best attribute using Attribute Selection Measures(ASM) to split the records.Make that attribute a decision node and breaks the dataset into smaller subsets.Starts tree building by repeating this process recursively for each child until one of the condition will match:All the tuples belong to the same attribute value.There are no more remaining attributes.There are no more instances. ![How Decsion Trree works](https://res.cloudinary.com/dyd911kmh/image/upload/f_auto,q_auto:best/v1545934190/2_btay8n.png)  Entropy**Definition:** Entropy provides an absolute limit on the shortest possible average length of a lossless compression encoding of the data produced by a source, and if the entropy of the source is less than the channel capacity of the communication channel,the data generated by the source can be reliably communicated to the receiver.The definition is extremely difficult to understand, and it is not necessarily pertinent to our discussions of decision trees. Shannon(1948) used the concept of entropy for the theory of communication, to determine how to send encoded (bits) information from a sender to a receiver without loss of information and with the minimum amount of bits. > High Entropy : More uncertainty\> Low Entropy : More Predictability  Information GainNow that we have discussed Entropy we can move forward into information gain. This is the concept of a decrease in entropy after splitting the data on a feature. The greater the information gain, the greater the decrease in entropy or uncertainty.![image](https://miro.medium.com/max/470/1*ixgJCZRgq9v0AOKJro-3Iw.png)**T:** Target population prior to the split T=∑ {All Splits}, the total number of observation before splitting.**Entropy(T):** Measure the disorder before the split,or level of uncertainty**s{i}:** is the number of observations on the i{th} split**Entropy(s{i}):** Measures the disorder for the target variable on split s{i}  For More Information visit ([More Details](https://www.datacamp.com/community/tutorials/decision-tree-classification-python))  If and Else Statement in Decsion Tree Classifier<jupyter_code># import all the lib
import pandas as pd
import seaborn as sns
import numpy as np
import sklearn.metrics as sm
import matplotlib.pyplot as plt

# Importing Linear Regression model from scikit learn
from sklearn.linear_model import LinearRegression
# Importing metrics for the evaluation of the model
from sklearn.metrics import r2_score,mean_squared_error
from sklearn.tree import DecisionTreeClassifier # Import Decision Tree Classifier
from sklearn.model_selection import train_test_split # Import train_test_split function
from sklearn import metrics #Import scikit-learn metrics module for accuracy calculation

# read the dataset using pandas
df = pd.read_csv('D:/Python ka Chilla/python_chilla/data/mldata.csv')
# df = sns.load_dataset('iris')

df.head()
def entropy(col,df,option=2):
# Takes the column number and data frame as an input
    #Grabs the column we specify and the last column (which we assume is the decision col)
    new_df=df.iloc[:,[col,len(df.columns)-1]]
    #rename the columns
    new_df.columns=('col1','col2')
    #return the unique values in this feature along with their counts
    names,count=np.unique(new_df.col1,return_counts=True)
    
    
    #create an empty list
    entropy_list=list()
    
    #for loop on each split to get the entry at the split
    for i in range(0,(len(names))):
            if(option==2):
                dff=new_df[new_df.col1==names[i]]
                entropy_list.append(count[i])
                entropy_list.append(names[i])
            else:
                dff=new_df
            den=len(dff)
            columns=new_df.col2.unique()
            p1 = dff.col2.eq(columns[0]).sum()/den
            p2 = dff.col2.eq(columns[1]).sum()/den
            P=[p1,p2]
            ent=0
            k=0
            for p in P:
                k=k+1
                ent += -p * math.log(p,2)
                if(k==len(P)):
                    entropy_list.append(ent)
    return entropy_list
import numpy as np
import math

print("Entropy for Emotion Split",entropy(0,df)[1:3])
print("Confirm",-(3/5)*math.log((3/5),2)-(2/5)*math.log((2/5),2),"\n")
print("Entropy for Emotion Split",entropy(0,df)[4:6])
print("Confirm",-(1/3)*math.log((1/3),2)-(2/3)*math.log((2/3),2))<jupyter_output><empty_output><jupyter_text>Feature Selection<jupyter_code>#split dataset in features and target variable
df['gender'] = df['gender'].replace('Male',  1) 
df['gender'] = df['gender'].replace('Female',  0) 
feature_cols = ['age', 'gender', 'weight']
X = df[feature_cols] # Features
y = df.likeness # Target variable
df['likeness'].value_counts()<jupyter_output><empty_output><jupyter_text>Splitting DataTo understand model performance, dividing the dataset into a training set and a test set is a good strategy.<jupyter_code># Split dataset into training set and test set
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=1) # 70% training and 30% test<jupyter_output><empty_output><jupyter_text>Building Decision Tree ModelLet's create a Decision Tree Model using Scikit-learn.<jupyter_code># Create Decision Tree classifer object
clf = DecisionTreeClassifier()

# Train Decision Tree Classifer
clf = clf.fit(X_train,y_train)

#Predict the response for test dataset
y_pred = clf.predict(X_test)
# Model Accuracy, how often is the classifier correct?
print("Accuracy:",metrics.accuracy_score(y_test, y_pred))
# Create Decision Tree classifer object
clf = DecisionTreeClassifier(criterion="entropy", max_depth=3)

# Train Decision Tree Classifer
clf = clf.fit(X_train,y_train)

#Predict the response for test dataset
y_pred = clf.predict(X_test)

# Model Accuracy, how often is the classifier correct?
print("Accuracy:",metrics.accuracy_score(y_test, y_pred))<jupyter_output>Accuracy: 0.6081081081081081<jupyter_text>Assignments for Decsion Tree  1. Decision Tree Model SavingLet's Save a Decision Tree Model<jupyter_code>import pickle
import joblib
filename = 'finalized_model_descsion_tree.joblib'
joblib.dump(clf, filename)
# using pickle
filename = 'finalized_model.sav'
pickle.dump(clf, open(filename, 'wb'))<jupyter_output><empty_output><jupyter_text>2. Decision Tree Model LoadingLet's load the saved a Decision Tree Model using pickle.<jupyter_code># using job lib
# load the model from disk
loaded_model = joblib.load(filename)
result = loaded_model.score(X_test, y_test)
print(result)
# using pickle
 
# load the model from disk
loaded_model = pickle.load(open(filename, 'rb'))
result = loaded_model.score(X_test, y_test)
print(result)<jupyter_output>0.6081081081081081<jupyter_text>3. Visalize the Tree of Our Model<jupyter_code>from sklearn import tree
# tree.plot_tree(clf);
fn=['age', 'gender', 'weight']
cn=['age', 'gender', 'weight']
fig, axes = plt.subplots(nrows = 1,ncols = 1,figsize = (4,4), dpi=300)
tree.plot_tree(clf,
               feature_names = fn, 
               class_names=cn,
               filled = True)
fig.savefig('descion_tree.png')
tree.export_graphviz(clf, out_file='tree.dot')<jupyter_output><empty_output><jupyter_text>4. Visalize the Tree and Show Image in HD<jupyter_code>fn=['age', 'gender', 'weight']
cn=['age', 'gender', 'weight']
fig, axes = plt.subplots(nrows = 1,ncols = 1,figsize = (4,4), dpi=500)
tree.plot_tree(clf,
               feature_names = fn, 
               class_names=cn,
               filled = True)
fig.savefig('descion_tree.png')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09546511627906977, 'fraction_numerical': 0.018837209302325582, 'mean_word_length': 4.636304062909567, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 39, 'https://': 4, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '485917', 'license': 'Apache-2.0', 'n_tokens_mistral': 2708, 'n_tokens_neox': 2563, 'n_words': 878, 'path': 'machine_learning/11_decision_tree_classifier_image_classification.ipynb', 'repo_name': 'Aliktk/Python_Chilla', 'size': '8616'}