<jupyter_start><jupyter_text>Assignment 3  Course: Programming for Analytics (MSCA 31012) Date: 10/29/2021 Author: <NAME> Dataset: https://en.wikipedia.org/wiki/Iris_flower_data_set Additional Resources: https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html Your task is to optimize this model so it produces better predictions. A common technique is to do 'hyper parameter optimization.' Scikit-learn has methods built in to do this (see `GridSearchCV` and `RandomSearchCV`). However, in order to learn about loops, we will do this ourselves.Try different values for the following parameters:- criterion: 'entropy' and 'gini'- max_depth: betwen 1 and 10- min_samples_split: between 2 and 5You are welcome to try optimizations beyond these values, but the list above is the minimunm required for this assignment.Find the best set of parameters and have your program return them, along with the score.If you reach accuracy of 1 (perfect prediction), stop searching and show the parameters.You are expected to write a program to solve this optimization problem. You may not cust and paste the model training code for each test (that's A LOT of cutting and pasting). You may also not manually run the code many times where you manually try different values.  HintsYou will need to train this model and score it multiple times (once for each set of parameters you are testing). Think in terms of loops (perhaps multiple nested loops).You will also need to keep track of all three parameters and the score. Make sure you don't accidentally overwrite variables inside the loops.Somehow, you will need to compare the current score with the best score you have achieved so far. If the current score is better than what you have already achieved, you should update the score, and the associated variables.At the end of your program, print the best score and the associated variables.<jupyter_code>from sklearn import datasets
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier<jupyter_output><empty_output><jupyter_text>Load the `iris` data setThis is a  very popular data set. Due to its simplicity, it is used in many text books.This link provides a good description: https://en.wikipedia.org/wiki/Iris_flower_data_set<jupyter_code>data_df = datasets.load_iris()
#print(data_df.DESCR) #uncomment this to get a quick description
data_df.feature_names
data_df.target_names<jupyter_output><empty_output><jupyter_text>Extract input (X) and output(y) variables<jupyter_code>X = data_df.data
y = data_df.target
#X
#y<jupyter_output><empty_output><jupyter_text>Take a look at the first ten rows of X<jupyter_code>X[:10,]<jupyter_output><empty_output><jupyter_text>Take a look at the first ten rows of y<jupyter_code>y[:10,]<jupyter_output><empty_output><jupyter_text>Total number of observations<jupyter_code>len(X)<jupyter_output><empty_output><jupyter_text>Separate the data into the part we will use for training and the part we will use for testingThe testing data is hidden during training time. Otherwise the model could just _memorize_ all the data and look up the answer when you ask for a prediction. This model will be perfect, except when you give it data it hasn't seen before.<jupyter_code>X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.10)
len(X_train), len(X_test), len(y_train), len(y_test)<jupyter_output><empty_output><jupyter_text>Build a model to predict the flower from the measurements of its petals Entropy index measures groups of classes that make up about 50% of the data.<jupyter_code>%%time

model = DecisionTreeClassifier(criterion='entropy', max_depth=1, min_samples_split=2)
model.fit(X_train, y_train)<jupyter_output>CPU times: user 1.32 ms, sys: 1.62 ms, total: 2.94 ms
Wall time: 5.27 ms<jupyter_text>Test the accuracy of your model (the closer it is to 1, the better it is)<jupyter_code>model.score(X_test, y_test)<jupyter_output><empty_output><jupyter_text>MODEL 1 - Gini Gini index measures the degree or probability of a particular variable being wrongly classified when it is randomly chosen.<jupyter_code>%%time

model = DecisionTreeClassifier(criterion='gini', max_depth=1, min_samples_split=2)
model.fit(X_train, y_train)
model.score(X_test, y_test)<jupyter_output><empty_output><jupyter_text>Observations: Both gini and entorpy  measure the quality of a split. Gini is intended for continuous attributes, and Entropy for attributes that occur in classes which is why gini is better suited for this model  MODEL 2 - Max Depth Maximum depth refers to the the length of the longest path from a root to a leaf.  Max Depth = 0.1<jupyter_code>%%time

model = DecisionTreeClassifier(criterion='gini', max_depth=0.1, min_samples_split=2)
model.fit(X_train, y_train)
model.score(X_test, y_test)<jupyter_output><empty_output><jupyter_text>Max Depth = 10<jupyter_code>%%time

model = DecisionTreeClassifier(criterion='gini', max_depth=10, min_samples_split=2)
model.fit(X_train, y_train)
model.score(X_test, y_test)<jupyter_output><empty_output><jupyter_text>Observations: Increase in max depth leads to higher model score  MODEL 3 - Min Samples Split min_samples_split refers to the the minimum number of samples required to split an internal node  Min_samples_split = 2<jupyter_code>%%time

model = DecisionTreeClassifier(criterion='gini', max_depth=10, min_samples_split=2)
model.fit(X_train, y_train)
model.score(X_test, y_test)
%%time

model = DecisionTreeClassifier(criterion='gini', max_depth=10, min_samples_split=5)
model.fit(X_train, y_train)
model.score(X_test, y_test)<jupyter_output><empty_output><jupyter_text>Observations: This model requires a greater of samples to split the internal node.  Note: Line [90] shows score of optimized model givenbasic parameters  MODEL 4 - Optimized Model Species Prediction<jupyter_code>import numpy as np
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
import sklearn.datasets
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error
%matplotlib inline
# Reload Data
from sklearn.datasets import load_iris
data_df = datasets.load_iris()

# Rename DF
petal_df = pd.DataFrame(data= data_df.data, columns= data_df.feature_names)
petal_df
# Descriptive statistics of dataset
petal_df.describe()
# DF Summary
petal_df.info()
#Ignore warnings
import warnings
warnings.warn("ignore")

# Summary Plots
sns.pairplot(petal_df, hue= 'petal length (cm)')
# Assigning str value to species in DF
main_df = pd.DataFrame(data= data_df.target, columns= ['species'])
def converter(species):
    if species == 0:
        return 'setosa'
    elif species == 1:
        return 'versicolor'
    else:
        return 'virginica'
main_df['species'] = main_df['species'].apply(converter)

# Combine DF using concatenate function
petal_df = pd.concat([petal_df, main_df], axis= 1)
petal_df
petal_df.info()
# Species (object - str) to float (number)
petal_df.drop('species', axis= 1, inplace= True)
main_df = pd.DataFrame(data= data_df.target, columns= ['species'])
petal_df = pd.concat([petal_df, main_df], axis= 1)
# New train/test variables
X= petal_df.drop(labels= ['petal length (cm)'], axis= 1)
y= petal_df['petal length (cm)']
# Splitting the Dataset 
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size= 0.10)

# Dataset Results
print(f"80 % of Train Data Size : {len(X_train)} and Data Size of Training Labels : {len(y_train)}")
print()
print(f"20 % of Test Data Size : {len(X_test)} and Data Size of Testing Labels : {len(y_test)}")
# Linear Regression
lr = LinearRegression()

# Train the model
lr.fit(X_train, y_train)

# Testing model predictions
model_predict = lr.predict(X_test)
# Model performance
print('Mean Squared Error =', mean_squared_error(y_test, model_predict))
print ('Model Score =', lr.score(X_test, y_test))<jupyter_output>Mean Squared Error = 0.09937855454706045
Model Score = 0.9678843864571288<jupyter_text>Check Work<jupyter_code>petal_df.loc[0]
input_values = {'sepal length (cm)' : [5.1],
    'sepal width (cm)' : [3.5],
    'petal length (cm)' : [1.4],
    'petal width (cm)' : [0.2],
    'species' : 0}
test_df = pd.DataFrame(data= input_values)
test_df 
model_predict = lr.predict(X_test)
print('Predicted Petal Length (cm):', model_predict[0])
print('Actual Petal Length (cm):', 1.4)<jupyter_output>Predicted Petal Length (cm): 1.4992462987815423
Actual Petal Length (cm): 1.4
{'fraction_non_alphanumeric': 0.08864954432477216, 'fraction_numerical': 0.017635223103325836, 'mean_word_length': 6.065217391304348, 'pattern_counts': {'":': 0, '<': 59, '<?xml version=': 0, '>': 59, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '179611', 'license': 'MIT', 'n_tokens_mistral': 2831, 'n_tokens_neox': 2668, 'n_words': 929, 'path': 'File/3Assignment.ipynb', 'repo_name': 'jpezan/MSCA37010_Programming_for_Analytics_Assignment_3', 'size': '8459'}