<jupyter_start><jupyter_text>Iris ClassificationThis project was started as a motivation for learning Machine Learning Algorithms and to learn the different data preprocessing techniques such as Exploratory Data Analysis, Feature Engineering, Feature Selection, Feature Scaling and finally to build a machine learning model.In this project we will classify the Iris species based on their features such as Sepal length, Sepal width, Petal length and Petal width.The dataset is collected from [Kaggle](https://www.kaggle.com/uciml/iris). This dataset consists of 3 categories of species which is setosa, versicolor and virginica.<jupyter_code># TODO : Import necessary python libraries needed for this project

# For Data preparation and manipulation
import numpy as np
import pandas as pd
# For reading database
import sqlite3
# For visualization
import matplotlib.pyplot as plt
import seaborn as sb

# Set Seaborn styles
sb.set()
# TODO : Load Iris Dataset

# Load Iris csv dataset
iris_csv = pd.read_csv('../data/Iris.csv')

# Load Iris sqlite data
data = sqlite3.connect('../data/database.sqlite')
query = data.execute('SELECT * FROM Iris')
columns = [col[0] for col in query.description]
iris_db = pd.DataFrame.from_records(data = query.fetchall(), columns = columns)
# TODO : First five rows of iris csv dataset
iris_csv.head()
# TODO : First five rows iris sqlite database
iris_db.head()<jupyter_output><empty_output><jupyter_text>Since both datas are similar, we can choose any one of them. I am going to choose `iris_db`.<jupyter_code># TODO : Splitting Features and Species as separate dataframes 
features = iris_db.drop(columns = ['Id', 'Species'])
Species = iris_db.Species
print("Iris dataset has {} samples and {} features.".format(*features.shape))<jupyter_output>Iris dataset has 150 samples and 4 features.<jupyter_text>Explorative Data Analysis StatisticsLet us know some descriptive statistics on iris dataset<jupyter_code>features.describe()<jupyter_output><empty_output><jupyter_text>Let group them by species and do Descriptive statistics<jupyter_code>iris_db.drop('Id', axis = 1, inplace = True)
iris_db.groupby('Species').describe().T<jupyter_output><empty_output><jupyter_text>- **count** shows that there 50 samples for each species.- **SETOSA**    - Average sepal length is 5cm    - Average sepal width is 3cm    - Average petal length is 1.5cm    - Average petal width is 0.25cm- **VERSICOLOR**    - Average sepal length is 6cm    - Average sepal width is 2.8cm    - Average petal length is 4.26cm    - Average petal width is 1.32cm- **VIRGINICA**    - Average sepal length is 6.6cm    - Average sepal width is 3cm    - Average petal length is 6cm    - Average petal width is 2cm    From the above information, - Based on **Petal length** we can easily classify them as Setosa(1.5cm), Versicolor(4.2cm) and Virginica(6cm).- Based on **Petal width** we can easily classify Setosa(0.25cm) from Versicolor(1.32cm) and Virginica(2cm).- **Sepal width** looks similar for all three species - Setosa(3cm), Versicolor(2.8cm) and Virginica(3cm).- Based on **Sepal length**, there are only small changes on three species (5cm, 6cm and 6.6cm)Since Sepal width looks similar for all the species, we can drop that feature.  Feature ObservationLet's plot the data<jupyter_code># TODO : Correlation between the features
features_correlation = iris_db.drop('Species', axis = 1).corr()
features_correlation
plt.figure(figsize = (12, 6))
sb.heatmap(data = features_correlation)
plt.show()
plt.figure(figsize = (6, 6))
sb.pairplot(data = iris_db, hue = 'Species')
sb.relplot(x = 'PetalLengthCm', y = 'PetalWidthCm', data = iris_db, hue = 'Species', aspect = 2, height = 6)<jupyter_output><empty_output><jupyter_text>- From the above scatter plot, setosa species clearly classified from other two species.- Other two species can also be classified based on **Petal Length** and **Petal Width** easily.<jupyter_code>sb.relplot(x = 'PetalLengthCm', y = 'SepalLengthCm', data = iris_db, hue = 'Species', aspect = 2, height = 6)<jupyter_output><empty_output><jupyter_text>- From the above plot, we can separate versicolor and virginica by means of **Petal Length** alone.- Sepal Length does not contribute more here<jupyter_code>sb.relplot(x = 'PetalWidthCm', y = 'SepalLengthCm', data = iris_db, hue = 'Species', aspect = 2, height = 6)<jupyter_output><empty_output><jupyter_text>- From the above plot, we can separate based on **Petal Width** alone.- Sepal Length doesn't contribute more here.  Feature Selection - From the above discussion, we can easily classify the iris species based on petal features than sepal features.<jupyter_code>iris_features = iris_db.loc[:, ['PetalLengthCm', 'PetalWidthCm']]
iris_species = iris_db.loc[:, 'Species']
iris_features
iris_species<jupyter_output><empty_output><jupyter_text>Building a Machine Learning ModelBased on above explorations on the data, we came to know that we can classify the iris species based on some conditions on Petal Length and PetalWidth.Let's build a Decision Tree Classier model to the data.<jupyter_code>features_values = iris_features.values
species_values = iris_species.values
# TODO : Split the dataset as Train set and Test set.
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(features_values, species_values, 
                                                    test_size = 0.25, random_state = 42)
# TODO : Print the shape of X_train, X_test, y_train, y_test
print("Shape of X_train : ", X_train.shape)
print("Shape of X_test : ", X_test.shape)
print("Shape of y_train : ", y_train.shape)
print("Shape of y_test : ", y_test.shape)
# TODO : Build Decision Tree Classifier Model

from sklearn.tree import DecisionTreeClassifier

decision_tree_classifier = DecisionTreeClassifier(criterion = 'gini')

decision_tree_classifier.fit(X_train, y_train)<jupyter_output><empty_output><jupyter_text>Decision Tree classifying Iris species<jupyter_code>from sklearn import tree
def plot_decision_tree(decision_tree_classifier):
    plt.figure(figsize=(16, 14))
    tree.plot_tree(decision_tree_classifier, fontsize = 15, feature_names = iris_features.columns, 
                   class_names = ['Setosa', 'Versicolor', 'Virginica'], filled = True)
plot_decision_tree(decision_tree_classifier)<jupyter_output><empty_output><jupyter_text>Model Performance  Checking model performance by changing max_depth parameter and the criterion as gini<jupyter_code>max_depths = list(range(1, 10))
training_scores = []
test_scores = []
for max_depth in max_depths:
    decision_tree_classifier = DecisionTreeClassifier(criterion = 'gini', max_depth = max_depth)
    decision_tree_classifier.fit(X_train, y_train)
    training_scores.append(decision_tree_classifier.score(X_train, y_train)*100)
    test_scores.append(decision_tree_classifier.score(X_test, y_test)*100)
    
plt.figure(figsize = (14, 6))
plt.plot(max_depths, training_scores, 'ro-', linewidth = 4, markersize = 15)
plt.plot(max_depths, test_scores, 'go-', linewidth = 4, markersize = 15)
plt.xlabel('MAX DEPTHS', fontsize = 18)
plt.ylabel('SCORES in precent', fontsize = 18)
plt.legend(labels = ['Train scores', 'Test scores'], fontsize=18)
plt.show()

print ("Train Scores : ", training_scores)
print ("Test  Scores : ", test_scores)<jupyter_output><empty_output><jupyter_text>Checking model performance by changing max_depth parameter and the criterion as entropy<jupyter_code>max_depths = list(range(1, 10))
training_scores = []
test_scores = []
for max_depth in max_depths:
    decision_tree_classifier = DecisionTreeClassifier(criterion = 'entropy', max_depth = max_depth)
    decision_tree_classifier.fit(X_train, y_train)
    training_scores.append(decision_tree_classifier.score(X_train, y_train) * 100)
    test_scores.append(decision_tree_classifier.score(X_test, y_test) * 100)
    
plt.figure(figsize = (14, 6))
plt.plot(max_depths, training_scores, 'ro-', linewidth = 4, markersize = 15)
plt.plot(max_depths, test_scores, 'go-', linewidth = 4, markersize = 15)
plt.xlabel('MAX DEPTHS', fontsize = 18)
plt.ylabel('SCORES in percent', fontsize = 18)
plt.legend(labels = ['Train scores', 'Test scores'], fontsize=18)
plt.show()

print ("Train Scores : ", training_scores)
print ("Test  Scores : ", test_scores)<jupyter_output><empty_output><jupyter_text>We can choose max_depth = 6  Let's build the optimal model<jupyter_code>classifier = DecisionTreeClassifier(max_depth = 6, criterion = 'gini')
classifier.fit(X_train, y_train)
plot_decision_tree(classifier)
from sklearn.metrics import confusion_matrix
print(" Confusion Matrix : \n", confusion_matrix(y_test, classifier.predict(X_test)))
print("Training Accuracy : {0:.2f} %".format(classifier.score(X_train, y_train) * 100))
print("Testing Accuracy : {} %".format(classifier.score(X_test, y_test) * 100))<jupyter_output>Training Accuracy : 99.11 %
Testing Accuracy : 100.0 %<jupyter_text>Let's predict the test data predictions.<jupyter_code>for i in range(X_test.shape[0]):
    print("Petal Length = {}\tPetal Width = {}\tSpecies is {}".format(*X_test[i], classifier.predict([X_test[i]])))<jupyter_output>Petal Length = 4.7	Petal Width = 1.2	Species is ['Iris-versicolor']
Petal Length = 1.7	Petal Width = 0.3	Species is ['Iris-setosa']
Petal Length = 6.9	Petal Width = 2.3	Species is ['Iris-virginica']
Petal Length = 4.5	Petal Width = 1.5	Species is ['Iris-versicolor']
Petal Length = 4.8	Petal Width = 1.4	Species is ['Iris-versicolor']
Petal Length = 1.5	Petal Width = 0.4	Species is ['Iris-setosa']
Petal Length = 3.6	Petal Width = 1.3	Species is ['Iris-versicolor']
Petal Length = 5.1	Petal Width = 2.3	Species is ['Iris-virginica']
Petal Length = 4.5	Petal Width = 1.5	Species is ['Iris-versicolor']
Petal Length = 3.9	Petal Width = 1.2	Species is ['Iris-versicolor']
Petal Length = 5.1	Petal Width = 2.0	Species is ['Iris-virginica']
Petal Length = 1.4	Petal Width = 0.1	Species is ['Iris-setosa']
Petal Length = 1.3	Petal Width = 0.2	Species is ['Iris-setosa']
Petal Length = 1.5	Petal Width = 0.1	Species is ['Iris-setosa']
Petal Length = 1.5	Petal Width = 0.3	Species is ['Iris-setosa']
Petal L[...]
{'fraction_non_alphanumeric': 0.10717100078802207, 'fraction_numerical': 0.019897557131599686, 'mean_word_length': 5.369510664993727, 'pattern_counts': {'":': 0, '<': 54, '<?xml version=': 0, '>': 54, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '312361', 'license': 'Apache-2.0', 'n_tokens_mistral': 3588, 'n_tokens_neox': 3279, 'n_words': 1012, 'path': '01.Iris_Species_Classification/Notebooks/.ipynb_checkpoints/Iris-checkpoint.ipynb', 'repo_name': 'ASHOKKUMAR-K/Machine-Learning-Projects', 'size': '10152'}