<jupyter_start><jupyter_text>Sparks Foundation Task 1: Prediction using Unsupervised MLTask: From the given ‘Iris’ dataset, predict the optimum number of clusters and represent it visually.Let's start by importing the necessary libraries.<jupyter_code>#Data Processing
import pandas as pd
import numpy as np

#Data Visualization
import matplotlib.pyplot as plt
import seaborn as sns

#Machine Learning
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score

%matplotlib inline<jupyter_output><empty_output><jupyter_text>First let's read the file.<jupyter_code>#Reading the file
iris = pd.read_csv('Iris.csv')<jupyter_output><empty_output><jupyter_text>Let's then see what our data looks like.<jupyter_code>iris.head()
iris.describe()
iris.Species.unique()
iris.isnull().values.any()<jupyter_output><empty_output><jupyter_text>Looks like the dataset is complete, no null value, and every value is relevant to our analysis.Since there are 3 different types of species, there might just be 3 clusters, assuming each species has different sepal/petal length/width.To determine the number of clusters, let's use the elbow method. I'll get all the length and width values, and then initialize an empty array for the sum of squared errors, what we need to calculate.I'll then assume a max of 11 clusters, and use the KMeans function to cluster the data in 'k' groups, minimizing the inertia, the sum-of-squares criterion. Next, I'll append the inertia to the sum_squared_errors list.I'll then plot sum_squared_errors, and look for a bend in the chart, hence the "Elbow" method, as it looks like a bend in the elbow. I'll categorize this as the point where the dataset no longer decreases by a significant amount.<jupyter_code>x_points = iris.iloc[:,[1,2,3,4]].values
sum_squared_errors = []
for k in range (1,11):
    kmeans = KMeans(n_clusters = k).fit(x_points)
    sum_squared_errors.append(kmeans.inertia_)

plt.plot(range(1,11),sum_squared_errors)<jupyter_output><empty_output><jupyter_text>Looks like we have a bend at 3, denoting 3 clusters (as we predicted).Let's then graph the datasets and identify the 3 clusters.<jupyter_code>fig, axs = plt.subplots(2, 3, figsize = (16, 8))

kmeans = KMeans(n_clusters = 3)
y_kmeans = kmeans.fit_predict(x_points)

m = 0
n = 0
for i in range(0, 4):
    for j in range(i+1, 4):
        if m < 3:
            axs[0, m].scatter(x_points[y_kmeans == 0, i],x_points[y_kmeans == 0, j], c = 'coral', label = 'Iris-setosa')
            axs[0, m].scatter(x_points[y_kmeans == 1, i], x_points[y_kmeans == 1, j], c = 'tan', label = 'Iris-versicolour')
            axs[0, m].scatter(x_points[y_kmeans == 2, i], x_points[y_kmeans == 2, j], c = 'skyblue', label = 'Iris-verginica')
            axs[0, m].scatter(kmeans.cluster_centers_[:, i], kmeans.cluster_centers_[:,j], s = 70, c = 'black', label = 'Centroids', marker = "D")
            m = m + 1
        elif n < 3:
            axs[1, n].scatter(x_points[y_kmeans == 0, i],x_points[y_kmeans == 0, j], c = 'coral', label = 'Iris-setosa')
            axs[1, n].scatter(x_points[y_kmeans == 1, i], x_points[y_kmeans == 1, j], c = 'tan', label = 'Iris-versicolour')
            axs[1, n].scatter(x_points[y_kmeans == 2, i], x_points[y_kmeans == 2, j], c = 'skyblue', label = 'Iris-verginica')
            axs[1, n].scatter(kmeans.cluster_centers_[:, i], kmeans.cluster_centers_[:,j], s = 70, c = 'black', label = 'Centroids', marker = "D")
            n = n + 1
plt.legend()
axs[0, 0].set_title('Sepal Length vs. Sepal Width')
axs[0, 1].set_title('Sepal Length vs. Petal Length')
axs[0, 2].set_title('Sepal Length vs. Petal Width')
axs[1, 0].set_title('Sepal Width vs. Petal Length')
axs[1, 1].set_title('Sepal Width vs. Petal Width')
axs[1, 2].set_title('Petal Length vs. Petal Width')<jupyter_output><empty_output><jupyter_text>We can also validate the optimal number of clusters by using the silhouette method as well!<jupyter_code>x_values = iris.iloc[:,[1,2,3,4]]
scores = []

for i in range(1,7):
    kmean = KMeans(n_clusters=k)
    kmean.fit(x_values)
    label = kmean.predict(x_values)
    scores.append(silhouette_score(x_values, label))

(np.array(scores)*100).round(2)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12606837606837606, 'fraction_numerical': 0.01899335232668566, 'mean_word_length': 4.609853528628495, 'pattern_counts': {'":': 0, '<': 27, '<?xml version=': 0, '>': 25, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '170490', 'license': 'MIT', 'n_tokens_mistral': 1599, 'n_tokens_neox': 1482, 'n_words': 448, 'path': '4 - Sparks Foundation Unsupervised Machine Learning/.ipynb_checkpoints/Untitled-checkpoint.ipynb', 'repo_name': 'quinnha/Data_Science_Adventures', 'size': '4212'}