<jupyter_start><jupyter_text>Context The DataWe will be using the famous [Iris flower data set](http://en.wikipedia.org/wiki/Iris_flower_data_set). The data set consists of 50 samples from each of three species of Iris (Iris setosa, Iris virginica and Iris versicolor), so 150 total samples. Four features were measured from each sample: the length and the width of the sepals and petals, in centimeters. ___ Tasks***1- Perform the Data Preprocessing******2- Perform EDA******3- Standardize the variables******4- Fit the SVM classifier on the dataset******5- Perform Predictions and Evaluations******6- Using a confusion Matrix, check model's accuracy******7- Perform the SVM Hyperparameter Tuning using GridSearchCV******8- Retrain the model******9- Repeat step 6.*** ___ Recommended methodology***You will be using sklearn's SVM Classifier `from sklearn.svm import SVC`.******For Model evaluation, you can use `from sklearn.metrics import classification_report,confusion_matrix`******For GridSearch, you can use `from sklearn.model_selection import GridSearchCV`*** ___***Things to do***- Perform the data pre-processing.- Deal with missing values- Perform EDA- Using `StandardScalar` standardize the variables expect the taregt variable- Convert the scaled features to a dataframe- Split into training and testing set- Fit SVM classifier- Using a confusion Matrix, check model's accuracy- Perform the SVM Hyperparameter Tuning using GridSearchCV- Retrain the model- Repeat step 6.***What will be new***- You will learn the SVM classifier and how it works.- How to tune the hyperparameter using GridSearchCV ***What will be tricky***- SVM Hyperparameter Tuning using GridSearchCV will be tricky in this project. What you can do is:Follow this tutorial:[How to tune hyperparameter using GridSearchCV in Python](https://www.geeksforgeeks.org/svm-hyperparameter-tuning-using-gridsearchcv-ml/)<jupyter_code>import seaborn as sns
import pandas as pd

iris = sns.load_dataset('iris')
# tranfor iris array into a dataframe, for convenience

iris = pd.DataFrame(iris,columns=['sepal_length','sepal_width','petal_length','petal_width', 'species']  )
iris.head()<jupyter_output><empty_output><jupyter_text>Standardise the data set<jupyter_code># lets standardize the data
from sklearn import preprocessing
from sklearn.preprocessing import StandardScaler

# separate the data from the target attributes
X = iris[['sepal_length', 'sepal_width','petal_length','petal_width']]
y = iris['species']

# Get column names first (provided data was organised this way)
names = ['sepal_length','sepal_width','petal_length','petal_width']   

# standardize the data attributes
scaler = preprocessing.StandardScaler()
iris = scaler.fit_transform(X)

# save as pandas datagrame
iris = pd.DataFrame(iris, columns=names)
iris = pd.concat((iris,y),axis=1)                # add the column with string values
iris.head()<jupyter_output><empty_output><jupyter_text>EDA<jupyter_code># is it better to perform EDA before standardisation of dataset?
sns.set_style('whitegrid')

sns.FacetGrid(iris,hue='species',size=6).map(plt.scatter,'sepal_length','sepal_width').add_legend()
plt.show();
sns.set_style('whitegrid')

sns.FacetGrid(iris,hue='species',size=6).map(plt.scatter,'petal_length','petal_width').add_legend()
plt.show();
features_mean=list(iris.columns[0:4])

# split dataframe into 4 based on label
setosa = iris[iris['species'] == 'setosa']
versicolor = iris[iris['species'] == 'versicolor']
virginica = iris[iris['species'] == 'virginica']


#Stack the data
plt.rcParams.update({'font.size': 9})
fig, axes = plt.subplots(nrows=1, ncols=4, figsize=(20,6))
axes = axes.ravel()

for idx,ax in enumerate(axes):
    ax.figure
   
    ax.hist([setosa[features_mean[idx]],versicolor[features_mean[idx]],virginica[features_mean[idx]]], bins=20, alpha=0.5,stacked=True, label=['setosa','versicolor','virginica'],color=['tab:blue','tab:orange','tab:green'])
    ax.legend(loc='upper right')
    ax.set_title(features_mean[idx])<jupyter_output><empty_output><jupyter_text>Machine Learning  Preparation<jupyter_code># separate the data from the target attributes
X = iris[['sepal_length', 'sepal_width','petal_length','petal_width']]
y = iris['species']
from sklearn.model_selection import train_test_split

# Prepare the test / train sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=1)<jupyter_output><empty_output><jupyter_text>SVM<jupyter_code>from sklearn.svm import SVC, LinearSVC

svc = SVC()
svc.fit(X_train, y_train)
y_pred = svc.predict(X_test)

svc_train = round(svc.score(X_train, y_train) * 100, 2)
svc_test = round(svc.score(X_test, y_test) * 100, 2)

print("Train Accuracy: ",svc_train)
print("Test Accuracy: ", svc_test)



from sklearn.metrics import classification_report, confusion_matrix
cm = confusion_matrix(y_test, y_pred)
print(cm)

print(classification_report(y_test, y_pred))<jupyter_output>[[11  0  0]
 [ 0 12  1]
 [ 0  0  6]]
              precision    recall  f1-score   support

      setosa       1.00      1.00      1.00        11
  versicolor       1.00      0.92      0.96        13
   virginica       0.86      1.00      0.92         6

    accuracy                           0.97        30
   macro avg       0.95      0.97      0.96        30
weighted avg       0.97      0.97      0.97        30<jupyter_text>Perform the SVM Hyperparameter Tuning using GridSearchCV<jupyter_code># is GridSearchCV method transposable to other models, or is it specific to SVM?

https://towardsdatascience.com/svm-hyper-parameter-tuning-using-gridsearchcv-49c0bc55ce29
    
kernels = ['Polynomial', 'RBF', 'Sigmoid','Linear']

#A function which returns the corresponding SVC model

def getClassifier(ktype):
    if ktype == 0:
        # Polynomial kernal
        return SVC(kernel='poly', degree=3, gamma="auto")
    elif ktype == 1:
        # Radial Basis Function kernal
        return SVC(kernel='rbf', gamma="auto")
    elif ktype == 2:
        # Sigmoid kernal
        return SVC(kernel='sigmoid', gamma="auto")
    elif ktype == 3:
        # Linear kernal
        return SVC(kernel='linear', gamma="auto")
for i in range(4):

# Train a SVC model using different kernal
    svclassifier = getClassifier(i) 
    svclassifier.fit(X_train, y_train)

    # Make prediction
    y_pred = svclassifier.predict(X_test)

    # Evaluate our model
    print("Evaluation:", kernels[i], "kernel")
    print(classification_report(y_test,y_pred))
from sklearn.model_selection import GridSearchCV

param_grid = {'C': [0.1,1, 10, 100], 'gamma': [1,0.1,0.01,0.001],'kernel': ['rbf', 'poly', 'sigmoid']}
grid = GridSearchCV(SVC(),param_grid,refit=True,verbose=1)
grid.fit(X_train,y_train)

print(grid.best_estimator_)
grid_predictions = grid.predict(X_test)
print(confusion_matrix(y_test,grid_predictions))
print(classification_report(y_test,grid_predictions))<jupyter_output>[[11  0  0]
 [ 0 13  0]
 [ 0  0  6]]
              precision    recall  f1-score   support

      setosa       1.00      1.00      1.00        11
  versicolor       1.00      1.00      1.00        13
   virginica       1.00      1.00      1.00         6

    accuracy                           1.00        30
   macro avg       1.00      1.00      1.00        30
weighted avg       1.00      1.00      1.00        30<jupyter_text>Applying K-Mean Clustering<jupyter_code>iris = sns.load_dataset('iris')
iris = pd.DataFrame(iris,columns=['sepal_length','sepal_width','petal_length','petal_width', 'species']  )

X = iris[['sepal_length', 'sepal_width']]



from sklearn.cluster import KMeans

inertia = []
for i in range(2,20):
    kmeans = KMeans(n_clusters=i,init='k-means++',n_init=15,max_iter=500)
    kmeans.fit(X)
    inertia.append(kmeans.inertia_)
from sklearn.metrics import silhouette_score

silhouette = {}
for i in range(2,20):
    kmeans = KMeans(n_clusters=i,
               init='k-means++',
               n_init=15,
               max_iter=500,
               random_state=17)
    kmeans.fit(X)
    silhouette[i] = silhouette_score(X, kmeans.labels_, metric='euclidean')
sns.set(style='white',font_scale=1.1, rc={'figure.figsize':(12,5)})

plt.subplot(1, 2, 1)

plt.plot(range(2,len(inertia)+2), inertia, marker='o',lw=2,ms=8,color='tab:orange')
plt.xlabel('Number of clusters')
plt.title('K-means Inertia',fontweight='bold')
plt.grid(True)

plt.subplot(1, 2, 2)

plt.bar(range(len(silhouette)), list(silhouette.values()), align='center',color= 'tab:orange',width=0.5)
plt.xticks(range(len(silhouette)), list(silhouette.keys()))
plt.grid(True)
plt.title('Silhouette Score',fontweight='bold')
plt.xlabel('Number of Clusters')


plt.show()

# Number of clusters
kmeans = KMeans(n_clusters=3)

# Fitting the input data
kmeans = kmeans.fit(X)

# Getting the cluster labels
labels = kmeans.predict(X)

# Centroid values
centroids = kmeans.cluster_centers_

print(centroids)
# Fitting K-Means to the dataset
kmeans = KMeans(n_clusters=3, init = 'k-means++', random_state = 15)
kmeans = kmeans.fit(X)

# creating the centroids df
centroids_df = pd.DataFrame(kmeans.cluster_centers_, columns = list(X.columns.values))

# creating the plot
fig, ax = plt.subplots(1, 1)
iris.plot.scatter(x = 'sepal_length', y = 'sepal_width', c= kmeans.labels_, figsize=(12,8), colormap='jet', ax=ax, mark_right=False)
centroids_df.plot.scatter(x = 'sepal_length', y = 'sepal_width', ax = ax,  s = 100, mark_right=False, c='r')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1283110828565374, 'fraction_numerical': 0.027972027972027972, 'mean_word_length': 4.3783475783475785, 'pattern_counts': {'":': 0, '<': 27, '<?xml version=': 0, '>': 27, 'https://': 2, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '644145', 'license': 'BSD-2-Clause', 'n_tokens_mistral': 3462, 'n_tokens_neox': 3202, 'n_words': 785, 'path': 'Machine Learning/Support Vector Machine/Support Vector Machine.ipynb', 'repo_name': 'k21k/Python-Notes', 'size': '9438'}