<jupyter_start><jupyter_text>ML 1 - Pattern Recognition Preliminaries: Feature Extraction* Name: <NAME>* Date: December 14, 2020* Student Number: 2015-06435 PreliminariesImport all important packages from Python: * **Pandas**: Reading and loading the file; make dataframes * **Numpy**: For multidimensional array and operations on arrays* **Matplotlib/Seaborn**: For our visualizations (2D and 3D)<jupyter_code># Import important packages

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

from mpl_toolkits.mplot3d import Axes3D
# Create dataframe for each fruit class

mango = pd.read_excel('new_features.xlsx', sheet_name = 'Class 1 - Mango')
orange = pd.read_excel('new_features.xlsx', sheet_name = 'Class 2 - Orange')
banana = pd.read_excel('new_features.xlsx', sheet_name = 'Class 3 - Banana')
mango = pd.DataFrame(mango)
orange = pd.DataFrame(orange)
banana = pd.DataFrame(banana)
orange.head()<jupyter_output><empty_output><jupyter_text>Hue* If Red is max, then Hue = (G-B)/(max-min)* If Green is max, then Hue = 2.0 + (B-R)/(max-min)* If Blue is max, then Hue = 4.0 + (R-G)/(max-min)<jupyter_code>def hue(R,G,B):
    vals = [R, G, B]
    max_vals = max(vals)
    min_vals = min(vals)
    
    if max_vals == R:
        hue = 60 * (G-B)/(max_vals - min_vals)
    elif max_vals == G:
        hue = 60 * (2+(B-R)/(max_vals - min_vals))
    else:
        hue = 60 * (4+(R-G)/(max_vals - min_vals))
    
    return hue
mango['Hue'] = mango.apply(lambda x: hue(x['R'], x['G'], x['B']), axis = 1)
mango["Class"] = 'Mango'
orange['Hue'] = orange.apply(lambda x: hue(x['R'], x['G'], x['B']), axis = 1)
orange["Class"] = 'Orange'
banana['Hue'] = banana.apply(lambda x: hue(x['R'], x['G'], x['B']), axis = 1)
banana["Class"] = 'Banana'

mango.head()<jupyter_output><empty_output><jupyter_text>VisualizationsLet's check if clustering is present when the feature points are plotted in 2D or 3D space<jupyter_code>data = pd.concat([mango, orange, banana], ignore_index = True, sort = False)
data = data.drop(columns = ['Image', 'R', 'G', 'B'])
data['Hue_norm'] = data['Hue']/max(data['Hue'])
data['Median'] = data['Median']/max(data['Median'])
data['Skew'] = data['Skew']/max(data['Skew'])
data['AR'] = data['AR']/max(data['AR'])
data.tail()
data['species']=pd.Categorical(data['Class'])
my_color=data['species'].cat.codes

v = data['Median']
w = data['AR']
x = data['Skew']
y = data['Round']
z = data['Hue_norm']

fig, ax = plt.subplots(2,2, figsize=(15,10))
ax[0,0].scatter(x[my_color == 1], y[my_color == 1] , s = 60, marker = 'x', label = 'Mango')
ax[0,0].scatter(x[my_color == 2], y[my_color == 2] , s = 90, marker = '*', label = 'Orange')
ax[0,0].scatter(x[my_color == 0], y[my_color == 0] , s = 60, marker = 'o', label = 'Banana')
ax[0,0].legend()
ax[0,0].set_xlabel('Skewness')
ax[0,0].set_ylabel('Roundness')

ax[0,1].scatter(z[my_color == 1], y[my_color == 1] , s = 60, marker = 'x', label = 'Mango')
ax[0,1].scatter(z[my_color == 2], y[my_color == 2] , s = 90, marker = '*', label = 'Orange')
ax[0,1].scatter(z[my_color == 0], y[my_color == 0] , s = 60, marker = 'o', label = 'Banana')
ax[0,1].legend()
ax[0,1].set_xlabel('Normalized Hue')
ax[0,1].set_ylabel('Roundness')

ax[1,0].scatter(v[my_color == 1], z[my_color == 1] , s = 60, marker = 'x', label = 'Mango')
ax[1,0].scatter(v[my_color == 2], z[my_color == 2] , s = 90, marker = '*', label = 'Orange')
ax[1,0].scatter(v[my_color == 0], z[my_color == 0] , s = 60, marker = 'o', label = 'Banana')
ax[1,0].legend()
ax[1,0].set_ylabel('Normalized Hue')
ax[1,0].set_xlabel('Median')

ax[1,1].scatter(z[my_color == 1], w[my_color == 1] , s = 60, marker = 'x', label = 'Mango')
ax[1,1].scatter(z[my_color == 2], w[my_color == 2] , s = 90, marker = '*', label = 'Orange')
ax[1,1].scatter(z[my_color == 0], w[my_color == 0] , s = 60, marker = 'o', label = 'Banana')
ax[1,1].legend()
ax[1,1].set_ylabel('Aspect Ratio')
ax[1,1].set_xlabel('Normalized Hue')

plt.savefig('2D.png')
data['species']=pd.Categorical(data['Class'])
my_color=data['species'].cat.codes


fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.scatter(z[my_color == 1], x[my_color == 1], y[my_color == 1], s=40, label = 'Mango', alpha = 0.8)
ax.scatter(z[my_color == 2], x[my_color == 2], y[my_color == 2], s=40, label = 'Orange', alpha = 0.8)
ax.scatter(z[my_color == 0], x[my_color == 0], y[my_color == 0], s=40, label = 'Banana', alpha = 0.8)
ax.legend(loc = 2)
ax.set_xlabel('Normalized Hue')
ax.set_ylabel('Roundness')
ax.set_zlabel('Skewness')

plt.savefig('3D.png')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.2174197773411919, 'fraction_numerical': 0.03710980135341629, 'mean_word_length': 5.183535762483131, 'pattern_counts': {'":': 0, '<': 14, '<?xml version=': 0, '>': 14, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '620611', 'license': 'MIT', 'n_tokens_mistral': 2040, 'n_tokens_neox': 1888, 'n_words': 417, 'path': 'Applied Physics 186/ML1 - Pattern Recognition.ipynb', 'repo_name': 'gpgloria/Physics-Projects', 'size': '4591'}