<jupyter_start><jupyter_text>Predict Pumpkin SeedsAuthor: <NAME>Course Project, UC Irvine, Math 10, W22 Student ID: 90294322  IntroductionIntroduce your project here.  About 3 sentences.The goal of my project is to predict the category of the pumpkin seeds given by a series of input data, like its perimeter, compactness, area and so on. The dataset I use only contain two category of the pumpkin seeds, so I use the Logistic Regression as my training model. Besides, I also try to find the relationship between each variable and the final output, and choose two most significant variables to plot a relationship chart.   Main portion of the project(You can either have all one section or divide into multiple sections)<jupyter_code>pip install openpyxl 
import pandas as pd
import altair as alt
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import log_loss
import torch.nn as nn
import numpy as np<jupyter_output>/shared-libs/python3.7/py/lib/python3.7/site-packages/tqdm/auto.py:22: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm<jupyter_text>Import data<jupyter_code>df = pd.read_excel("Pumpkin_Seeds_Dataset.xlsx")
df.columns<jupyter_output><empty_output><jupyter_text>Feature selectionSelect important features. Eliminate highly related x variables. In such a way we can prevent the model from overfitting to some extent.<jupyter_code>X_unsel = df[['Area', 'Perimeter', 'Major_Axis_Length', 'Minor_Axis_Length',
       'Convex_Area', 'Equiv_Diameter', 'Eccentricity', 'Solidity', 'Extent',
       'Roundness', 'Aspect_Ration', 'Compactness' ]]
corr = X_unsel.corr()

corr.style.background_gradient(cmap='coolwarm')<jupyter_output><empty_output><jupyter_text>We saw Area, Equv_Diameter and Convex Area, Perimeter, and Major Axis Length are closely related. Compactness, Aspect_Ration, Roundness, and Eccentrity are closely related. (abs of corr > 0.9).Thus, We can just pick Perimeter and Compactness as two representative features from these six features. All other features do not have such close relationship. We can leave them unchanged.<jupyter_code>X = df[['Perimeter', 'Minor_Axis_Length', 'Solidity', 'Extent', 'Compactness' ]]
Y = df['Class']<jupyter_output><empty_output><jupyter_text>Check imbalanced dataNext, we check whether it's balanced data set. If it's inbalanced, we cannot simply use the (correct_num_of_pred / total_num) to judge whether the model has a good performance.The result shows it's approximately balanced.<jupyter_code>Y.value_counts()<jupyter_output><empty_output><jupyter_text>Standardize the data<jupyter_code>scaler = StandardScaler()
scaler.fit(X)
X_scaled = scaler.transform(X)
X_scaled<jupyter_output><empty_output><jupyter_text>Build training and test set<jupyter_code>X_train, X_test, y_train, y_test = train_test_split(X_scaled, Y, test_size=0.2)<jupyter_output><empty_output><jupyter_text>Train the model<jupyter_code>clf = LogisticRegression()
clf.fit(X_train, y_train)
clf.predict_proba(X_train)
clf.predict(X_train)<jupyter_output><empty_output><jupyter_text>Check whether overfits<jupyter_code>train_error = log_loss(y_train, clf.predict_proba(X_train))
test_error = log_loss(y_test, clf.predict_proba(X_test))
train_error
test_error
test_accuracy =np.count_nonzero(clf.predict(X_test) == y_test)/len(X_test)
train_accuracy =np.count_nonzero(clf.predict(X_train) == y_train)/len(X_train)
test_accuracy
train_accuracy
print(f"The log error for the training set is {train_error}, and the log error for the test set is {test_error}")
print(f"The accuracy for the training set is {train_accuracy}, and the accuracy for the test set is {test_accuracy}")<jupyter_output>The log error for the training set is 0.3226528735934736, and the log error for the test set is 0.30269970620960607
The accuracy for the training set is 0.875, and the accuracy for the test set is 0.874<jupyter_text>Visualize the feature importance<jupyter_code>list(clf.coef_[0])
df_coef = pd.DataFrame()
df_coef['feature'] = ['Perimeter', 'Minor_Axis_Length', 'Solidity', 'Extent', 'Compactness']
df_coef['value'] = list(abs(clf.coef_[0]))
df_coef['type'] = ['negative' if n < 0 else 'positive'for n in clf.coef_[0]]
df_coef
alt.Chart(df_coef).mark_bar().encode(
    x = 'feature',
    y = 'value',
    color = 'type',
    opacity=alt.value(0.5),
).properties(
    title = 'Feature Importance'
)<jupyter_output><empty_output><jupyter_text>Finally, we can plot the relationship chart using the two most important features, which are compactness and Perimeter, and find the relationship between these two x variables and y variables.<jupyter_code>tmp = []
for i in y_test.index:
    tmp.append(df['Compactness'][i])

tmp2 = []
for i in y_test.index:
    tmp2.append(df['Perimeter'][i])

len(tmp2)
df_pred = pd.DataFrame()
df_pred['Compactness'] = tmp
df_pred['Perimeter'] = tmp2
df_pred['type'] = clf.predict(X_test)
df_pred
c1 = alt.Chart(df_pred).mark_circle().encode(
    x = 'Compactness',
    y = 'Perimeter',
    color = 'type'   
).properties(
    title = 'pred'
)

df_true = pd.DataFrame()
df_true['Compactness'] = tmp
df_true['Perimeter'] = tmp2
df_true['type'] = list(y_test)
c2 = alt.Chart(df_true).mark_circle().encode(
    x = 'Compactness',
    y = 'Perimeter',
    color = 'type'   
).properties(
    title = 'true'
)
c1|c2<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10316888728844076, 'fraction_numerical': 0.01458408354339215, 'mean_word_length': 6.233072916666667, 'pattern_counts': {'":': 0, '<': 45, '<?xml version=': 0, '>': 45, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '177009', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1943, 'n_tokens_neox': 1865, 'n_words': 529, 'path': '_build/html/_sources/Proj/StudentProjects/ZianDong.ipynb', 'repo_name': 'ChristopherDavisUCI/UCI-Math-10-W22', 'size': '5557'}