<jupyter_start><jupyter_text>Part IV: Machine learning model training<jupyter_code>import pandas as pd

# import ML libraries
from sklearn.model_selection import train_test_split
from sklearn.dummy import DummyClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier

# import metrics
from sklearn.metrics import confusion_matrix, classification_report
# read Part III csv as DataFrame
df = pd.read_csv("../Dataset/thebiglist_clean_extended_final.csv")
df
# look at Stars
df["Stars"].value_counts()
# import plotting library
import matplotlib.pyplot as plt
plt.hist(df["Stars"])
df["Stars"].median()
# cut-off for a "Good" rating can be anything more than 3.75
len(df)
len(df[df["Stars"] > 3.75])
# check to see if dividing would create an imbalance
# qcut() automatically splits data into intervals
pd.qcut(df["Stars"], q = 2, labels = [0, 1])
df["binStars"] = pd.qcut(df["Stars"], q = 2, labels = [0, 1])
df
# check if split is roughly equal
df["binStars"].value_counts()
# prepare independent and dependent variables
X = df.drop(["binStars", "Stars"], axis = 1)
y = df["binStars"]
# split dataset into train and test
X_train, X_test, y_train, y_test = train_test_split(X, y,
                                                   test_size = 0.2,
                                                   stratify = y)
# set baseline with a DummyClassifier
dummy = DummyClassifier()
dummy.fit(X_train, y_train)
dummy_pred = dummy.predict(X_test)
# assess performance of DummyClassifier
confusion_matrix(y_test, dummy_pred)<jupyter_output><empty_output><jupyter_text>![confusion_matrix-2.png](attachment:confusion_matrix-2.png) Ideally, (1,1) and (2,2) are maximised, (1,2) and (2,1) are minimised.<jupyter_code>print(classification_report(y_test, dummy_pred))<jupyter_output>precision    recall  f1-score   support

           0       0.56      0.56      0.56       407
           1       0.46      0.46      0.46       332

    accuracy                           0.52       739
   macro avg       0.51      0.51      0.51       739
weighted avg       0.52      0.52      0.52       739<jupyter_text>Hence, any accuracy above 0.52 is an improvement.<jupyter_code># start with LogisticRegression
logr = LogisticRegression()
logr.fit(X_train, y_train)
logr_pred = logr.predict(X_test)
confusion_matrix(y_test, dummy_pred)
confusion_matrix(y_test, logr_pred)
print(classification_report(y_test, logr_pred))<jupyter_output>precision    recall  f1-score   support

           0       0.67      0.69      0.68       407
           1       0.61      0.59      0.60       332

    accuracy                           0.65       739
   macro avg       0.64      0.64      0.64       739
weighted avg       0.64      0.65      0.65       739<jupyter_text>Higher accuracy score of 0.66.<jupyter_code># DecisionTreeClassifier
tree = DecisionTreeClassifier()
tree.fit(X_train, y_train)
tree_pred = tree.predict(X_test)
confusion_matrix(y_test, tree_pred)
print(classification_report(y_test, tree_pred))
# RandomForestClassifier
forest = RandomForestClassifier()
forest.fit(X_train, y_train)
forest_pred = forest.predict(X_test)
confusion_matrix(y_test, forest_pred)
print(classification_report(y_test, forest_pred))<jupyter_output>precision    recall  f1-score   support

           0       0.66      0.63      0.65       407
           1       0.57      0.59      0.58       332

    accuracy                           0.62       739
   macro avg       0.61      0.61      0.61       739
weighted avg       0.62      0.62      0.62       739<jupyter_text>Compare all 4 confusion matrices:<jupyter_code>confusion_matrix(y_test, dummy_pred)
confusion_matrix(y_test, logr_pred)
confusion_matrix(y_test, tree_pred)
confusion_matrix(y_test, forest_pred)
# array containing a list of coefficients of the column values
# positive -> factor of 1
# negative -> factor of 0
logr.coef_[0]
logr_feature = pd.DataFrame({"feature": X.columns,
                            "importance": logr.coef_[0]})

# factors that tend to give higher Stars (top) and lower Stars (bottom)
logr_feature.sort_values("importance", ascending = False)
tree_feature = pd.DataFrame({"feature": X.columns,
                            "importance": tree.feature_importances_})

# factors that tend to give higher Stars (top) and lower Stars (bottom)
tree_feature.sort_values("importance", ascending = False)
forest_feature = pd.DataFrame({"feature": X.columns,
                            "importance": forest.feature_importances_})

# factors that tend to give higher Stars (top) and lower Stars (bottom)
forest_feature.sort_values("importance", ascending = False)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09236609025727541, 'fraction_numerical': 0.043652467313369886, 'mean_word_length': 2.9557964970809008, 'pattern_counts': {'":': 6, '<': 18, '<?xml version=': 0, '>': 21, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 16, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '479509', 'license': 'MIT', 'n_tokens_mistral': 1617, 'n_tokens_neox': 1506, 'n_words': 390, 'path': 'Notebooks/Project Ramen Rating Analysis Part IV.ipynb', 'repo_name': 'Gamers-Blended/Ramen-Rating-Analysis', 'size': '4742'}