<jupyter_start><jupyter_text>Dino Game model fitting  Summary* Initial go at dino game* Read-in Spencer's data and do some very minimal pre-processing to make things play nicely with popular ML pacakges like sklearn, keras, and PyTorch* Fit basic Softmax Regression model -- performs WAY better than I expected* Though this nice performance could be super misguiding -- could be a result of the data structure I mentioend above, or the fact that these samples aren't IID (one frame of the game relies on another) so we are inherently violating the assumptions of a LogReg model. For this, I suggest a "thinning" type of approach across even more dino-games if we want to learn an actual mapping* Will likely need a lot more data to train a deep-net, especially since LogReg performs so well. Inputs* Spencer went all-out and sent massive image files in RGB channels (lol) so for now, I will just flatten these, drop that third channel (will mess up scaling of values in matrix though), and train on these vectors. Outputs* A label prediction for the key to press (or what action the model should take given the pixel values in the image?) Modeling task* Given an input image $X$, output a label for the action to be taken by the model (jump, duck, nothing) Evaluation metric* Classification accuracy * Cross-entropy loss for training Models* Multinomial logistic/Softmax regression* ConvNets (1D and 2D) -- Spencer suggest ResNet, but I have so little knowledge in this field, I kinda of want to do a "survey" first* SVM models? To-do* Data pre-processing to make this task more "learnable" is needed.     * Images should be converted to grayscale -- models not learning well    * Perhaps move from predicting the integer value of the key-press and move to a one-hot encoding (I can write a utility function to go back and forth from the two to make the actual dinosaur move)   * More modeling     * Move to Google Collab for ConvNets -- LogReg was painfully slow on my local machine to fit     * Flat try fancy models next!<jupyter_code>import random
import pickle
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import confusion_matrix, classification_report, accuracy_score
from pathlib import Path
from keras.utils.np_utils import to_categorical
from typing import List
import cv2 as cv
# Using SMOTE for the over sampling portion.
from imblearn.over_sampling import SMOTE
%matplotlib inline
target_address = os.path.join(Path(os.getcwd()).parent,'Window_capture\\Data\\command_keys.npy')
# screenshot_address = os.path.join(Path(os.getcwd()).parent,'Window_capture\\Data\\screenshots.npy')
screenshot_address = os.path.join(Path(os.getcwd()).parent,'Window_capture\\Data\\screenshots.npy')

labels = np.load(target_address)
images = np.load(screenshot_address, allow_pickle = True)


print("Length Command Keys Shape: ",labels.shape)
print("Length Screenshot Shape: ",images.shape)
print("Screenshot Shape: ",images[0].shape)
np.unique(labels, return_counts = True) # We see quite a bit of imbalance among the do nothing / jump / duck
np.unique(labels == -1, return_counts = True)
# res_list = [i for i, value in enumerate(labels) if value == -1] # Let's get rid of some -1 values.
# idx = np.random.choice(res_list, 2000, replace=False) # Randomly choose X number of entries to be deleted specified as -1
# images = pd.DataFrame(images) # flatten images then converted to dataframe for easier removal of idx
# images = np.array(images.drop(images.index[idx])) # flatten images then converted to dataframe for easier removal of idx
# labels = np.delete(labels, idx)
# print(images.shape, labels.shape)
# print(np.unique(labels, return_counts = True))
X_train, X_test, y_train, y_test = train_test_split(images, labels, test_size = 0.25)
np.unique(y_train, return_counts = True)
#Oversampling the data
smote = SMOTE(random_state = 101)
X_train_samp, y_train_samp = smote.fit_resample(X_train, y_train)
np.unique(y_train_samp, return_counts = True) # Oversampled Balanced.
# Fit LogReg model
log_reg = LogisticRegression()
log_reg.fit(X_train_samp,y_train_samp)
y_hat = log_reg.predict(X_test)
print(f'LogReg accuracy on held-out frames = {round(accuracy_score(y_test, y_hat),4)}')
confusion_matrix(y_test, y_hat, labels=[-1, 38, 40])
target_names = ['nothing', 'up', 'down']
print(classification_report(y_test, y_hat, target_names=target_names))
pickle.dump(log_reg, open('Existing_Models/log-reg.pkl', 'wb'))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08153114186851211, 'fraction_numerical': 0.005190311418685121, 'mean_word_length': 5.283967391304348, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '371282', 'license': 'MIT', 'n_tokens_mistral': 1401, 'n_tokens_neox': 1342, 'n_words': 562, 'path': 'Dinosaur Game/Modeling/basic-models.ipynb', 'repo_name': 'SpencerPao/ComputerVision', 'size': '4624'}