<jupyter_start><jupyter_text>FormulaAI Hack 2022 - Challenge 1: Data AnalyticsThis Notebook is the main notebook of our repository. It walks you through from every-step of our story with the Challenge 1 from reading the data to evaluation of our model in a readable way. To get more details about the functions check out our src folder. IntroductionFormula 1 is one of the most competitive sports in the world where milliseconds matters. Engineers and technicians from every team use every data that they get in order to get ahead of their opponents. Ubimet provides weather radar screens to teams which allows them to track the current weather and make predictions during the race. Therefore weather forecasting takes a big part on the possible outcome of a race.Similarly, F1 2021, the official Formula 1 videogame developed by Codemasters, uses a physics engine that behaves like the real world.Our task was to use this generated data from Formula 1 Video Game and use it to forecast weather in 5,10,15,30,60 minutes intervals.  Getting Started<jupyter_code># Import Libraries
import random
import os.path as op
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from collections import Counter
import seaborn as sns
import sklearn
import pickle

#Import Custom Module
from src.data_prep import prepare_datasets, get_dfs

%load_ext autoreload
%autoreload 2
%matplotlib inline

# Set seeds in order to get consistent outputs
seed = 42
random.seed(seed)
np.random.seed(seed)
#Weather Classes
LABELS = {0: "clear",
          1: "light cloud",
          2: "overcast",
          3: "light rain",
          4: "heavy rain",
          5: "storm"}

single_val_cols = [
    "M_FORECAST_ACCURACY",
    "M_TOTAL_LAPS",
    'M_PIT_STOP_WINDOW_IDEAL_LAP',
    'M_PIT_STOP_WINDOW_LATEST_LAP',
    'M_PIT_STOP_REJOIN_POSITION',
    'M_STEERING_ASSIST',
    'M_BRAKING_ASSIST',
    'M_GEARBOX_ASSIST',
    'M_PIT_ASSIST',
    'M_PIT_RELEASE_ASSIST',
    'M_ERSASSIST',
    'M_DRSASSIST',
    'M_DYNAMIC_RACING_LINE',
    'M_DYNAMIC_RACING_LINE_TYPE'
]

multi_val_cols = [
    "M_WEATHER_FORECAST_SAMPLES_M_WEATHER",
    "M_WEATHER_FORECAST_SAMPLES_M_TRACK_TEMPERATURE",
    "M_WEATHER_FORECAST_SAMPLES_M_AIR_TEMPERATURE",
    "M_TRACK_TEMPERATURE_CHANGE",
    "M_AIR_TEMPERATURE_CHANGE",
    "M_RAIN_PERCENTAGE"]


use_flag_info = False

# Read the Data 
if op.exists(op.join("data", "train.csv")) and op.exists(op.join("data", "val.csv")) and op.exists(op.join("data", "test.csv")):
    train_df = pd.read_csv(op.join("data", "train.csv"))
    val_df = pd.read_csv(op.join("data", "val.csv"))
    test_df = pd.read_csv(op.join("data", "test.csv"))
else:
    #If its the first time prepare the datasets 
    train_df, val_df, test_df = prepare_datasets(op.join("data", "weather.csv"), single_val_cols, multi_val_cols, use_flag_info=use_flag_info)

df_dct = {"train": train_df, "val": val_df, "test": test_df}
df_timed_dct = get_dfs(df_dct, single_val_cols, multi_val_cols)

print("Time Offsets:",df_timed_dct.keys())
print("(Example) Datasets for 5 Minute Forecasting:",df_timed_dct["5"].keys())<jupyter_output>Time Offsets: dict_keys(['5', '10', '15', '30', '60'])
(Example) Datasets for 5 Minute Forecasting: dict_keys(['train', 'val', 'test'])<jupyter_text>Exploratory Data AnalysisAfter we split the dataset to train,validation,test sets we see that there are only 3 classes available on the dataset.By splitting each of the classes into 3 dataset x 5 time_offsets we got 15 datasets for our project.In this part we try give you how data looks like after our process:<jupyter_code>from src.plotting import plot_class_histograms
import copy

CLASS_IDX_BOUNDARY = 2

_df_timed_dct = copy.deepcopy(df_timed_dct)

df_timed_dct_subset = {}
for time_offset, df_timed in _df_timed_dct.items():
    df_timed_dct_subset[time_offset] = {}
    for typ, d_set in df_timed.items():
        df_timed_dct_subset[time_offset][typ] = d_set[d_set['TARGET_WEATHER'] <= CLASS_IDX_BOUNDARY].copy()
fig = plot_class_histograms(df_timed_dct_subset, max_class_idx=CLASS_IDX_BOUNDARY+1)

sns.distplot(df_timed_dct["5"]["train"]["TARGET_RAIN_PERCENTAGE"],kde=True)

sns.jointplot(x="M_WEATHER_FORECAST_SAMPLES_M_TRACK_TEMPERATURE",
 y="TARGET_RAIN_PERCENTAGE", data=df_timed_dct["5"]["train"])

table = data = df_timed_dct["5"]["train"]
sns.pairplot(table[multi_val_cols])<jupyter_output><empty_output><jupyter_text>Model Training & Evaluation  Weather<jupyter_code>from sklearn.ensemble import RandomForestClassifier, RandomForestRegressor
from sklearn.svm import SVC
from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from src.plotting import plot_model_performance

model = RandomForestClassifier(max_depth=10, random_state=0)
# pipeline = Pipeline([('scaler', StandardScaler()), ('lr', LogisticRegression())])
plot_model_performance(model, df_timed_dct_subset, model_type="weather", save=False)<jupyter_output>/Users/haldunbalim/Desktop/KuKu/src/plotting.py:118: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples,), for example using ravel().
  model.fit(X_train, y_train)<jupyter_text>Rain Percentage<jupyter_code>from sklearn.ensemble import RandomForestRegressor
import warnings
warnings.filterwarnings("ignore")

model = RandomForestRegressor(max_depth=5, random_state=0)
plot_model_performance(model, df_timed_dct_subset, model_type="rain_percentage", save=False)<jupyter_output>Rain Percentage 5:
Train MAE: 0.24116187100478642
Val MAE: 1.1920268096371107
Test MAE: 1.3753584198873374

Rain Percentage 10:
Train MAE: 0.3108515952106674
Val MAE: 2.0471277169050484
Test MAE: 2.8446667611077556

Rain Percentage 15:
Train MAE: 0.32535373047586624
Val MAE: 2.8433448657208675
Test MAE: 4.678089116511499

Rain Percentage 30:
Train MAE: 0.4368962352155224
Val MAE: 4.668916200658173
Test MAE: 6.403711065141063

Rain Percentage 60:
Train MAE: 0.33460539206381795
Val MAE: 4.5917902449179575
Test MAE: 7.176089420207825<jupyter_text>Export Model<jupyter_code>from src.model import Model

cls_temp = RandomForestClassifier(max_depth=10, random_state=0)
reg_temp = RandomForestRegressor(max_depth=5, random_state=0)
model = Model(cls_temp, reg_temp, cols=single_val_cols+multi_val_cols)
model.fit(df_timed_dct)
with open("models/model.pkl","wb") as file:
    pickle.dump(model, file)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0931167826759474, 'fraction_numerical': 0.05119876256767208, 'mean_word_length': 6.113311331133113, 'pattern_counts': {'":': 3, '<': 19, '<?xml version=': 0, '>': 18, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '653603', 'license': 'MIT', 'n_tokens_mistral': 2535, 'n_tokens_neox': 2293, 'n_words': 585, 'path': 'main.ipynb', 'repo_name': 'Hsgngr/KuKu', 'size': '6465'}