<jupyter_start><jupyter_text>Data Analysis on Video Game Sales  Pre Processing Data  Final Objective - **Predict Japanese Sales using North America, European and Other Sales, Genre and Platform.**> This result is important for games that want to enter the Japenese Market, but don't know what to expect for their respective sales team. Also, it will be a good measure to invest (or not) in japanese translations.  Importing Libraries<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

from sklearn.preprocessing import LabelBinarizer
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler<jupyter_output><empty_output><jupyter_text>Importing Data<jupyter_code>path = "../Data/vgsales.csv"
df_games = pd.read_csv(path, index_col=0)
df_games<jupyter_output><empty_output><jupyter_text>DataFrame summary<jupyter_code>df_games.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
Int64Index: 16598 entries, 1 to 16600
Data columns (total 10 columns):
Name            16598 non-null object
Platform        16598 non-null object
Year            16327 non-null float64
Genre           16598 non-null object
Publisher       16540 non-null object
NA_Sales        16598 non-null float64
EU_Sales        16598 non-null float64
JP_Sales        16598 non-null float64
Other_Sales     16598 non-null float64
Global_Sales    16598 non-null float64
dtypes: float64(6), object(4)
memory usage: 1.4+ MB<jupyter_text>About Features - Year -> year values- NA_Sales -> million dollars- EU_Sales -> million dollars- JP_Sales -> million dollars- Other_Sales -> million dollars- Global_Sales -> million dollars  Removing Rows with NaN values<jupyter_code>df_games = df_games.dropna(how='any',axis=0)
df_games.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
Int64Index: 16291 entries, 1 to 16600
Data columns (total 10 columns):
Name            16291 non-null object
Platform        16291 non-null object
Year            16291 non-null float64
Genre           16291 non-null object
Publisher       16291 non-null object
NA_Sales        16291 non-null float64
EU_Sales        16291 non-null float64
JP_Sales        16291 non-null float64
Other_Sales     16291 non-null float64
Global_Sales    16291 non-null float64
dtypes: float64(6), object(4)
memory usage: 1.4+ MB<jupyter_text>AboutAfter dropping NaN values, we have lost **307** rows of data.  Understanding Simple Metric for Numeric Columns<jupyter_code>df_games.describe()<jupyter_output><empty_output><jupyter_text>As we can see, from Global Sales, we have a massive almost **50%** of the 16291 games selling less than 170k dollars worldwide.  Columns to be used for Modelling - Platform- Genre- NA_Sales- EU_Sales- JP_Sales- Other_Sales  Observations - Global Sales is just the sum for all Sales columns.- Platform and Genre need to have their respective key as a number for the purpose of modelling them into a machine learning algorithm.  Generating correct .CSVs  CSV for Visualizations<jupyter_code>df_visualizations = df_games
df_visualizations.to_csv("../Data/visualizations_sales.csv")
df_visualizations<jupyter_output><empty_output><jupyter_text>CSV for Modelling<jupyter_code>df_model = df_games[["Platform", "Genre", "NA_Sales", "EU_Sales", "Other_Sales", "JP_Sales"]]
df_model<jupyter_output><empty_output><jupyter_text>Defining key values for platforms<jupyter_code>df_model_sklearn = df_model.copy()

lb = LabelBinarizer()
lb_results = lb.fit_transform(df_model_sklearn['Platform'])
lb_results_df = pd.DataFrame(lb_results, columns=lb.classes_, index=df_model.index)

df_model_result = pd.concat([df_model_sklearn, lb_results_df], axis=1)

df_model = df_model_result.drop("Platform", axis="columns")<jupyter_output><empty_output><jupyter_text>Defining key values for genres<jupyter_code>df_model_sklearn = df_model.copy()

lb = LabelBinarizer()
lb_results = lb.fit_transform(df_model_sklearn['Genre'])
lb_results_df = pd.DataFrame(lb_results, columns=lb.classes_, index=df_model.index)

df_model_result = pd.concat([df_model_sklearn, lb_results_df], axis=1)

df_model = df_model_result.drop("Genre", axis="columns")<jupyter_output><empty_output><jupyter_text>Results for df_model<jupyter_code>df_model.head()<jupyter_output><empty_output><jupyter_text>It looks correct now! Now the only important thing to do is to separate data into **TRAIN** and **VALIDATION**. I chose not to normalize my data because of the range of Sales values (from 0 to 41), having 75% of the values below 1.  Separating Data<jupyter_code>df_output_model = df_model[["JP_Sales"]]
df_input_model  = df_model.drop("JP_Sales", axis="columns")
input_train, input_validation, output_train, output_validation = train_test_split(
                                                                                  df_input_model, 
                                                                                  df_output_model, 
                                                                                  test_size=0.25,
                                                                                 )<jupyter_output><empty_output><jupyter_text>Saving Main DataFrames as CSV<jupyter_code>input_train.to_csv("../Data/input_train_sales.csv")
input_validation.to_csv("../Data/input_validation_sales.csv")
output_train.to_csv("../Data/output_train_sales.csv")
output_validation.to_csv("../Data/output_validation_sales.csv")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08371412904405044, 'fraction_numerical': 0.03509413269968927, 'mean_word_length': 4.020183486238532, 'pattern_counts': {'":': 0, '<': 49, '<?xml version=': 0, '>': 56, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 12, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '639597', 'license': 'MIT', 'n_tokens_mistral': 1834, 'n_tokens_neox': 1643, 'n_words': 447, 'path': 'AnalysisNotebook/pre_processing.ipynb', 'repo_name': 'MatheusMAssis/Data-Analysis-Game-Sales', 'size': '5471'}