<jupyter_start><jupyter_text>**Overview**.    The gaming industry is certainly one of the thriving industries of the modern age and one of those that are most influenced by the advancement in technology. With the availability of technologies like AR/VR in consumer products like gaming consoles and even smartphones, the gaming sector shows great potential. In this hackathon, you as a data scientist must use your analytical skills to predict the sales of video games depending on given factors. Given are **8 distinguishing factors** that can influence the sales of a video game. Your objective as a data scientist is to build a machine learning model that can accurately predict the sales in millions of units for a given game. Project dataset source link: [MachineHack Hackathon](https://www.machinehack.com/hackathons/video_game_sales_prediction_weekend_hackathon_10) After registering for the hackathon we receive 3 files viz. Sample submission, Train and Test.csv. **Data Description**:-The unzipped folder will have the following files.Train.csv –  3506 observations.     Test.csv –  1503 observations.    Sample Submission – Sample format for the submission.    **Target Variable**: SalesInMillions Once downloaded we can run below code cell to upload the files to this notebook. Run the below code cell, then click on `Choose Files` button to upload files to google colab.<jupyter_code>from google.colab import files
uploaded = files.upload()
for fn in uploaded.keys():
  print('User uploaded file "{name}" with length {length} bytes'.format(
      name=fn, length=len(uploaded[fn])))<jupyter_output><empty_output><jupyter_text>File Imports<jupyter_code>#Import necessary libraries
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split
import warnings
warnings.filterwarnings(action='ignore')
# Read the csv files
input = pd.read_csv("Train.csv")
#print all columns to understand the dataset
input.head()<jupyter_output><empty_output><jupyter_text>Data cleaning<jupyter_code>input.isnull().sum()<jupyter_output><empty_output><jupyter_text>There are no null values in the dataset. So we can move to the next step of removing unnecessary columns.  From dataset, we can observe that except `id` column, all the other columns play a significant role in final sales of videogames. So it can be dropped.<jupyter_code>input = input.drop(columns=['ID'])
train, test = train_test_split(input, test_size=0.2, random_state=42, shuffle=True)<jupyter_output><empty_output><jupyter_text>Descriptive Statistics<jupyter_code>train.shape, test.shape
train.nunique()
#If you are seeing the output below for the first time visit this link
#to understand what the values in each of this rows(mean, std, min, max) actually 
#are:- https://www.w3resource.com/pandas/dataframe/dataframe-describe.php
train.describe()<jupyter_output><empty_output><jupyter_text>From above table, my first insight is I can create bar charts of **console, year**, **category** and **ratings** columns easily. For other columns I might have to go for some other visual representation since the the number of unique values is high. *   From **SalesInMillions** column we can see that average sales have been around 2 million and max sales have reached a mark of about 84 million🤩 and min sales were around just 1500😔.*   From **year** column we can see that data covers sales from the year 1997 to 2019*   **Critic Points** range from 0.5 to 23.25 while **user points** range from 0.0003 to 2.32. We might need to noramlise this values on same scale else critic points will have higher impact than user points on final prediction although in reality both of them should have equal importance.  EDA I am first opting for auto EDA packages like pandas-profiling for generating visualisations and there corresponding reports.<jupyter_code>!pip install https://github.com/pandas-profiling/pandas-profiling/archive/master.zip
from pandas_profiling import ProfileReport
report = ProfileReport(train, title="Report", html={'style': {'full_width':True}}, explorative=True, missing_diagrams={'bar': True})
report.to_notebook_iframe()
#Save the report in file
report.to_file("pandas_profiling_report.html")<jupyter_output><empty_output><jupyter_text>From the above reports we can gain following insights:-   *   Console column graph:         The sales of **PS2** were the highest in the data set*   Years Column graph:     The sales were highest between the period **2005-2010**. *   Game category column graph:        **Action** category games are most popular Now let's compare individual columns with target(SalesInMillions) column to gain a few more insights into the data.<jupyter_code>#Sales of games that happened corresponding to each console. 
df = pd.DataFrame(train.groupby(['CONSOLE']).agg({'SalesInMillions': 'sum'}))
df.plot.bar(figsize=(12, 6))<jupyter_output><empty_output><jupyter_text>**💡Insight**:  From the above graph we can see that sales were highest for PS3 platform followed by Xbox360<jupyter_code>df = pd.DataFrame(train.groupby(['YEAR']).agg({'SalesInMillions': 'sum'}))
df.plot.bar(figsize=(12, 6))<jupyter_output><empty_output><jupyter_text>**💡Insight**:  From the above graph we can see that sales were highest in the year 2010<jupyter_code>df = pd.DataFrame(train.groupby(['CATEGORY']).agg({'SalesInMillions': 'sum'}))
df.plot.bar(figsize=(12, 6))<jupyter_output><empty_output><jupyter_text>**💡Insight**:  From the above graph we can see that sales were highest for action genre  Model training<jupyter_code>!pip install catboost
import catboost as cat
cat_feat = ['CONSOLE','CATEGORY', 'PUBLISHER', 'RATING']
features = list(set(train.columns)-set(['SalesInMillions']))
target = 'SalesInMillions'
model = cat.CatBoostRegressor(random_state=100,cat_features=cat_feat,verbose=0)
model.fit(train[features],train[target])<jupyter_output><empty_output><jupyter_text>Model Accuracy<jupyter_code>y_true= pd.DataFrame(data=test[target], columns=['SalesInMillions'])
test_temp = test.drop(columns=[target])
y_pred = model.predict(test_temp[features])
from sklearn.metrics import mean_squared_error
from math import sqrt

rmse = sqrt(mean_squared_error(y_true, y_pred))
print(rmse)
import pickle
filename = 'finalized_model.sav'
pickle.dump(model, open(filename, 'wb'))
loaded_model = pickle.load(open(filename, 'rb'))
test_temp[features].head(1)
loaded_model.predict(test_temp[features].head(1))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09375, 'fraction_numerical': 0.012007389162561576, 'mean_word_length': 6.251116071428571, 'pattern_counts': {'":': 0, '<': 45, '<?xml version=': 0, '>': 45, 'https://': 3, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '164423', 'license': 'MIT', 'n_tokens_mistral': 1963, 'n_tokens_neox': 1851, 'n_words': 695, 'path': 'Video_Game_Sales_prediction.ipynb', 'repo_name': 'saurabhnative/video-game-sales-prediction-jupyter-notebook', 'size': '6496'}