<jupyter_start><jupyter_text>IndieP: Predicting the Success of Indie Games on Steam  Part 3: Data Exploration  So. We've got a lot of data. We've got titles, genres, release dates, availability of external media sources, available platforms and languages, price and release date. Then we've got positive and negative ratings, overall copies owned, number of people playing the game at the same time, and statistics on how long people have spent playing each game. What do we do with it? How do we use all of this information, and actually ask and discover answers to the question "What makes an indie game successful?"  **While Machine Learning is as the forefront of essential Data Science methodologies, there's a lot one can do to understand their data without (or before) applying ML techniques.** In Part 2 of this project, we cleaned and investigated each of the features in our dataset individually, but one question that we could as is: "How do the features correlate to each other?" For example, how does the price of the game correlate to features such as positive and negative ratings, number of owners, or playtime? One could even ask how features such as the number of languages, platforms, or achievements correlate to the price. Are games with more of these attributes typically more expensive? Are there categories or genres that are more expensive? We could further investigate questions such as: is there a certain time of the year when more successful games are released?<jupyter_code>import csv
import os

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import missingno
import seaborn as sns
from ast import literal_eval
import re

pd.set_option("max_columns", 1000)
data_full = pd.read_csv('./data/indie_games_steam.csv')
data_full.head(2)<jupyter_output><empty_output><jupyter_text>3.1 Additional Feature Cleaning In Part 2, we created a clean dataset of indie games including all of the features that may be in some way correlated to the success of a game. However, in order to prepare our data for input to Machine Learning algorithms and feature selection algorithms such as Principle Component Analysis, we need to take care of a few more things.1. Ensure all numerical data is provides as float or int 2. Bin continuous numerical data where appropriate3. Decide how to deal with outliers Additionally, for the `release_date` feature, we intentionally left in missing data because we didn't want to remove thousands of rows that were otherwise useful. To account for this missing data, we will create a subset of our dataset to be used for analyzing the effect of the release date on other features, which has the missing rows removed.<jupyter_code>data_full.dtypes
data_sample = data_full.sample(frac=0.1)
len(data_sample)<jupyter_output><empty_output><jupyter_text>3.2 Feature Engineering In addition to making sure our features are appropriately cleaned to be useful, we also can use human intuition to create new features out of existing ones so that the data can be used more effectively. Some examples of features we may want to create:1. `ratings_ratio`: ratio of positive to negative ratings - if greater than 1, the game has more positive than negative ratings2. `ratings_net_pos`: binomial - 1 if `ratings_ratio` is greater than 1, 0 otherwise.3. `total_ratings`: total number of positive and negative ratings4. `n_dev_games`: total number of games created by the developer of the game5. `n_platforms`: number of platforms on which the game is available6. `is_single_player`: 1 if has single-player option, 0 otherwise7. `is_multi_player`: 1 if has multi-player option, 0 otherwise8. `n_genres`: number of genres listed9. `n_categories`: number of categories listed10. `n_tags`: number of tags listed  3.3 Investigating Feature Correlations  Bi-Viariate Feature Analysis  Correlation Matrices  3.4 Feature Encoding   3.5 Feature Importance  Principal Component Analysis<jupyter_code># Interaction between pairs of features.
#sns.pairplot(data_sample[['n_languages', 'achievements', 'ccu']], 
#             hue="ccu", 
#             diag_kind="kde",
#             height=4);<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.050924813836175836, 'fraction_numerical': 0.010329089598846986, 'mean_word_length': 4.99136690647482, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '60174', 'license': 'MIT', 'n_tokens_mistral': 1078, 'n_tokens_neox': 1014, 'n_words': 590, 'path': 'IndieP_Data_Exploration.ipynb', 'repo_name': 'argwood/IndieP', 'size': '4163'}