<jupyter_start><jupyter_text>I moved the player data logic into a module, let's make sure it's still working. I also changed the victory point margin calculation to be how many points above the average of all players a user scored. The meaning is a bit less intuitive but I think it will have better statistical properties since there won't be the cluster around 0.<jupyter_code>%load_ext autoreload
%autoreload 2
import pandas as pd
import seaborn as sns
from terra_mystica_models.features import parse_interim, player_data
pd.options.display.max_rows = 999
all_games_df = parse_interim.interim_full_pipe()
players = player_data.TerraMysticaPlayers(all_games_df)<jupyter_output><empty_output><jupyter_text>Take a look at the player stats we compute<jupyter_code>players.player_df<jupyter_output><empty_output><jupyter_text>Quick check, the game dataframe should have been modified to replace anyone not in the above dataframe with "Other Terra Mystica User"<jupyter_code>players.game_df["player_1_user"]<jupyter_output><empty_output><jupyter_text>VP margin distributionI changed the logic for this to be deviation from mean rather than deviation from minimum, let's see what it looks like<jupyter_code>vp_margin_cols = [col for col in all_games_df.columns if col.endswith("vp_margin")]
all_vp_margins = (
    all_games_df
    .reset_index()
    .melt(value_vars=vp_margin_cols)
    .dropna()
    ["value"]
)
sns.distplot(all_vp_margins);
all_vp_margins.describe()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.06760374832663989, 'fraction_numerical': 0.004016064257028112, 'mean_word_length': 6.666666666666667, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '153657', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 492, 'n_tokens_neox': 472, 'n_words': 150, 'path': 'notebooks/09_ip_refactor_test.ipynb', 'repo_name': 'ianepreston/terra-mystica-models', 'size': '1494'}