<jupyter_start><jupyter_text>Premium vs. basic usersLooking at differences between preimum and basic users. Import packages<jupyter_code>import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import sqlalchemy as sal
import seaborn as sns
from plotting_utils import target_corr_plot
sns.set()

# For calculating standard error in mean
from scipy.stats import sem

import warnings
warnings.filterwarnings("ignore")<jupyter_output><empty_output><jupyter_text>Import dataImport the profile data from the SQL database<jupyter_code># Create SQL engine
engine = sal.create_engine('mysql://oskari:pw@localhost/chess_crawler', pool_recycle=3600, echo = False)

# Load data from the table 'profiles'
with engine.connect() as conn:
    profiles_df = pd.read_sql_table('profiles', conn)
    
# Remove duplicates of the same user
profiles_df.drop_duplicates(subset = 'id', inplace = True)

# Add log versions for numbers of games
n_games_columns = list(profiles_df.columns[[("n_games" in column) for column in profiles_df.columns]])
profiles_df = profiles_df.merge(np.log10(1+profiles_df[n_games_columns]).rename(columns = lambda x: x + "_log"), left_index = True, right_index = True)

display(profiles_df)<jupyter_output><empty_output><jupyter_text>BasicsSummarizing some basic data about premium and basic users<jupyter_code>basic_stats = profiles_df.groupby(by = 'status').agg(count = ('rapid_rating', len), mean_rapid = ('rapid_rating', np.mean), error = ('rapid_rating', sem))
basic_stats<jupyter_output><empty_output><jupyter_text>Rapid ratingDoes getting a premium account make you better at chess? Testing here if preimum users have a higher rapid rating on average in my sample. This of course doesn't necessarily mean that getting premium makes you better - could just be that people who are better at chess and presumably spend more time on it find premium more worthwhile.Start by making histogram of rapid ratings for basic and premium users:<jupyter_code>axes = sns.histplot(profiles_df[profiles_df.status.isin(['premium', 'basic', 'closed:fair_play_violations'])], x = 'rapid_rating', hue = 'status', stat = 'percent', common_norm = False)
axes.figure.set_size_inches((16,9))

# Also print the average ratings
## Basic users
basic_rapid = profiles_df[profiles_df.status == 'basic'].rapid_rating.mean()
basic_rapid_uc = sem(profiles_df[profiles_df.status == 'basic'].rapid_rating)
print(f"Average rapid rating for basic users {basic_rapid:.1f}+/-{basic_rapid_uc:.1f}")

## Premium users
premium_rapid = profiles_df[profiles_df.status == 'premium'].rapid_rating.mean()
premium_rapid_uc = sem(profiles_df[profiles_df.status == 'premium'].rapid_rating)
print(f"Average rapid rating for premium users {premium_rapid:.1f}+/-{premium_rapid_uc:.1f}")

# Calculate difference in ratings
rating_difference = premium_rapid - basic_rapid
rating_difference_uc = np.sqrt(premium_rapid_uc**2 + basic_rapid_uc**2)
print(f"Rating difference between basic and premium {rating_difference:.1f}+/-{rating_difference_uc:.1f}")<jupyter_output>Average rapid rating for basic users 1610.3+/-9.5
Average rapid rating for premium users 1684.2+/-21.8
Rating difference between basic and premium 73.9+/-23.8<jupyter_text>There isn't a very noticeable difference between the premium and basic users in the histograms, although the mean is somewhat higher for premium users as aI would have expected. By far the highest average rating is for accounts that have been closed for cheating.Now let's do some formal hypothesis testing. My hypothesis is that premium users are on average more highly rated than basic users. The corresponding null hypothesis is then that the average rating of premium users is the same or lower than that of basic users. The data is not very close to a normal distribution, so to test the hypothesis I will use a premutation test: - Merge the data for basic and premium users- Randomly draw resamples of the corresponding sizes for basic and premium users- Calculate the mean rapid ratings for the new "basic" and "premium" user distributions (these distributions are drawn from the merged data), and calculate the difference- Repeat this R times to get a distribution of the rating differences- Calculate probability of observed difference in the real data being due to chance using the obtained distribution<jupyter_code># Get the ratings of basic and premium players
merged_ratings = profiles_df[profiles_df.status.isin(['premium', 'basic'])].rapid_rating.reset_index(drop = True)

# Figure out the numbers of basic and premium users so can resample correct number
n_basic = basic_stats.loc['basic', 'count']
n_premium = basic_stats.loc['premium', 'count']

# Define number of trials for permutation test
R = 2000

# Define a function that performs the resampling and calculates the difference in the mean
def resample_func(data, n1: int, n2: int) -> float:
    """
    Resamples data by choosing n1 items for distribution 1 and n2 for distribution 2
    """
    # Get indices for resampling of data
    idx1 = set(np.random.choice(range(len(data)), size = n1, replace = False))
    idx2 = set(range(len(data))) - idx1
    
    return data.loc[idx1].mean() - data.loc[idx2].mean()

# Loop over R resamplings and calculate diffence in mean
differences = np.zeros(R)
for i in range(R):
    differences[i] = resample_func(merged_ratings, n_premium, n_basic)<jupyter_output><empty_output><jupyter_text>Now plot the distribution of differences:<jupyter_code>axes = sns.histplot(differences)
axes.figure.set_size_inches((16,9))
axes.axes.set_xlabel("Difference in means")
axes.axes.axvline(rating_difference, ls = '--', c = 'k', linewidth = 3)

# Print the probability of the resampled differences being above the observed difference
p_value = np.mean(rating_difference < differences)
print(f"p value: {p_value:.3f}")<jupyter_output>p value: 0.001
{'fraction_non_alphanumeric': 0.08892302440689537, 'fraction_numerical': 0.01092336576207544, 'mean_word_length': 5.877934272300469, 'pattern_counts': {'":': 0, '<': 24, '<?xml version=': 0, '>': 24, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '461391', 'license': 'MIT', 'n_tokens_mistral': 1831, 'n_tokens_neox': 1716, 'n_words': 618, 'path': 'data-analysis/Premium vs basic.ipynb', 'repo_name': 'otimgren/chessdotcom-crawler', 'size': '5859'}