<jupyter_start><jupyter_text>Effect Size Analysis  1.  IntroductionDefine with hypothesis tests what kind of difference in effect we want to prove- d-cohen between ms and hc- d-cohen time of the day, morning vs afternoon- d-cohen practice effect, do the people learn after certain time? k weeks? x number of tests later? In statistics, an effect size is a quantitative measure of the strength of a phenomenon.[1] Examples of effect sizes are the correlation between two variables, the regression coefficient in a regression, the mean difference, or even the risk with which something happens, such as how many people survive after a heart attack for every one person that does not survive. For each type of effect size, a larger absolute value always indicates a stronger effect [here](https://en.wikipedia.org/wiki/Effect_size)<jupyter_code>import sys
import pandas as pd
import numpy as np #si
import matplotlib as mpl
import matplotlib.pyplot as plt
import matplotlib.dates as mdates
from datetime import datetime, timedelta
from scipy import stats #usamos
from math import *
from dateutil.parser import parse as parse_datetime
import warnings
warnings.filterwarnings('ignore')
%matplotlib inline
from ggplot import *
from math import *
!pwd
sys.path.insert(0, '/Users/pedrohserrano/neuroscience/utils') #internal packages
import buildms as ms 
import statsms as stms
color_ms = '#386cb0' #blue, This is the color chosen for patients with Multiple Sclerosis
color_hc = 'red'#This is the color chosen for health control participants<jupyter_output><empty_output><jupyter_text>--- 1.1 Datasets<jupyter_code>!python ~/neuroscience/utils/create_dataset.py ~/neuroscience/data/processed/dataset.csv
df_measures = pd.read_csv('../data/interim/df_measures.csv', encoding="utf-8")
df_measures_users = pd.read_csv('../data/interim/df_measures_users.csv', encoding="utf-8")
df_measures_users.head()<jupyter_output><empty_output><jupyter_text>2. Methods  Cohen's effect sizeThere is one other common way to express the difference between distributions.  Cohen's $d$ is the difference in means, standardized by dividing by the standard deviation.  Here's the math notation:$$ d = \frac{\bar{x}_1 - \bar{x}_2} s $$where $s$ is the pooled standard deviation:$$s = \sqrt{\frac{(n_1-1)s^2_1 + (n_2-1)s^2_2}{n_1+n_2 - 2}}$$<jupyter_code>score_variable = 'correct.answers'<jupyter_output><empty_output><jupyter_text>2.1 Difference of Groups Split the patients MS (Multiple Sclerosis) and HC (Health Control)<jupyter_code>summary_ms = df_measures_users[df_measures_users['ms']==1][score_variable].describe()
print('Average Score of MS Group: {} (SD {})'.format(
    round(summary_ms.loc['mean'],2), round(summary_ms.loc['std'],2)))
summary_hc = df_measures_users[df_measures_users['ms']==0][score_variable].describe()
print('Average Score of HC Group: {} (SD {})'.format(
    round(summary_hc.loc['mean'],2), round(summary_hc.loc['std'],2)))  
cohens_d = stms.dCohen(df_measures_users[df_measures_users['ms']==1][score_variable],
                       df_measures_users[df_measures_users['ms']==0][score_variable]).effect_size()
print('Cohen`s d-value: {}'.format(round(cohens_d,3)))<jupyter_output>Cohen`s d-value: 0.837<jupyter_text>|Study|Difference Value|Benchmark||---|---|---||Benedict 2016 | $d=1.11$ | Large (Cohen, 1988)||**Study 2017**| $d=0.837$ |Large (Cohen, 1988)| There's a significative difference between the scores performed on MS against HC, we are able to say that those are groups with different characteristics What d-Cohen means is that if there was no difference between the two groups, that is, they are the same, then the probability of guessing which of the two groups comes from just seeing their scores would be that of a flown, Or 50%, then the bigger the d is made, the clearer it is to see the difference of the scores just guessing  2.2 Difference on Time of the Day Is there any difference if the participants respond in the morning or in the afternoon?<jupyter_code>plt.figure(figsize=[15, 5])

times = pd.Series(df_measures[df_measures['ms']==1]['timestamp']).map(
    lambda x: '{:02}:00'.format(parse_datetime(str(x)).hour))
time_counts = times.value_counts().sort_index()
relative = time_counts/time_counts.sum()
relative.plot(kind='bar', width=0.8, alpha=0.7, color=color_ms, label='MS')

times = pd.Series(df_measures[df_measures['ms']==0]['timestamp']).map(
    lambda x: '{:02}:00'.format(parse_datetime(str(x)).hour))
time_counts = times.value_counts().sort_index()
relative = time_counts/time_counts.sum()
relative.plot(kind='bar', width=0.8, alpha=0.5, color=color_hc, label='HC')

plt.xlabel('Times on the day')
plt.ylabel('Frequency')
plt.legend()
plt.show()
# This is the tradeoff to decide morning and afternoon
print('The half percent of the distribution, median: {}'.format(df_measures['hour'].median()))<jupyter_output>The half percent of the distribution, median: 15.0<jupyter_text>Split two groups, Morning and Afternoon of MS and HC subgroup<jupyter_code>df_ms = df_measures[df_measures['ms']==1]
df_hc = df_measures[df_measures['ms']==0]
m_ms = df_ms[df_ms['hour'] < df_ms['hour'].median()]
a_ms = df_ms[df_ms['hour'] >= df_ms['hour'].median()]
m_hc = df_hc[df_hc['hour'] < df_hc['hour'].median()]
a_hc = df_hc[df_hc['hour'] >= df_hc['hour'].median()]
d1 = stms.dCohen(m_ms[score_variable], a_ms[score_variable]).effect_size()
d2 = stms.dCohen(m_hc[score_variable], a_hc[score_variable]).effect_size()
print('Cohen´s d-value: \n MS: {} \n HC: {}'.format(round(d1,3),round(d2,3)))
from scipy import stats
plt.figure(figsize=(15, 5))

norm1 = stats.norm(m_ms[score_variable].mean(), m_ms[score_variable].std())
xs, ys = stms.MSstats(norm1).eval_pdf()
plt.plot(xs, ys, label='MS Morning', linewidth=3, color=color_ms, alpha=0.3)
plt.vlines(x=m_ms[score_variable].mean(),ymin=0,ymax=max(ys), color=color_ms, linestyles='--')

norm2 = stats.norm(a_ms[score_variable].mean(), a_ms[score_variable].std())
xs, ys = stms.MSstats(norm2).eval_pdf()
plt.plot(xs, ys, label='MS Afternoon', linewidth=2, color=color_ms)
plt.vlines(x=a_ms[score_variable].mean(),ymin=0,ymax=max(ys), color=color_ms, linestyles='--')

norm3 = stats.norm(m_hc[score_variable].mean(), m_hc[score_variable].std())
xs, ys = stms.MSstats(norm3).eval_pdf()
plt.plot(xs, ys, label='HC Morning', linewidth=3, color=color_hc, alpha=0.3)
plt.vlines(x=m_hc[score_variable].mean(),ymin=0,ymax=max(ys), color=color_hc, linestyles='--')

norm4 = stats.norm(a_hc[score_variable].mean(), a_hc[score_variable].std())
xs, ys = stms.MSstats(norm4).eval_pdf()
plt.plot(xs, ys, label='HC Afternoon', linewidth=2, color=color_hc)
plt.vlines(x=a_hc[score_variable].mean(),ymin=0,ymax=max(ys), color=color_hc, linestyles='--')


plt.xlabel('Scores Distribution'); plt.legend(loc='best'); plt.show()
print('MS Morning: {} (SD {})'.format(round(m_ms[score_variable].mean(),2), round(m_ms[score_variable].std(),2)))
print('MS Afternoon: {} (SD {})'.format(round(a_ms[score_variable].mean(),2), round(a_ms[score_variable].std(),2)))
print('HC Morning: {} (SD {})'.format(round(m_hc[score_variable].mean(),2), round(m_hc[score_variable].std(),2)))
print('HC Afternoon: {} (SD {})'.format(round(a_hc[score_variable].mean(),2), round(a_hc[score_variable].std(),2)))<jupyter_output><empty_output><jupyter_text>There's not significance difference between means, so it means that a person is going to score alike no matter the moment on the day. But in the other hand is clearly to see that respond in the morning for MS group is presumably less variance  F - test Variance Difference $$F={\frac  {S_{X}^{2}}{S_{Y}^{2}}}$$Has an F-distribution with n − 1 and m − 1 degrees of freedom if the null hypothesis of equality of variances is true. Otherwise it has a non-central F-distribution. The null hypothesis is rejected if F is either too large or too small.<jupyter_code>F_ms = m_ms[score_variable].var() / a_ms[score_variable].var()
degrees1ms = len(m_ms[score_variable]) -1
degrees2ms = len(a_ms[score_variable]) -1
print('Difference in Variance for MS: {}%  p-value: {}'.format(
    round(F_ms*100,2), stats.f.cdf(F_ms, degrees1ms, degrees2ms)))
#F_hc = m_hc[score_variable].var() / a_hc[score_variable].var()
F_hc = a_hc[score_variable].var() / m_hc[score_variable].var()
degrees1hc = len(m_hc[score_variable]) -1
degrees2hc = len(a_hc[score_variable]) -1
print('Difference in Variance for HC: {}%  p-value: {}'.format(
    round(F_hc*100,2), stats.f.cdf(F_hc, degrees1hc, degrees2hc)))<jupyter_output>Difference in Variance for HC: 97.0%  p-value: 0.4664223119264743
{'fraction_non_alphanumeric': 0.1485901485901486, 'fraction_numerical': 0.020358020358020357, 'mean_word_length': 6.564601769911504, 'pattern_counts': {'":': 0, '<': 28, '<?xml version=': 0, '>': 28, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '485959', 'license': 'MIT', 'n_tokens_mistral': 3041, 'n_tokens_neox': 2898, 'n_words': 766, 'path': 'notebooks/Story 2 - Effect Size Analysis.ipynb', 'repo_name': 'pedrohserrano/Neuroscience', 'size': '8547'}