<jupyter_start><jupyter_text>1. one-way ANOVA- model: y ~ x1 (**y is explained by x1 only**)- reference    - https://partrita.github.io/posts/ANOVA-python/<jupyter_code>url = 'https://raw.githubusercontent.com/thomas-haslwanter/statsintro_python/master/ipynb/Data/data_altman/altman_910.txt'
data = pd.read_csv(url, error_bad_lines=False)
data.columns = ['value', 'group']
print(data.head())<jupyter_output>value  group
0    251      1
1    275      1
2    291      1
3    347      1
4    354      1<jupyter_text>- 세 집단 간의 평균차이는 유의미한가?(모집단의 평균이 다른가?)<jupyter_code>ax = data.boxplot(by='group', column='value', return_type='axes')
plt.title('')
plt.tight_layout()
plt.show()<jupyter_output><empty_output><jupyter_text>1) 집단간 분산 vs 집단내 분산<jupyter_code>c = data['group'] == 1
group1 = data[c].value
c = data['group'] == 2
group2 = data[c].value
c = data['group'] == 3
group3 = data[c].value
plt.figure(figsize=(5, 5))
plt.subplot(211)
sns.distplot(data.value)
plt.title('Dist b/w Groups')

plt.subplot(212)
sns.distplot(group1, label='g1')
sns.distplot(group2, label='g2')
sns.distplot(group3, label='g3')
plt.title('Dist within each Group')
plt.legend()

plt.tight_layout()
plt.show()
# 전체 집단 분산
data.std()[0]
# 집단 내 분산
print(data.groupby('group').agg(['std', 'mean']))<jupyter_output>value            
             std        mean
group                       
1      54.679151  327.142857
2      37.121797  256.444444
3      33.756481  278.000000<jupyter_text>2.1) ANOVA(`scipy`)- group1, 2, 3 일부 모집단의 평균은 다름- 그런데 어느 그룹간 다른지는 모름 -> Dunnett test<jupyter_code>f_stat, pval = sp.stats.f_oneway(group1, group2, group3)
f_stat, pval<jupyter_output><empty_output><jupyter_text>2.2) ANOVA(`statsmodels`)<jupyter_code>f = 'value ~ C(group)'
mod = sm.OLS.from_formula(f, data)
r = mod.fit()
print(sm.stats.anova_lm(r))<jupyter_output>df        sum_sq       mean_sq         F    PR(>F)
C(group)   2.0  20015.492063  10007.746032  5.373915  0.014791
Residual  18.0  33521.079365   1862.282187       NaN       NaN<jupyter_text>2.3) Tukey's range test(다중비교)- 어떤 집단 간 평균 차이가 유의미 한가: 그룹1, 2- Dunnett test도 가능- reference     - http://cleverowl.uk/2015/07/01/using-one-way-anova-and-tukeys-test-to-compare-data-sets/    - https://support.minitab.com/ko-kr/minitab/18/help-and-how-to/modeling-statistics/anova/supporting-topics/multiple-comparisons/using-multiple-comparisons-to-assess-differences-in-means/<jupyter_code>from statsmodels.stats.multicomp import pairwise_tukeyhsd
from statsmodels.stats.multicomp import MultiComparison

mc = MultiComparison(data['value'], data['group'])
result = mc.tukeyhsd(alpha=0.05)
print(result)
print(mc.groupsunique)<jupyter_output>Multiple Comparison of Means - Tukey HSD,FWER=0.05
================================================
group1 group2 meandiff   lower    upper   reject
------------------------------------------------
  1      2    -70.6984 -126.1814 -15.2154  True 
  1      3    -49.1429 -113.6083 15.3226  False 
  2      3    21.5556   -39.8529  82.964  False 
------------------------------------------------
[1 2 3]<jupyter_text>2.4) T-test- 위 결과와 동일: group1, 2의 모평균이 다름<jupyter_code>sp.stats.ttest_ind(group1, group3), sp.stats.ttest_ind(group2, group3), sp.stats.ttest_ind(group1, group2)<jupyter_output><empty_output><jupyter_text>---  2. two-way ANOVA by F-statistic- model: y ~ x1, x2(**y is explained by x1, x2( and interaction)**) 1) Sample1- 검사자observer에 따라 태아fetus의 머리크기head-size에 차이가 있는가?- $H0_1$: 검사자x1 간 머리크기y 차이는 없다- $H0_2$: 태아x2 간 머리크y 차이 없다- reference: https://partrita.github.io/posts/ANOVA-python/<jupyter_code>import urllib
file = 'altman_12_6.txt'
base = 'https://raw.githubusercontent.com/thomas-haslwanter/statsintro_python/master/ipynb/Data/data_altman/'
url = base + file
data = np.genfromtxt(urllib.request.urlopen(url), delimiter=',')
df = pd.DataFrame(data, columns=['head_size', 'fetus', 'observer'])
print(df.groupby(['fetus','observer']).agg(list).unstack())
f = 'head_size ~ C(observer) + C(fetus) + C(observer):C(fetus) '
mod = sm.OLS.from_formula(f, df)
res = mod.fit()
print(res.summary2())
print(sm.stats.anova_lm(res))<jupyter_output>df  sum_sq  mean_sq     F    PR(>F)
C(car_type)  2.0  9300.0   4650.0  93.0  0.010638
C(company)   1.0  1350.0   1350.0  27.0  0.035099
Residual     2.0   100.0     50.0   NaN       NaN<jupyter_text>---  2) Sample2- 차종x1/보험회사x2에 따라 보험료- reference: https://rfriend.tistory.com/135<jupyter_code>data = {'com_K': [140, 210, 220],
       'com_M': [100, 180, 200],}
test = pd.DataFrame(data, index=list(map(lambda x: 'car_' + x, list('ABC'))))
print(test)
data = {'y':[140, 100, 210, 180, 220, 200], 'car_type':['A', 'A', 'B', 'B', 'C', 'C'], 'company':['K', 'M', 'K', 'M', 'K', 'M']}
test_modified = pd.DataFrame(data)
print(test_modified)
f = 'y ~ C(car_type) + C(company)'
mod = sm.OLS.from_formula(f, test_modified)
res = mod.fit()
print(res.summary2())
print(sm.stats.anova_lm(res))
# R^2 by ANOVA
(9300 + 1350)/(9300 + 1350 + 100)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1889716240692292, 'fraction_numerical': 0.08231032400885491, 'mean_word_length': 3.715370018975332, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 35, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '643826', 'license': 'MIT', 'n_tokens_mistral': 2416, 'n_tokens_neox': 2260, 'n_words': 387, 'path': '06.Math/Recap/Recap_test_ANOVAegok.ipynb', 'repo_name': 'HenryPaik1/Study', 'size': '4969'}