<jupyter_start><jupyter_text>Hypothesis Testing Part II  Error bars Confidence intervals Power  Two-sample t-testLet's consider a more realistic experiment.Load in the data file `two_samples.npy`.This file contains a 2D array with 2 rows, each corresponding to a set of sampled values from different experimental groups.<jupyter_code># load in the data
data = np.load('two_samples.npy')

data.shape

sample_a = data[0]
sample_b = data[1]<jupyter_output><empty_output><jupyter_text>Are these groups statistically different from one another?<jupyter_code># 1. State hypotheses:
    # Null hypothesis: these 2 samples come from the same population
    # Alternative hypothesis: these 2 samples come from different populations
    
# 2. Compute the t-statistic

# 3. Compute the p-value

# 4. Compare p-value to your chosen threshold (alpha)
from scipy.stats import ttest_ind

tstat,pvalue = ttest_ind(sample_a, sample_b)

print(pvalue)

if pvalue < 0.05:
    print('These groups most likely are not from the same population.')
else:
    #print('These groups are most likely from the same population.')
    print('We failed to discover evidence that they come from different populations.')
ttest_ind??<jupyter_output><empty_output><jupyter_text>effect size (again)  Paired t-test Suppose we now learn that each pair of measurements came from a single animal. For example, the first value in each row corresponds to measurements from animal A, with and without some experimental perturbation.How might we modify the way we ask our statistical question?<jupyter_code>from scipy.stats import ttest_rel

ttest_rel<jupyter_output><empty_output><jupyter_text>Non-parametric tests* tests for normality<jupyter_code># what about distribtions? not normal! what to do

# tests for normality

# if not normal:

# an intuitive, manual mann whitney

# wilcoxon
x = [1, 5, 4, 7, 3, 8]
y = [10, 13, 5, 17, 10]

x = np.array(x)
y = np.array(y)

count_y_greater = 0
total_comparisons = 0

for i in range(0,len(y)):
    num_y_greater = np.sum(y[i]>x)
    total_comparisons = total_comparisons + len(x)
    count_y_greater = count_y_greater + num_y_greater
    
print(count_y_greater)
print(total_comparisons)
print(count_y_greater / total_comparisons)
# mann whitney u: considered a nonparametric analog of two-sample t-test
# nonparametric: no assumptions about normality of distributions
from scipy.stats import mannwhitneyu

mstat, pvalue = mannwhitneyu(sample_a, sample_b)

print(pvalue)
from scipy.stats import wilcoxon

wilcoxon?<jupyter_output><empty_output><jupyter_text>Resampling & bootstrapping<jupyter_code>data = np.load('two_samples.npy')

# grab some samples from 2 normal distributions

sample_a = np.random.normal(5, 3, size=1000)
sample_b = np.random.normal(6, 3, size=1000)

tstat,pvalue = ttest_ind(sample_a, sample_b)

pooled = np.append(sample_a, sample_b)

#resampled_a = np.random.choice(pooled, size=len(sample_a), replace=False)
#resampled_b = np.random.choice(pooled, size=len(sample_b), replace=False)

np.random.shuffle(pooled)

resampled_a = pooled[0:len(sample_a)]
resampled_b = pooled[len(sample_a):]

ttest_ind(resampled_a, resampled_b)
x = 5

x = x + 1
# identical to:
x += 1
x = np.array([1,2,3,4,5,6,7,8,9])

print(x)
np.random.shuffle(x)
# note that this is NOT:
# x = np.random.shuffle(x)
print(x)<jupyter_output>[1 2 3 4 5 6 7 8 9]
[9 8 1 3 6 2 7 5 4]
{'fraction_non_alphanumeric': 0.09501187648456057, 'fraction_numerical': 0.022268408551068885, 'mean_word_length': 5.308988764044944, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '252481', 'license': 'Apache-2.0', 'n_tokens_mistral': 1250, 'n_tokens_neox': 1166, 'n_words': 360, 'path': 'Week2_Math/Day3_Stats2/Hypothesis testing part II.ipynb', 'repo_name': 'bensondaled/PNI_Programming_Statistics_Bootcamp_2018', 'size': '3368'}