<jupyter_start><jupyter_text>용어 정의<jupyter_code>#가설설정
# A hypothesis test is a statistical method that uses sample data to evaluate a hypothesis about a population.

1. First, we state a hypothesis about a population. Usually the hypothesis concerns the value of a population parameter. 
2. Before we select a sample, we use the hypothesis to predict the characteristics that the sample should have. 
3. Next, we obtain a random sample from the population. 
4. Finally, we compare the obtained sample data with the prediction that was made from the hypothesis.

## 가설설정 프로세스
1. State the hypothesis. null hypothesis(H0) 

귀무가설 : 독립변수가 종속변수에 어떤 영향을 미치지 않는다는 것 => 레스토랑의 웨이터가 레드 셔츠 입는 것이 팁에 영향이 없다.
The null hypothesis (H0) states that in the general population 
there is no change, no difference, or no relationship.
In the context of an experiment, 
H0 predicts that the independent variable (treatment) 
has no effect on the dependent variable (scores) for the population.

m = 15.8

대안가설 : 어떤 변인이 종속 변수에 효과가 있다는 것 => 레스토랑의 웨이터가 레드 셔츠 입는 것 팁에 영향이 있다. 
The alternative hypothesis (H1) states that there is a change, a difference, 
or a relationship for the general population. 
In the context of an experiment,
H1 predicts that the independent variable (treatment) does have an effect on the dependent variable.
    
m != 15.8 이다.

이 실험에서는 
m > 15.8

directional hypothisis test

2. set the criteria for a decision

a. Sample means that are likely to be obtained if H0 is true; 
that is, sample means that are close to the null hypothesis

b. Sample means that are very unlikely to be obtained if H0 is true;
that is, sample means that are very different from the null hypothesis


The Alpha Level

alpha levels are α = .05 (5%), α = .01 (1%), and α = .001 (0.1%).

The alpha level, or the level of significance, 
is a probability value that is used to define the concept of 
“very unlikely” in a hypothesis test.

The critical region is composed of the extreme sample values that are very unlikely (as defined by the alpha level) to be obtained if the null hypothesis is true. The boundaries for the critical region are determined by the alpha level. 
If sample data fall in the critical region, the null hypothesis is rejected.


3. Collect data and compute sample statistics.

z = sample mean - hypothesized population mean / standard error between M and m

4. Make a decision

1. Thesampledataarelocatedinthecriticalregion.
Bydefinition,asamplevaluein the critical region is very unlikely to occur if the null hypothesis is true. 

2. The sample data are not in the critical region. 
In this case, the sample mean is reasonably close to the population mean specified in the null hypothesis (in the center of the distribution).<jupyter_output><empty_output><jupyter_text>Problems<jupyter_code>1. Identify the four steps of a hypothesis test as presented in this chapter.

1)State the hypothesis.
귀무가설과 대안가설 언급

2)alpha level 설정, 신뢰 구간 설정

3) Collect data and compute sample statistics. 
데이터 수집과 샘플 통계적 계산

4)make decision
결론 결정

2. Define the alpha level and the critical region for a hypothesis test.
독립변수와 종속변수에 대한 귀무가설을 reject하기 위해 그 통계치를 통상적인 수치를 벗어나 의미있는 수가 나온 것을 설정해준다. 
3. Define a Type I error and a Type II error and explain the consequences of each.
가설검증에서 실제효과가 없는데 효과가 있는 것으로 나온것, 실제 효과가 있는데, 없는 것으로 나온것. 가설 설정에 문제
4. If the alpha level is changed from α = .05 to α = .01,
a. What happens to the boundaries for the critical
region?
신뢰구간이 줄어든다.
b. What happens to the probability of a Type I error?
에러 확률은 낮아진다. 
6. Although there is a popular belief that herbal remedies such as Ginkgo biloba and Ginseng may improve learning and memory in healthy adults, these effects are usually not supported by well- controlled research (Persson, Bringlov, Nilsson, and Nyberg, 2004). In a typical study, a researcher
obtains a sample of n = 16 participants and has each person take the herbal supplements every day for
90 days. At the end of the 90 days, each person takes a standardized memory test. For the general popula- tion, scores from the test form a normal distribution with a mean of μ = 50 and a standard deviation of
σ = 12. The sample of research participants had an average of M = 54.
a. Assuming a two-tailed test, state the null hypoth-
esis in a sentence that includes the two variables
being examined.
b. Using the standard 4-step procedure, conduct a
two-tailed hypothesis test with α = .05 to evaluate the effect of the supplements.



from scipy import stats

sample_number = 16 # 샘플수 
population_mean = 50 # 모집단의 평균
standard_deviation = 12 # 표준편차
sample_mean = 54 # 샘플의 평균


result = stats.ttest_1samp(sample_mean, 50)   # 비교집단, 관측치

result
sample_mean - population_mean
## Import 
import numpy as np
from scipy import stats


sample_number = 16 # 샘플수 
population_mean = 50 # 모집단의 평균
standard_deviation = 12 # 표준편차
sample_mean = 54 # 샘플의 평균


## 신뢰구간을 벗어나는지 아닌지 확인 함수     

alpha_level05 = 1.96
alpha_level01 = 2.58

def h_test(sample_mean, population_mean, standard_deviation, sample_number, alpha_level):
    
    
    result = (sample_mean - population_mean)/ (standard_deviation/np.sqrt(sample_number))
                                         
                                               
    if result> alpha_level or result< - alpha_level:
       print("a = .05 신뢰구간에서 귀무가설 reject되고, 가설이 ok")

    else:
       print("귀무가설이 reject 되지 않아 가설이 기각됩니다.")

    return result


##Compute Cohen’s d
def Cohen(sample_mean, population_mean, standard_deviation):
    result = (sample_mean - population_mean) / (standard_deviation)
    
    
    if result<=0.2:
        print("small effect")

    elif result<= 0.5:
        print("medium effect")

    elif result<= 0.8:
        print("Large effect")
    
    return result

  
## 신뢰구간을 벗어나는지 아닌지 확인 함수                                             
h_test(sample_mean, population_mean, standard_deviation, sample_number, alpha_level05)

                                               

Cohen(sample_mean, population_mean, standard_deviation)
함수를 활용해서, 신뢰구간과 cohen's d를 구할 수 있다. 

# ## Import the packages
# import numpy as np
# from scipy import stats


# ## 함수로 만들기

# #Sample Size
# sample_number = 16 
# population_mean = 50 # 모집단의 평균
# standard_deviation = 12 # 표준편차
# sample_mean = [54,54,58,53,52] # 샘플의 평균

# def h_test(sample_mean, population_mean, standard_deviation, sample_number):
    
#     #For unbiased max likelihood estimate we have to divide the var by N-1, and therefore the parameter ddof = 1
#     var_sample_mean = sample_mean.var(ddof=1)
#     var_population_mean = population_mean.var(ddof=1)
    
#     #std deviation
#     std_deviation = np.sqrt((var_sample_mean + var_population_mean)/2)
    
#     ## Calculate the t-statistics
#     t = (a.mean() - b.mean())/(s*np.sqrt(2/N))
    
# ## Define 2 random distributions

# N = 10
# #Gaussian distributed data with mean = 2 and var = 1
# a = np.random.randn(N) + 2
# #Gaussian distributed data with with mean = 0 and var = 1
# b = np.random.randn(N)


# ## Calculate the Standard Deviation
# #Calculate the variance to get the standard deviation

# #For unbiased max likelihood estimate we have to divide the var by N-1, and therefore the parameter ddof = 1
# var_a = a.var(ddof=1)
# var_b = b.var(ddof=1)

# #std deviation
# s = np.sqrt((var_a + var_b)/2)
# s



# ## Calculate the t-statistics
# t = (a.mean() - b.mean())/(s*np.sqrt(2/N))



# ## Compare with the critical t-value
# #Degrees of freedom
# df = 2*N - 2

# #p-value after comparison with the t 
# p = 1 - stats.t.cdf(t,df=df)


# print("t = " + str(t))
# print("p = " + str(2*p))
# ### You can see that after comparing the t statistic with the critical t value (computed internally) we get a good p value of 0.0005 and thus we reject the null hypothesis and thus it proves that the mean of the two distributions are different and statistically significant.


# ## Cross Checking with the internal scipy function
# t2, p2 = stats.ttest_ind(a,b)
# print("t = " + str(t2))
# print("p = " + str(p2))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.06942025379447624, 'fraction_numerical': 0.01953222194575765, 'mean_word_length': 3.3642779587404994, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '265', 'license': 'MIT', 'n_tokens_mistral': 2909, 'n_tokens_neox': 3005, 'n_words': 1116, 'path': '08_OSK.ipynb', 'repo_name': 'seokyeongheo/slow_statistics', 'size': '8038'}