<jupyter_start><jupyter_text>Introduction to Hypothesis Testing  What is Hypothesis?- Proposed explanation for a phenomenon.- A hypothesis is an educated guess about something in the world around you. It should be testable, either by experiment or observation. - Proposed explanation- Objectively testable - Singular - hypothesis- Plural - hypotheses__Examples__- A new medicine you think might work.- A way of teaching you think might be better.- A possible location of new species.  What is a Hypothesis Statement?“If I…(do this to an **independent variable**)….then (this will happen to the **dependent variable**).”__Example__- If I (decrease the amount of water given to herbs) then (the herbs will increase in size).- If I (give patients counseling in addition to medication) then (their overall depression scale will decrease).- If I (give exams at noon instead of 7) then (student test scores will improve).A good hypothesis statement should:- Include an “if” and “then” statement (according to the University of California).- Include both the independent and dependent variables.- Be testable by experiment, survey or other scientifically sound technique.- Be based on information in prior research (either yours or someone else’s).- Have design criteria (for engineering or programming projects).  What is Hypothesis Testing?Hypothesis testing in statistics is a way for you to test the results of a survey or experiment to see if you have meaningful results. You’re basically testing whether your results are valid by figuring out the odds that your results have happened by chance. If your results may have happened by chance, the experiment won’t be repeatable and so has little use.  Null and Alternative Hypothesis- Hypothesis 0 (H0): Assumption of the test holds and is failed to be rejected at some level of significance.- Hypothesis 1 (Ha): Assumption of the test does not hold and is rejected at some level of significance.  Errors in Statistical Tests- **Type I Error:** The incorrect rejection of a true null hypothesis or a false positive.- **Type II Error:** The incorrect failure of rejection of a false null hypothesis or a false negative.![imgaes](../img/errors.png)  Alpha($\alpha$)- $\alpha$ is probability of rejecting H0 when H0 is true. - $\alpha$ = Probability of Type-I error. - Ranges from 0 to 1- **High α is not good**  p-value- If p-value > alpha: Fail to reject the null hypothesis (i.e. not significant result).- If p-value <= alpha: Reject the null hypothesis (i.e. significant result).  Hypothesis Testing Process  - **Step-1:** Null Hypothesis H0    - True until proven false    - Usually posits no relationship- **Step-2:** Select Test    - Pick from vast library    - Know which one to choose- **Step-3:** Significance Level    - Usually 1% or 5%    - What threshold for luck?- **Step-4:** Alternative Hypothesis    - Negation of null hypothesis    - Usually asserts specific relationship    - **Step-5:** Test Statistic    - Convert to p-value    - How likely it was just luck?- **Step-6:** Accept or Reject    - Small p-value? Reject H0     - Small: Below significance level ![images](../img/hypothesis.png)  Common Statistical Tests Variable Distribution Type Tests (Gaussian)- Shapiro-Wilk Test- D’Agostino’s K^2 Test- Anderson-Darling Test Variable Relationship Tests (correlation)- Pearson’s Correlation Coefficient- Spearman’s Rank Correlation- Kendall’s Rank Correlation- Chi-Squared Test Compare Sample Means (parametric)- Student’s t-test- Paired Student’s t-test- Analysis of Variance Test (ANOVA)- Repeated Measures ANOVA Test Compare Sample Means (nonparametric)- Mann-Whitney U Test- Wilcoxon Signed-Rank Test- Kruskal-Wallis H Test- Friedman Test<jupyter_code>import pandas as pd 
import numpy as np 
import seaborn as sns 
import matplotlib.pyplot as plt 
df = pd.read_excel('../data/height_weight.xlsx')
df 
df.dtypes
df.info() 
df['Gender'].value_counts()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07828730681530276, 'fraction_numerical': 0.004813782619711173, 'mean_word_length': 4.264, 'pattern_counts': {'":': 0, '<': 6, '<?xml version=': 0, '>': 6, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '653201', 'license': 'CC0-1.0', 'n_tokens_mistral': 1066, 'n_tokens_neox': 973, 'n_words': 534, 'path': 'hypothesis/notebooks/01-Introduction to Hypothesis Testing.ipynb', 'repo_name': 'scicomforbio/ISCB20.06', 'size': '3947'}