<jupyter_start><jupyter_text>Elmentary Statistics (STAT 201) <NAME>, PhD candidate in Statistics  What is Statistics?   Let data tell a story -> The art and science of learning from data* [One story:  searching for a fisherman lost at sea using Statistics method](https://stats.stackexchange.com/questions/119952/how-to-apply-bayes-theorem-to-the-search-for-a-fisherman-lost-at-sea)    * There are two fishermen on a boat in the middle of the night. While one is asleep, the other falls into the ocean. The boat continues to troll along on autopilot all through the night until the first guy finally wakes up and notifies the Coast Guard. * A long version<jupyter_code>from IPython.display import IFrame
IFrame('https://www.nytimes.com/2014/01/05/magazine/a-speck-in-the-sea.html?smid=pl-share', width=900,height=500)<jupyter_output><empty_output><jupyter_text>Basic concepts in Statistics  * **Individuals (subjects)**: the entities that we measure in a study. Individuals are often    people, but they don’t have to be. * **Variable**: any characteristic we measure on the individuals. The measurements are called    data or observations.    - Values of the same variable may differ among individuals. We call this difference variability. * **Population**: the collection of all individuals that we are interested in.* **Sample**: a subset of the population which we have (or plan to have) data.    - The number of individuals in the sample is called the **sample size**.   * **Parameter**: a numerical summary of the population, such as population mean or population    proportion.    - Unless every individual in the population is measured, the population parameter is usually unknown.* **Statistic**: a numerical summary of the sample, such as sample mean or sample proportion.    - Although always known, a key feature of a sample statistic is that it is random. The value of a sample statistic varies from sample to sample.  Population vs sample, parameter vs statistic   Three main components of statistics:* **Design of study**: stating the goal and/or question of interest and planning how to obtain    data that will address them.* **Descriptive statistics**: summarizing and analyzing the data that are obtained.* **Inferential statistics**: making decisions and predictions based on the data for answering    the statistical question.  Design of study - Sampling Method      Scheme    Defination    Pros    Cons        Census    Measureing every individual in the population    Comprehensive    non-feasible        Judgment Sample    A sample that an expert thinks to be representative          Potentially biased        Convenience Sample    A sample that is easy to access         Potentially biased        Volunteer Sample    A sample where individuals choose to/not to participate         Potentially biased        Systematic Sample    Individuals are sampled using systematic methods          Potentially biased        Simple Random Sample(SRS)    Every individual in the population is equally likely to be included in the sample    Represents the population if the sample size is large enough         Data Visualization (Graphical Display)  Categorical Data* **Categorical (qualitative)** : each observation belongs to one of a set of distinct categories.    - Major, home state, nationality, etc.    - (There are 10 types of people, those who understand binary system and those who don’t.)  Quantitative Data* **Quantitative**: each observation takes on a numerical value that represent a certain magni-    tude of the variable.    - Discrete quantitative: possible values of the variable form a set of separate numbers        such as 0, 1, 2, . . ..    - Continuous quantitative: possible values of the variable form an interval.  How to visualize categorical data* **Distribution** The distribution of a variable describes how the observations fall (are dis-    tributed) across the range of possible values.  Frequency table* The distribution of a categorical variable can be summarized in a **frequency table**, which displays all possible categories, together with the frequencies or relative frequencies of each category.    - **Frequency**: the number of observations (or occurrences) falling in one category.    - **Relative frequency**: the number of observations falling in one category, relative to the        sample size.    * We can obtain the relative frequency of a category by computing its **sample proportion** or percentage.* **Sample proportion**: the number of observations falling in one category divided by the total    number of observations. In other words, sample proportion is the frequency of one category    divided by the sample size. We often denote sample proportion by $\hat{p}$ (p-hat).$$ \hat{p} = \frac{frequency}{sample \ size}  $$ * An example: number of car accidents in each state in 2017<jupyter_code>IFrame("https://www.iihs.org/iihs/topics/t/general-statistics/fatalityfacts/state-by-state-overview", width = 900, height = 600)<jupyter_output><empty_output><jupyter_text>Pie chart* The earliest pie chart ever created dates back to 1801 (From Wikipedia):  Bar graph* Huamn losses in WWII (From Wikipedia)  How to visualize quantitative data  Dot plot* From Wikipedia  Stem and Leaf plot<jupyter_code>IFrame("https://www.mathsisfun.com/data/stem-leaf-plots.html", width=900, height=400)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.06619561175158051, 'fraction_numerical': 0.008925251022685012, 'mean_word_length': 4.157238734419942, 'pattern_counts': {'":': 0, '<': 14, '<?xml version=': 0, '>': 15, 'https://': 4, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '117431', 'license': 'CC0-1.0', 'n_tokens_mistral': 1325, 'n_tokens_neox': 1232, 'n_words': 674, 'path': 'teaching/Chapter_1_2.ipynb', 'repo_name': 'statcao/statcao.github.io', 'size': '5385'}