<jupyter_start><jupyter_text>Intro to Probability Written by: <NAME> & <NAME>Probability is an exceptionally important foundational topic in statistics and computer science. Probability tries to answer: _given knowledge about the population, what can I conclude about a random sample?_ Here, we will not be going too deeply into probability (that will be the role of Stat 134, Stat 140, or CS 70). Instead, we will be giving you a taste of probability for those who are planning to dive into these topics in the future. If you aren't already stoked, here are just a few of the countless applications of probability:+ Machine learning+ Finance+ Robotics+ Hardware engineering+ Gambling strategies+ Polling predictions For the rest of this notebook, we use capital letters for random events, or random variables. Therefore, we define $\mathbb{P}(X=x)$ as the probability of the random event $X$ is actually $x$. To make things more concrete, let $D$ be the number that shows up after we roll a die. Then $\mathbb{P}(D=5)$ is the probability that we roll a $5$. You could get more creative with this notation:+ $\mathbb{P}(X>x)$ is the probability that event $X$ is greater than $x$.+ $\mathbb{P}(X\neq x)$ is the probability that event $X$ is not $x$.+ $\mathbb{P}(X = x, Y \geq y)$ is the probability that event $X$ is $x$ AND event $Y$ is greater than or equal to $y$.+ $\mathbb{P}(X = cloudy)$ is the probability that it is cloudy. Yep, it doesn't even have to be a math expression, just needs to be an event!+ $\mathbb{P}(X = x | Y = y)$ is the probability that event $X$ is $x$, GIVEN that event $Y$ is $y$. Known as a conditional probability, this is arguably the be most confusing to understand, so we dedicate the next section to this.  Conditional ProbabilityLet's start with a quick example. Say I have a fair coin and a biased coin that shows heads 75% of the time. Since they look identical, you choose one with equal probability and flip it. If it's heads, you get 10 dollars, and if it's tails, you get nothing. If I want to know your chance of winning 10 dollars, I would need to calculate $\mathbb{P}(W=10)$ where $W$ is your winnings. Here, I have no clue what coin you chose, so we need to calculate everything (chance you pick the fair coin and win plus the chance you pick the biased coin and win). Now suppose I know for a fact you chose the fair coin and I want to calculate your chance of winning 10 dollars. Then, letting $C$ be the coin you chose, I would need to find $\mathbb{P}(W=10|C=fair)$ which is just $\frac{1}{2}$. Do you see the difference between $\mathbb{P}(W=10)$ and $\mathbb{P}(W=10|C=fair)$?In summary, conditional probability is the probability of something happening given some additional information.  Multiplication RuleTo calculate the probability of 2 events happening, we can decompose $\mathbb{P}(A, B)$:$\mathbb{P}(A, B) = \mathbb{P}(A| B) \mathbb{P}(B) = \mathbb{P}(B|A)\mathbb{P}(A)$  IndependenceMathematically, we say is two events $A$ and $B$ are independent if and only if:$\mathbb{P}(A, B) = \mathbb{P}(A) \mathbb{P}(B)$Intuitively, this means knowing $A$ or $B$ will not tell you anything about the other. If I flip a coin two times, the first flip and second flip are independent of each other. This also means:$\mathbb{P}(A | B) = \mathbb{P}(A)$ and $\mathbb{P}(B | A) = \mathbb{P}(B)$  Bayes' TheoremThis is one of the most crucial equations statistics, so commit it to memory! It's also derivable from the multiplication rule ;)$\mathbb{P}(A| B) =  \frac{\mathbb{P}(A, B)}{\mathbb{P}(B)} = \frac{\mathbb{P}(B|A) \mathbb{P}(A)}{\mathbb{P}(B)}$Notice how it essentially flips $A|B$ on the left to $B|A$ on the right. This allows us to flexibility to calculate whichever one is easier and plug it into the appropriate Bayes' equation.  Sample vs. PopulationA sample consists of data drawn from a population. Some ways to sample are below:**Simple Random Sample:** All items are randomly divided into smaller groups. Then, an item is randomly chosen from each group. **Convenience Sample:** Each item is chosen based on if it is immediately available from a population.  Quick Check1) I have 2 dice. I roll one and observe that I got $N$. Then, I roll the second one $N$ times, and let $M$ be the maximum of these $N$ rolls. Are $M$ and $N$ independent?2) Assume the chance of giving birth to a boy or girl are each $\frac{1}{2}$. Given a family has 3 girls already, what is the probability the next one will be a girl? 3) I have 3 cards. The first is blue on both sides. The second is gold on both sides. The third is blue on one side and gold on the other. I pick one at random and show you one side of that chosen card. Given that you see that side is gold, what is the probability that the other side is blue?  ExpectationExpectation of $X$, or $\mathbb{E}(X)$, is the average outcome of a random event. It is defined as:$\mathbb{E}(X) = \sum_{\text{all }x} x \mathbb{P}(X=x)$It is important to note that the expected value does not have to be a possible outcome. In fact, in many cases, it is not! For instance, the average roll of a fair die is 3.5.Let $a$ and $b$ be constants and $X$ and $Y$ be random variables, then the following are always true:+ $\mathbb{E}(X + Y) = \mathbb{E}(X) + \mathbb{E}(Y)$ + $\mathbb{E}(aX + b) = a \mathbb{E}(X) + b$Sometimes, you may encounter $\mathbb{E}(X^n)$. This is called the _$n$th moment of $X$_.$\mathbb{E}(X^n) = \sum_{\text{all }x} x^n \mathbb{P}(X=x)$  VarianceVariance of $X$, or $var(X)$, roughly describes the amount of variation in $X$. Mathematically, $var(X) = \mathbb{E}(X^2) - \mathbb{E}(X)^2$Pretty neat!Let $a$ and $b$ be constants and $X$ be a random variable, then the following are always true:+ $var(X) \geq 0$+ $var(aX + b) = a^2 var(X)$+ $var(X + Y) = var(X) + var(Y)$  if $X$ and $Y$ are independent  Discrete Probability DistributionsYou can think of probability distributions as a function that takes in an event and returns the probability or probability density of that event happening. We'll talk about probability density later when we get to continuous distributions. Let $f$ be a function, the two criteria for a function to be a discrete probability distribution are:+ $f(x) \geq 0 \quad \forall x$+ $\sum_{\text{all } x} f(x) = 1$In the discrete case, we think of $f(x) = \mathbb{P}(X=x)$. For example, for a fair die, if $X$ is number we roll, $f(x) = \frac{1}{6}$ if $x \in \{1, 2, 3, 4, 5, 6\}$ and 0 otherwise. We'll explore some well known classes of discrete distributions. Note that all of these distributions (and others) can be simulated in NumPy. UniformA discete uniform distribution is one where each possible outcome is equally likely.$\mathbb{P}(X = x) = \frac{1}{n} \quad \forall \text{ possible } x $ where $n$ is the number of possible outcomes.Example: The distribution of a single die roll. BinomialA binomial distribution is one where we observe a sequence of $n$ identical random events, and we want to know the probability of a specific outcome happening $k$ times out of $n$. Let $p$ be the probability of this outcome happening for each event.$\mathbb{P}(X = k) = {n\choose k} p^k (1-p)^{n-k}$where ${n\choose k} = \frac{n!}{(n-k)!k!}$.You can think of ${n\choose k}$ as the number of ways you can pick combinations of $k$ things from $n$. For instance, the number of ways to draw 5 cards from a standard 52 card deck is ${52\choose 5}$Example: The probability of observing three 5's in 10 dice rolls: ${10\choose 3} {(\frac{1}{6})}^3 {(\frac{5}{6})}^{7}$ Geometric A geometric distribution is one where we care about the first instance of a sequence of identical random events, and we want to know the probability of an outcome occuring for the first time on the $k$th iteration. Let $p$ be the probability of this outcome happening for each event.$\mathbb{P}(X = k) = p(1-p)^{k-1}$Example: The probability of seeing the first 5 on a die on the 7th roll: $ (\frac{1}{6})(\frac{5}{6})^6$  Continuous Probability DistributionsBuckle your seatbelts because things get weird in the continuous realm. Generally in the continuous case, $f(x) \neq \mathbb{P}(X=x)$. In fact, $\mathbb{P}(X=x) = 0$ (think: what is the probability of picking an infinitesimally small point on line? answer: 0). Instead, we calculate probabilities of ranges of $X$ such as $\mathbb{P}(x_1 \leq X \leq x_2)$ by integrating $\int_{x_1}^{x_2} f(x)$. Therefore, the two criteria for a function to be a discrete probability distribution are:+ $f(x) \geq 0 \quad \forall x$+ $\int_{x} f(x) = 1$For instance, something like $f(x) = 2x \text{ for } 0 \leq x \leq 1$ is a valid continuous distribution. Although the usefulness of such a distribution is handwavy, we'll go into some simple and common ones. UniformA continuous uniform distribution is one where each possible outcome is equally likely.$f(x) = \frac{1}{b-a} \text{ for } a \leq x \leq b $Example: x-coordinate of a raindrop hitting a square meter tile. Normal A normal distribution is your standard bell curve. It is defined by the mean $\mu$ and standard deviation $\sigma$:$f(x) = \frac{1}{{\sigma \sqrt {2\pi } }} e^{ - \frac{( {x - \mu } )^2 }{2\sigma^2}} $Example: Distribution of heightTypically, if $X$ is normally distributed with mean $\mu$ and standard deviation $\sigma$, we write $X \sim \mathcal{N}(\mu, \sigma ^2)$. If $X \sim \mathcal{N}(\mu_x, \sigma^2_x)$, $Y \sim \mathcal{N}(\mu_y, \sigma ^2_y)$, and $X$ and $Y$ are independent, then $X+Y \sim \mathcal{N}(\mu_x + \mu_y, \sigma^2_x + \sigma^2_y)$.  Simulations  Uniform (Discrete)<jupyter_code>import numpy as np
import matplotlib.pyplot as plt

samples = 100000
a = 0
b = 10

x = np.random.randint(low=a, high = b, size=samples)
bins = np.linspace(min(x), max(x), 1)
plt.hist(x, bins = 50)
plt.xlabel("x")
plt.ylabel("samples")
plt.show()<jupyter_output><empty_output><jupyter_text>Binomial<jupyter_code>import numpy as np
import matplotlib.pyplot as plt

samples = 100000
n = 10
p = 0.9

x = np.random.binomial(n, p, size=samples)
bins = np.linspace(min(x), max(x), 1)
plt.hist(x, bins = 50)
plt.xlabel("x")
plt.ylabel("samples")
plt.show()<jupyter_output><empty_output><jupyter_text>Geometric<jupyter_code>import numpy as np
import matplotlib.pyplot as plt

samples = 10000
p = 0.1

x = np.random.geometric(p, size=samples)
bins = np.linspace(min(x), max(x), 1)
plt.hist(x, bins = 50)
plt.xlabel("x")
plt.ylabel("samples")
plt.show()<jupyter_output><empty_output><jupyter_text>Uniform (Continuous)<jupyter_code>import numpy as np
import matplotlib.pyplot as plt

samples = 1000000
a = 0
b = 1

x = np.random.uniform(low=a, high = b, size=samples)
bins = np.linspace(min(x), max(x), 0.1)
plt.hist(x, bins = 50)
plt.xlabel("x")
plt.ylabel("samples")
plt.show()<jupyter_output>C:\Users\hdong\Anaconda3\lib\site-packages\ipykernel_launcher.py:9: DeprecationWarning: object of type <class 'float'> cannot be safely interpreted as an integer.
  if __name__ == '__main__':<jupyter_text>Normal<jupyter_code>import numpy as np
import matplotlib.pyplot as plt

samples = 10
mu = 0
sigma = 1

x = np.random.normal(loc=mu, scale=sigma, size=samples)
bins = np.linspace(min(x), max(x), 0.1)
plt.hist(x, bins = 50)
plt.xlabel("x")
plt.ylabel("samples")
plt.show()<jupyter_output>C:\Users\hdong\Anaconda3\lib\site-packages\ipykernel_launcher.py:9: DeprecationWarning: object of type <class 'float'> cannot be safely interpreted as an integer.
  if __name__ == '__main__':
{'fraction_non_alphanumeric': 0.12158374211632796, 'fraction_numerical': 0.014453398738612474, 'mean_word_length': 4.591087169441724, 'pattern_counts': {'":': 0, '<': 23, '<?xml version=': 0, '>': 24, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '461535', 'license': 'MIT', 'n_tokens_mistral': 3677, 'n_tokens_neox': 3477, 'n_words': 1651, 'path': 'Workshops/Workshop_Probability/Probability.ipynb', 'repo_name': 'Mphan6139/Spring-2021-Career-Exploration', 'size': '11430'}