<jupyter_start><jupyter_text>"[Prob] Basics of the Poisson Distribution"> "Some useful facts about the Poisson distribution"- toc:false- branch: master- badges: false- comments: true- author: <NAME>- categories: [category, learning, probability]  Introduction The Poisson distribution is an important discrete probability distribution prevalent in a variety of fields. In this post, I will present some useful facts about the Poisson distribution. Here's the concepts I will discuss in the post:* PMF, expectation and variance of Poisson* In what situation we can use it?* Sum of indepentent Poisson is also a Poisson* Relationship with the Binomial distribution  PMF, Expectation and Variance First, we define what's Poisson distribution. Let X be a Poisson random variable with a parameter $\lambda$, where $\lambda >0$. The pmf of X would be:$$P(X=x) = \frac{e^{-\lambda}\lambda^{x}}{x!}, \quad \text{for } k = 0, 1,2,3,\dots$$where $x$ can only be non-negative integer.This is a valid pmf since $$\sum_{k=0}^{\infty} \frac{e^{-\lambda}\lambda^{k}}{k!} = e^{-\lambda}\sum_{k=0}^{\infty} \frac{\lambda^{k}}{k!}= e^{-\lambda}e^{\lambda}=1$$where $\displaystyle\sum_{k=0}^{\infty} \frac{\lambda^{k}}{k!}$ is the Taylor expansion of $e^{\lambda}$. The expectation and the variance of the Poisson distribution are both $\lambda$. The derivation of this result is just some pattern recognition of $\sum_{k=0}^{\infty} \frac{\lambda^{k}}{k!}=e^{\lambda}$, so I omit it here.  In what situation can we use it? The Poisson distribution is often applied to the situation where we are counting the number of successes or an event happening in a time interval or a particular region, and there are a large number of trials with a small probability of success. The parameter $\lambda$ is the rate parameter which indicates the average number of successes in a time interval or a region.Here are some examples:* The number of emails you receive in an hour.* The number of chips in a chocolate chip cookie.* The number of earthquakes in a year in some region of the world.Also, let's consider an example probability problem.**Example problem 1**> Raindrops are falling at an average rate of 20 drops per square inch per minute. Find the probability that the region has no rain drops in a given 1-minute time interval.The success in this problem is one raindrop. The average rate is 20, so $\lambda=20$. Let $X$ be the raindrops that region has in a minute. We would model $X$ with Pois$(20)$, so the probability we concerned would be$$P(X=0) = \frac{e^{-20}20^0}{0!}=e^{-20} \approx 2.0611\times 10 ^{-9}$$If we are concerned with raindrops in a 3-second time interval in 5 square inches, then $$\lambda = 20\times\frac{1}{20} \text{ minutes} \times5 \text{ square inches} = 5$$Let $Y$ be raindrops in a 3-second time interval. $Y$ would be Pois$(5)$, so $P(Y=0) = e^{-5} \approx 0.0067$.  Sum of Independent Poisson The sum of independent Poisson would also be Poisson. Let $X$ be Pois$(\lambda_1)$ and $Y$ be Pois$(\lambda_2)$. If $T=X+Y$, then $T \sim \text{Pois}(\lambda_1 + \lambda_2)$.To get pmf of $T$, we should first apply the law of total probability:$$P(X+Y=t) =  \sum_{k=0}^{t}P(X+Y=t|X=k)P(X=k) $$Since they are independent, we got$$\sum_{k=0}^{t}P(X+Y=t|X=k)P(X=k)  = \sum_{k=0}^{t}P(Y=t-k)P(X=k)$$Next, we plug in the pmf of Poisson:$$\sum_{k=0}^{t}P(Y=t-k)P(X=k) = \sum_{k=0}^{t}\frac{e^{-\lambda_2}\lambda_2^{t-k}}{(t-k)!}\frac{e^{-\lambda_2}\lambda_1^k}{k!} = \frac{e^{-(\lambda_1+\lambda_2)}}{t!}\sum_{k=0}^{t} {t \choose k}\lambda_1^{k}\lambda_2^{t-k}$$Finally, by Binomial theorem, we got$$P(X+Y=t) = \frac{e^{-(\lambda_1+\lambda_2)}(\lambda_1+\lambda_2)^t}{t!}$$which is the pmf of Pois$(\lambda_1 + \lambda_2)$.  Relationship with the Binomial distribution We can obtain Poisson from Binomial and can also obtain Binomial to Poisson. Let's first see how we get the Binomial distribution from the Poisson distribution **From Poisson to Binomial**If $X \sim$ Pois$(\lambda_1)$ and $Y \sim$ Pois$(\lambda_2)$, and they are independent, then the conditional distribution of $X$ given $X+Y=n$ is Bin$(n, \lambda_1/(\lambda_1 + \lambda_2))$. Let's derive the pmf of $X$ given $X+Y=n$.By Bayes' rule and the indenpendence between $X$ and $Y$:$$P(X=x|X+Y=n) = \frac{P(X+Y=n|X=x)P(X=x)}{P(X+Y=n)} = \frac{P(Y=n-k)P(X=x)}{P(X+Y=n)}$$From the previous section, we know $X+Y \sim$ Poin$(\lambda_1 + \lambda_2)$. Use this fact, we get$$P(X=x|X+Y=n) = \frac{ \big(\frac{e^{-\lambda_2}\lambda_2^{n-k}}{(n-k)!}\big) \big( \frac{e^{\lambda_1\lambda_1^k}}{k!} \big)}{ \frac{e^{-(\lambda_1 + \lambda_2)}(\lambda_1 + \lambda_2)^n}{n!}} = {n\choose k}\bigg(\frac{\lambda_1}{\lambda_1+\lambda_2}\bigg)^k \bigg(\frac{\lambda_2}{\lambda_1+\lambda_2}\bigg)^{n-k}$$which is the Bin$(n, \lambda_1/(\lambda_1 + \lambda_2))$ pmf.**From Binomial to Poisson**We can approximate Binomial by Poisson when $n \rightarrow \infty$ and $p \rightarrow 0$, and $\lambda = np$.The pmf of Binomial is$$P(X=k) = {n \choose k}p^{k}(1-p)^{n-k} = {n \choose k}\big(\frac{\lambda}{n}\big)^{k}\big(1-\frac{\lambda}{n}\big)^n\big(1-\frac{\lambda}{n}\big)^{-k}$$By some algebra manipulation, we got$$P(X=k) = \frac{\lambda^{k}}{k!}\frac{n(n-1)\dots(n-k+1)}{n^k}\big(1-\frac{\lambda}{n}\big)^n\big(1-\frac{\lambda}{n}\big)^{-k}$$When $n \rightarrow \infty$, we got:$$\frac{n(n-1)\dots(n-k+1)}{n^k} \rightarrow 1,\\\big(1-\frac{\lambda}{n}\big)^n \rightarrow e^{-\lambda}, \text{and}\\ \big(1-\frac{\lambda}{n}\big)^{-k} \rightarrow 1$$Therefore, $P(X=k) = \frac{e^{-\lambda}\lambda^k}{k!}$ when $n \rightarrow \infty$.Let's see an example on how to use Poisson to approximate Binomial. **Example problem 2**>Ten million people enter a certain lottery. For each person, the chance of winning is one in ten million, independently. Find a simple, good approximation for the PMF of the number of people who win the lottery.Let $X$ be the number of people winning the lottery. $X$ would be Bin$(10000000, 1/10000000)$ and $E(X) = 1$. We can approximate the pmf of $X$ by Pois$(1)$:$$P(X=k) \approx \frac{1}{e\cdot k!}$$ Let's see if this approximation is accurate by Python code.<jupyter_code>#collapse-hide

from scipy.stats import binom
from math import factorial, exp
import numpy as np
import matplotlib.pyplot as plt

def pois(k):
    return 1 / (exp(1) * factorial(k))

n = 10000000
p = 1/10000000

k = np.arange(10)
binomial = binom.pmf(k, n, p)
poisson = [pois(i) for i in k]

fig, ax = plt.subplots(ncols=2, nrows=1, figsize=(15, 4), dpi=120)
ax[0].plot(k, binomial)
ax[0].set_title("PMF of Binomial")
ax[0].set_xlabel(r"$X=k$")
ax[0].set_xticks(k)

ax[1].plot(k, poisson)
ax[1].set_title("Approximation by Poisson")
ax[1].set_xlabel(r"X=k")
ax[1].set_xticks(k)

plt.tight_layout();<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.17824996311052088, 'fraction_numerical': 0.025822635384388372, 'mean_word_length': 4.956063268892795, 'pattern_counts': {'":': 0, '<': 6, '<?xml version=': 0, '>': 10, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 4, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '10135', 'license': 'Apache-2.0', 'n_tokens_mistral': 2453, 'n_tokens_neox': 2294, 'n_words': 793, 'path': '_notebooks/2021-05-07-Basics of Poisson.ipynb', 'repo_name': 'peiyiHung/mywebsite', 'size': '6781'}