<jupyter_start><jupyter_text>Table of Contents1&nbsp;&nbsp; Concept of Bayes' Theorem1.1&nbsp;&nbsp; Example 11.2&nbsp;&nbsp; Example 21.3&nbsp;&nbsp; Example 31.4&nbsp;&nbsp; Example 41.5&nbsp;&nbsp; Example 51.6&nbsp;&nbsp; Example 6<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
plt.style.use('ggplot')<jupyter_output><empty_output><jupyter_text>Concept of Bayes' Theorem The **Bayes' Theorem** is occasionally called **inverse probability theorem**. As the name suggests, once we have known $p(A|B)$, the Bayes' Theorem can tell us something about $P(B|A)$.  To see how Bayes' Theorem is derived, start from conditional probability$$P(A|B)P(B)=P(A\cap B)\\P(B|A)P(A)=P(A\cap B)$$ Equate them and arrange, we obtain the famous Bayes' Theorem:$$P(B|A) = \frac{P(A|B)P(B)}{P(A)}$$ If there are $k$ partitions of $B$, $P(A)$ can bee explicitly written in total probability form$$P(B_i|A)=\frac{P(A|B_i)P(B_i)}{\sum_{i=0}^k P(A|B_i)P(B_i)}$$ Since Bayes' Theorem can be somewhat counter-intuitive, it might take some time to grasp the tricky logic. We will go through some examples, strongly advise to study them thoroughly.   Example 1 A certain disease affects about 1 out of 10,000 people. There is a test to check whether the person has the disease. The test is quite accurate. In particular, we know that1. the probability that the test result is positive (suggesting the person has the disease), given that the person does not have the disease, is only 2 percent;2. the probability that the test result is negative (suggesting the person does not have the disease), given that the person has the disease, is only 1 percent.A random person gets tested for the disease and the result comes back positive. What is the probability that the person has the disease? $\text{Event A}$: $\text{the test result is positive}$$\text{Event B}$: $\text{the person has the disease}$$$P(B|A) = \frac{P(A|B)P(B)}{P(A|B)P(B)+P(A|B^c)P(B^c)}= \frac{(1- .01)\times.0001}{(1- .01)\times.0001+.02\times(1-.0001)}$$<jupyter_code>pB_A =( .99*.0001)/(.99*.0001+.02*.999)
print('The probability that a person have positive results and having the disease is %.3f%%.'%(pB_A*100))<jupyter_output>The probability that a person have positive results and having the disease is 0.493%.<jupyter_text>The result comes with a surprise. But if you dwell on it a while, you will notice an obvious fact that there are much more 'healthy people'(people who do not have this type of disease) i.e.  $9999:1$. Even if you got to guess, you should consider this fact, which we call **prior**. It means even if a person has a positive result, yet he is still likely to be from healthy group.   Example 2 Suppose a patient has an unidentified lesion in the liver, that internist recommend abdominal MRI for firmer evidences to exclude critical diseases. The sensitivity (true positive) rate of MRI is $92\%$, the prevalence of critical liver disease is $.06\%$. The question is if the result is positive, how likely this patient has critical liver disease? $\text{Event A}$: $\text{MRI positive results}$$\text{Event B}$: $\text{the person has critical disease}$$$p(B|A) = \frac{P(A|B)P(B)}{P(A|B)P(B)+P(A|B^C)P(B^C)} = \frac{.92\times .0006}{.92\times .0006+.08\times .9994}$$<jupyter_code>pB_A = (0.92*.0006)/(.92*.0006+.08*.9994)
print('The probability that a person who has symptoms truly have critical liver disease is %.3f%%.'%(pB_A*100))<jupyter_output>The probability that a person who has symptoms truly have critical liver disease is 0.686%.<jupyter_text>Example 3 A factory produces an item using three different brands of machines—$\text{A}$, $\text{B}$, and $\text{C}$—which account for $20\%$, $30\%$, and $50\%$ of output respectively. Of the items produced by machine $\text{A}$, $5\%$ are defective; similarly, $3\%$ of machine $\text{B}$'s items and $1\%$ of machine $\text{C}$'s are defective. If a randomly selected item is defective, what is the probability it was produced by machine $\text{C}$? $\text{Event A}$: $\text{Produced by machine A}$$\text{Event B}$: $\text{Produced by machine B}$$\text{Event C}$: $\text{Produced by machine C}$  $\text{Event D}$: $\text{Defective product}$  $$P(C|D) = \frac{P(D|C)P(C)}{P(D|A)P(A)+P(D|B)P(B)+P(D|C)P(C)}$$<jupyter_code>pB_A = .5*.01/(.2*.05+.3*.03+.5*.01)
print('The probability that a defective product made by machine C is %.3f%%.'%(pB_A*100))<jupyter_output>The probability that a defective product made by machine C is 20.833%.<jupyter_text>Example 4 Bag I contains 4 white and 6 black balls while Bag II contains 4 white and 3 black balls. One ball is drawn randomly from one of the bags and it is found to be black. Find the probability that it was drawn from Bag I. $\text{Event A}$: $\text{black ball}$$\text{Event B}$: $\text{from bag I}$$$p(B|A) = \frac{p(A|B)p(B)}{p(A)}= \frac{\frac{6}{10}\frac{1}{2}}{\frac{6}{10}\frac{1}{2}+\frac{3}{7}\frac{1}{2}}$$<jupyter_code>pB_A = 6/10*.5/(6/10*.5+3/7*.5)
print('The probability that the ball is from Bag I given it is black is %.5f%%.'%(pB_A*100))<jupyter_output>The probability that the ball is from Bag I given it is black is 58.33333%.<jupyter_text>Example 5 A man is known to speak truth 2 out of 3 times. He throws a die and reports that number obtained is a four. Find the probability that the number obtained is actually a four. $\text{Event A}$: $\text{reports number 4}$$\text{Event B}$: $\text{speak truth}$$$   P(A|B)=\frac{P(B|A)P(A)}{P(B|A)P(A)+P(B^C|A)P(A)}=\frac{\frac{2}{3}\frac{1}{6}}{\frac{2}{3}\frac{1}{6}+\frac{1}{3}\frac{5}{6}}$$<jupyter_code>pB_A = (1/6*2/3)/(1/6*2/3+5/6*1/3)
print('The probability that he truly obtains 4 given he speaks the truth is %.3f%%.'%(pB_A*100))<jupyter_output>The probability that he truly obtains 4 given he speaks the truth is 28.571%.<jupyter_text>Bayesian Inference  Bayes' Theorem won't be much helpful in scientific researches if it is only a tool for calculate the inverse probability. Without changing the theory, just reframe the definition, we will have a completely revolutionary view.  $\text{Event A}$: $\text{Newly collected data}$$\text{Event B}$: $\text{Hypothesis}$$$P(B|A) = \frac{P(A|B)P(B)}{P(A)} = \frac{P(A|B)P(B)}{P(A|B)P(B)+P(A|B^c)P(B^c)}$$ Hypothesis $\text{B}$ has a probability $P(B)$ which is called **prior** in **Bayesian Inference**, $P(A|B)$ is the **likelihood** (exactly the same meaning as in maximum likelihood), $P(B|A)$ is the **posterior** meaning which is also your new prior before next data collection.The reason why $P(A|B)$ is called likelihood is actually quite intuitive, it measures how likely data $A$ will present given $B$ hypothesis is true. $P(A)$ is calculated by law of total probability based on $B$ and $B^C$ hypothesis. If $B$ is called a hypothesis, then it expresses a subjective belief. Broadly speaking, you start with a scientific question and set forth two or more alternative hypotheses. You then assign a prior probability that each alternative hypothesis is true. Next, you collect data. Finally, you use Bayes’ Theorem to update the probability for each hypothesis considered. What if we change the prior gradually in the first example above, from $1$ to $1000$ people out of $10000$, and rest parameters are the same, let's see how the posteriors change.<jupyter_code>p_B = np.linspace(.0001, .1, 100)
pB_A =( .99*p_B)/(.99*p_B+.02*(1-p_B))
fig, ax = plt.subplots(figsize=(12, 6))

ax.plot(p_B, pB_A)
ax.set_xlabel('Prior')
ax.set_ylabel('Posterior')
ax.set_title('Example 1')
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14714077219052674, 'fraction_numerical': 0.03608862942815444, 'mean_word_length': 5.292153589315526, 'pattern_counts': {'":': 0, '<': 24, '<?xml version=': 0, '>': 24, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '483367', 'license': 'MIT', 'n_tokens_mistral': 2641, 'n_tokens_neox': 2400, 'n_words': 954, 'path': "Chapter 2 - Bayes' Theorem.ipynb", 'repo_name': 'evolvPy/Probab_Py-1', 'size': '7537'}