<jupyter_start><jupyter_text>Basic setup Create anaconda environment```bashconda create -n ml python=3.7.4 jupyter```Install fastai library```bashconda install -c pytorch -c fastai fastai```  Sigma-algebras Let $X$ be a set and $\mathcal{F} \subset 2^X$ with properties:- $X \in \mathcal{F}$- if $A \in \mathcal{F}$ then $X - A \in \mathcal{F}$- for every $(A_n)_{n = 1}^{\infty}$ such that $A_n \in \mathcal{F}$ for any $1 \le n \le \infty$: $\bigcup_{n=1}^{\infty}A_n \in \mathcal{F}$ The $\mathcal{F}$ with this properties is called the $\sigma$-algebra on X From the ebove properies we can easily conclude that for every $\sigma$-algebra $\mathcal{F}$ on the set $X$:- $\emptyset \in \mathcal{F}$- for every $(A_n)_{n = 1}^{\infty}$ such that $A_n \in \mathcal{F}$ for any $1 \le n \le \infty$: $\bigcap_{n=1}^{\infty}A_n \in \mathcal{F}$ For every set $X$ the "smallest" $\sigma$-algebra will be $\{X, \emptyset\}$ and the biggest will be - $2^X$ The elements of the $\sigma$-algebra are called mesurable sets and the pair $(X, \mathcal{F})$ is called mesurable space The function $f:(X, \mathcal{F}_X) \to (Y, \mathcal{F}_Y)$ between two mesurable spaces is called mesurable function if for every $F \in \mathcal{F}_Y$ the set $f^{-1}(F) \in \mathcal{F}_X$  Mesure Let $(X, \mathcal{F})$ be a mesurable space, the function $\mu:\mathcal{F} \to [0, \infty]$ is called mesure if:- $\mu(\emptyset) = 0$- For pairwise disjoint sets $(E_k)_{k=1}^\infty$ the following holds: $\mu\left(\bigcup_{k=1}^\infty E_k\right)=\sum_{k=1}^\infty \mu(E_k)$ Properties:- if $A \subset B$ then $\mu(A) \le \mu(B)$Proof: $(B - A) \cup (A \cap B) = (B - A) \cup A = B$ and $(B - A) \cap (A \cap B) = \emptyset$ so $\mu((B - A) \cup (A \cap B)) = \mu(B - A) + \mu(A \cap B) = \mu(B - A) + \mu(A) = \mu(B)$ so $\mu(A) \le \mu(B)$ The triple $(X, \mathcal{F}, \mu)$ is called mesure space  Probability Define $\sigma$-algebra on the sample space $\mathcal{F} \subset 2^{\Omega}$ and mesure $P:\mathcal{F} \subset 2^{\Omega} \to [0, 1] \subset \mathbb{R}_+$ mesure The mesure $P:\mathcal{F} \subset 2^{\Omega} \to [0, \infty]$ on $\sigma$-algebra is called probability if $P(\Omega) = 1$ Properties:- $P(\emptyset) = 0$- if $A \subset B$ then $P(A) \le P(B)$- $0 \le P(A) \le 1$- $P(A^c) = 1 - P(A)$ where $A^{c} = \Omega - A$- $P(A \cup B) = P(A) + P(B) - P(A \cap B)$ For $A \subset \Omega$ and $B \subset \Omega$ events we can assume that:$A \cap B$ is interpreted as $A$ and $B$ events simultaneously and sometimes we denote $AB$ $A \cup B$ mean $A$ or $B$ events  Example (Uniform probability distribution):Let $\Omega$ is finity set of samples and let probability of each outcome equaly likly, then for $A \subset \Omega$ $$\frac{|A|}{|\Omega|}$$For instance let $\Omega$ be a toss of coin, then we have $\Omega = \{H, T\}$ and $|\Omega| = 2$ and $|\{H\}| = 1$, so the probability of head $P(H) = \frac{|A|}{|\Omega|} = \frac{1}{2}$  Independent events:If we toss the coin twise, then the probability of two heads is $$P(\{H, H\}) = \frac{1}{2} \cdot \frac{1}{2} = \frac{1}{4}$$ Definition:Two events $A$ and $B$ are independent if$$P(A \cap B) = P(A)P(B)$$A set of events $(A_i)_{i \in I}$ are independent if$$P(\bigcap_{i \in I}A_i) = \prod_{i\in I}P(A_i)$$We can assume that two events are independent, for instance when we tossing the coints we know thet koints does not have a memory and thus each toss is independent for other.On the other hand we can prove that $P(AB) = P(A)P(B)$ and conclude that $A$ and $B$ are independent.Let $\Omega$ be a tossing the fair dice and let $A = \{2, 4, 6\}$ and $B = \{1, 2, 3, 4\}$ then $AB = A \cap B = \{2, 4\}$ and $P(AB) = \frac{2}{6} = \frac{1}{3}$. But $P(A) = \frac{3}{6}$ and $P(B) = \frac{4}{6}$ which iplies $P(AB) = \frac{1}{3}$ by which we can colude that this events are independent  Conditional probability:If $P(B) \gt 0$ then conditional probability of $A$ given $B$ is:$$P(A|B) = \frac{P(AB)}{P(B)}$$For fixed $B \subset \Omega$ such that $P(B) \gt 0$ let define function $P_B = P(.|B):\Omega \to \mathbb{R}_+$, then $P_B$ is probability mesure on $2^{\Omega}$- $P(\Omega|B) = 1$ ir contains $P(B|B) = 1$- $P(\bigcup_{i=1}^{n}A_i|B) = \sum_{i=1}^{n}P(A_i|B)$ for disjoint sets $(A_i)_{i=1}^{n}$  Example: Let we have a medical desease $D$ test with outcomes $+$ and $-$:                                     $D$                        $D^c$                                $+$                        $0.0081$                        $0.0900$                            $-$                        $0.0009$                        $0.9010$             Then:$$P(+|D) = \frac{P(+, D)}{P(D)} = \frac{0.0081}{0.0081 + 0.0009} = 0.9$$$$P(-|D^c) = \frac{P(-, D^c)}{P(D^c)} = \frac{0.9010}{0.0900 + 0.9010} \approx 0.9$$<jupyter_code>p1 = (0.0081) / (0.0081 + 0.0009)
p2 = (0.9010) / (0.0900 + 0.9010)
p1, p2<jupyter_output><empty_output><jupyter_text>The test is accurate, sick people get positive with $0.9$ probability or $90%$ accuratacy results as well as healthy people get negative result with $0.9$ probability or $90%$ accuratacy.The question is, if some person got test result as positive ($+$), what is the probability thet they have disease?The first answer is $0.9$ but it is wrong. $$P(D|+) = \frac{P(D, +)}{P(+)} = \frac{0.081}{0.081 + 0.0900} = \frac{0.081}{0.0981} \approx 0.48$$<jupyter_code>p3 = (0.081) / (0.081 + 0.0900)
p3<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.20192307692307693, 'fraction_numerical': 0.03753698224852071, 'mean_word_length': 3.2290852228303364, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 9, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '199785', 'license': 'Apache-2.0', 'n_tokens_mistral': 2115, 'n_tokens_neox': 1972, 'n_words': 712, 'path': 'content/week-06/Probability.ipynb', 'repo_name': 'GiorgiBeriashvili/school-of-ai', 'size': '5408'}