<jupyter_start><jupyter_text>Continuous version of Bayesian theorem $$\large f(\theta|y) = \frac{f(y|\theta)f(\theta)}{\int_{-\infty}^{\infty} f(y|\theta)f(\theta)d\theta}$$This can be read as:* ***likelihood * prior / normalizing constant***Because the integral can be hard to compute, we also say that this expression is proportional to* ***likelihood * prior***<jupyter_code>np.quantile(np.random.normal(0, 1, 1000000), .025)<jupyter_output><empty_output><jupyter_text>Exponential and Gamma  Recall that we used the conjugate gamma prior for λ, the arrival rate in busses per minute. Suppose our prior belief about this rate is that it should have mean 1/20 arrivals per minute with standard deviation 1/5. Then the prior is Gamma(a,b) with a=1/16.Find b.<jupyter_code># sqrt(a) / b = 1/5
# a / b = 1/20
.0625 / 0.05, .25 / .2<jupyter_output><empty_output><jupyter_text>Suppose that we wish to use a prior with the same mean (1/20), but with effective sample size of one arrival. Then the prior for λ is Gamma(1,20).In addition to the original Y1=12, we observe the waiting times for four additional busses: Y2=15, Y3=8, Y4=13.5, Y5=25.Recall that with multiple (independent) observations, the posterior for λ is Gamma(α,β) where α=a+n and β=b+∑yi.**What is the posterior mean for λ? Round your answer to two decimal places.**<jupyter_code>ys = [20, 12, 15, 8, 13.5, 25]
len(ys) / sum(ys)<jupyter_output><empty_output><jupyter_text>Normal  Known variance___Let's imagine a case where we know variance of a normally distributed variable, but the mean is unknown:$$\Large X \sim \mathcal{N}(\mu, \sigma_{o}^2)$$A conjugate prior for the mean will also be **normal**:$$\Large \mu \sim \mathcal{N}(m_o, s_{o}^2)$$Then the posterior will be:$$\Large \mu |X \sim \mathcal{N}(\frac{\frac{n \bar{X}}{\sigma_o^2} + \frac{m_0}{s_o^2}}{\frac{n}{\sigma_o^2} + \frac{1}{s_o^2}}, \frac{1}{\frac{n}{\sigma_o^2} + \frac{1}{s_o^2}})$$Effective sample size is:$$\Large ESS = \frac{\sigma_o^2}{s_o^2}$$The **more variablility** in the prior, the **less information** is in it and vice versa.<jupyter_code>def get_normal_posterior_mean_params(x, sigma2, m, s2):
    
    n = len(x)
    
    post_mean_mean = ((n * np.mean(x) / sigma2) + (m / s2)) / ((n / sigma2) + (1 / s2))
    post_mean_var  = 1 / ((n / sigma2) + (1 / s2))
    
    return post_mean_mean, post_mean_var
def get_effective_sample_size(sigma2, s2, sigma_known = True):
    return sigma2 / s2<jupyter_output><empty_output><jupyter_text>Unknown variance___Is both - **mean** and **variance** - are **unknown**, we can specify priors in a hierarchcal fashion.We have:$$\Large X \sim \mathcal{N}(\mu, \sigma^2)$$with two unknown parameters.We can specify a prior for $\mu$ as:$$\Large \mu | \sigma^2 \sim \mathcal{N}(m, \frac{\sigma^2}{w})$$where $$w = \frac{\sigma^2}{\sigma^2_{\mu}}$$is the **effective sample size** of the prior.A prior for $\sigma^2$ will be:$$\Large \sigma^2 \sim \Gamma^{-1}(\alpha, \beta)$$Then, the posterior for $\sigma^2$ will be:$$\Large \sigma^2 | X \sim \Gamma^{-1}(\alpha + \frac{n}{2}, \beta + \frac{1}{2} \sum_{i=1}^n (x_i - \bar{x})^2) + \frac{nw}{2(n + w)}(\bar{x} - m)^2$$and the posterior for $\mu$ will be:$$\Large \mu | \sigma^2, X \sim \mathcal{N}(\frac{n \bar{x} + wm}{n+w}, \frac{\sigma^2}{n + w})$$If we only care about $\mu$, then the marginal posterior will follow a $t$ distribution:$$\Large \mu | X \sim t$$ **Exercise 1**Suppose you are trying to calibrate a thermometer by testing the temperature it reads when water begins to boil. Because of natural variation, you take nnn independent measurements (experiments) to estimate $\theta$, the mean temperature reading for this thermometer at the boiling point. Assume a normal likelihood for these data, with mean $\theta$ and known variance $\sigma^2 = 0.25$ (which corresponds to a standard deviation of 0.5 degrees Celsius).Suppose your prior for $\theta$ is (conveniently) the conjugate normal. You know that at sea level, water should boil at 100 degrees Celsius, so you set the prior mean at $m_0 = 100$. You collect the following $n=5$ measurements: (94.6, 95.4, 96.2, 94.9, 95.9).What is the posterior distribution for $\theta$?<jupyter_code>measurements = [94.6, 95.4, 96.2, 94.9, 95.9]
get_normal_posterior_mean_params(measurements, .25, 100, .25)
get_normal_posterior_mean_params(measurements, .25, 100, 1/.25)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.15729569770088778, 'fraction_numerical': 0.040291372638288185, 'mean_word_length': 4.684346701164295, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '319363', 'license': 'MIT', 'n_tokens_mistral': 1623, 'n_tokens_neox': 1463, 'n_words': 513, 'path': 'bayesian-notes-santa-cruz/03_Continuous_Bayesian.ipynb', 'repo_name': 'AlxndrMlk/bayesian-stuff', 'size': '4393'}