<jupyter_start><jupyter_text>Bayesian Inference for Binomial Proportion For a random variable to be described by a Binomial distribution parameterised as $Y \sim \text{Binomial}(n,\pi)$ with the parameter prior described as $g(\pi)$ and the likelihood of the prior given the data (which is equivalent to the probability of seeing the data given the prior) being represented as $f(y | \pi)$, the posterior distribution of the parameters become proportional to $g(\pi) \times f(y | \pi)$ If $g(\pi)$ is not described by a Beta distribution, that's when shit hits the fan, and what this notebook is all about.   Non-Beta Priors A student is constructing their prior belief about $π$, the proportion of Hamilton residents who support building a casino in Hamilton. Let the prior beliefs about $\pi$  be quantified as$$ g(\pi) =  \begin{array}{cc}  \Bigg\{ &     \begin{array}{cc}      10\pi & \text{for}\; 0\leq\pi\leq 0.2 \\      2 & \text{for}\; 0.2\leq\pi\leq 0.3 \\      5-10\pi & \text{for}\; 0.3\leq\pi\leq 0.5    \end{array}\end{array}$$  The Data The student takes a random sample of $n = 100$Hamilton residents and find their views on the casino. Out of the random sample,$y = 26$ said they support building a casino in Hamilton. We want to find the posterior distribution of $\pi$$$ g(\pi | y) = \frac{g(\pi) \times f(y | \pi)}{\int_{0}^{1}g(\pi) \times f(y | \pi) d\pi} $$ We aim to numerically integrate to find the denominator for the above non-beta prior.<jupyter_code>pi = np.linspace(0, 1, 10000)
prior = (pi<0.2)*g1(pi) + ((pi>0.2) & (pi<0.3))*g2(pi) + ((pi>0.3) & (pi<0.5))*g3(pi)
plt.plot(pi, prior)
plt.title('Prior distribution of the parameter $\pi$')
plt.show()<jupyter_output><empty_output><jupyter_text>Integration as the Limit of a SumWe break the domain of integration in the denominator into three parts. $$ \int_{0}^{1}g(\pi) \times f(y | \pi) d\pi = \int_{0}^{0.2}g_1(\pi) \times f(y | \pi) d\pi + \int_{0.2}^{0.3}g_2(\pi) \times f(y | \pi) d\pi + \int_{0.3}^{0.5}g_3(\pi) \times f(y | \pi) d\pi$$where $g_i(\pi)$ refer to each of the pieces of the piecewise continuous prior distribution $g(\pi)$For example, $g_1(\pi) = 10\pi$We evaluate the three integrals numerically below.<jupyter_code>from scipy.special import comb 

def g1(x):
    return 10*x

def g2(x):
    return 2

def g3(x):
    return 5 - 10*x

def f(y, pi, n = 100):
    return comb(n, y, exact = True, repetition = False)*pi**(y) * (1-pi)**(n-y)
# Number of finite steps for integration
num_steps = 500<jupyter_output><empty_output><jupyter_text>Evaluating $$ \int_{0}^{0.2}g_1(\pi) \times f(y | \pi) d\pi \sim \sum_{i = 1}^{{num-steps}} g_1(\pi)*f(y|\pi)*\Delta\pi $$<jupyter_code>import numpy as np

int_1 = 0
for pi in np.linspace(0, 0.2, num = num_steps):
    int_1 = int_1 + g1(pi)*f(pi = pi, y = 26, n = 100)*(0.2/num_steps)
int_1<jupyter_output><empty_output><jupyter_text>Evaluating $$ \int_{0.2}^{0.3}g_2(\pi) \times f(y | \pi) d\pi $$<jupyter_code>int_2 = 0
for pi in np.linspace(0.2, 0.3, num = num_steps):
    int_2 = int_2 + g2(pi)*f(pi = pi, y = 26, n = 100)*(0.1/num_steps)
int_2<jupyter_output><empty_output><jupyter_text>Evaluating $$ \int_{0.3}^{0.5}g_3(\pi) \times f(y | \pi) d\pi$$<jupyter_code>int_3 = 0
for pi in np.linspace(0.3, 0.5, num = num_steps):
    int_3 = int_3 + g3(pi)*f(pi = pi, y = 26, n = 100)*(0.2/num_steps)
int_3<jupyter_output><empty_output><jupyter_text>All combining $$ \int_{0}^{1}g(\pi) \times f(y | \pi) d\pi $$<jupyter_code>denominator = int_1 + int_2 + int_3
denominator<jupyter_output><empty_output><jupyter_text>Plotting the posterior involves expressing the numerator in $$ g(\pi | y) = \frac{g(\pi) \times f(y | \pi)}{\int_{0}^{1}g(\pi) \times f(y | \pi) d\pi} $$as a function of $\pi$ and evaluating it for all values of $\pi \in [0,1]$. This will give us the general shape of the curve.To get the posterior probability distribution, we have to divide by the denominator<jupyter_code>pi = np.linspace(0,1,num_steps)
posterior = ((pi<0.2)*g1(pi)*f(y = 26, pi = pi, n = 100) + ((pi>0.2) & (pi<0.3))*g2(pi)*f(y = 26, pi = pi, n = 100) + ((pi>0.3) & (pi<0.5))*g3(pi)*f(y = 26, pi = pi, n = 100))/denominator
plt.plot(pi, posterior)
plt.title('Plot of Posterior distribution of $\\pi$ after observing \n y = 26 with n = 100')
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1732466326056665, 'fraction_numerical': 0.043892243381328376, 'mean_word_length': 4.151913875598086, 'pattern_counts': {'":': 0, '<': 35, '<?xml version=': 0, '>': 33, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 1, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '370443', 'license': 'MIT', 'n_tokens_mistral': 1766, 'n_tokens_neox': 1641, 'n_words': 500, 'path': 'Bayesian Inference non-beta priors.ipynb', 'repo_name': 'tamojit-maiti/machine-learning-from-scratch', 'size': '4306'}