<jupyter_start><jupyter_text>Lambda School Data Science Module 133 Introduction to Bayesian Inference  Assignment - Code it up!Most of the above was pure math - now write Python code to reproduce the results! This is purposefully open ended - you'll have to think about how you should represent probabilities and events. You can and should look things up, and as a stretch goal - refactor your code into helpful reusable functions!Specific goals/targets:1. Write a function `def prob_drunk_given_positive(prob_drunk_prior, prob_positive, prob_positive_drunk)` that reproduces the example from lecture, and use it to calculate and visualize a range of situations2. Explore `scipy.stats.bayes_mvs` - read its documentation, and experiment with it on data you've tested in other ways earlier this week3. Create a visualization comparing the results of a Bayesian approach to a traditional/frequentist approach4. In your own words, summarize the difference between Bayesian and Frequentist statisticsIf you're unsure where to start, check out [this blog post of Bayes theorem with Python](https://dataconomy.com/2015/02/introduction-to-bayes-theorem-with-python/) - you could and should create something similar!Stretch goals:- Apply a Bayesian technique to a problem you previously worked (in an assignment or project work) on from a frequentist (standard) perspective- Check out [PyMC3](https://docs.pymc.io/) (note this goes beyond hypothesis tests into modeling) - read the guides and work through some examples- Take PyMC3 further - see if you can build something with it!<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
#decided to try my hand at recursion because I'm a dummy... but I think it works

def prob_drunk_given_positive(prob_drunk_prior, 
                              prob_positive, 
                              prob_positive_given_drunk, 
                              iterations):
  if iterations == 1: #allows have a way out of the loop
    return prob_positive_given_drunk * prob_drunk_prior / prob_positive
  else:
    iterations = iterations - 1  #this lowers iterations until it hits 1
    prob_drunk_prior = prob_drunk_given_positive(prob_drunk_prior, prob_positive, 
                                                 prob_positive_given_drunk, 
                                                 iterations)
    return prob_positive_given_drunk * prob_drunk_prior / prob_positive
prob = prob_drunk_given_positive(1/1000, .08, 1, 1)
print(prob)
for i in range(10):
  prob = prob_drunk_given_positive(1/1000, .08, 1, i+1)
  print("Probablity of DUI after ", i+1, "Tests is: ", prob)<jupyter_output>Probablity of DUI after  1 Tests is:  0.0125
Probablity of DUI after  2 Tests is:  0.15625
Probablity of DUI after  3 Tests is:  1.953125
Probablity of DUI after  4 Tests is:  24.4140625
Probablity of DUI after  5 Tests is:  305.17578125
Probablity of DUI after  6 Tests is:  3814.697265625
Probablity of DUI after  7 Tests is:  47683.7158203125
Probablity of DUI after  8 Tests is:  596046.4477539062
Probablity of DUI after  9 Tests is:  7450580.596923828
Probablity of DUI after  10 Tests is:  93132257.46154785<jupyter_text>OOOOOOKKKKKKAAAAAAAY the greater than 1.00 numbers are HIGHLY suspicious, and most likely indicitive that this recursion is broken. But it does work for the first iteration.<jupyter_code>prob_drunk = prob_drunk_given_positive(1/1000, .08, 1, 1)
prob_drunk
prob_drunk = prob_drunk_given_positive(prob_drunk, .08, 1, 1)
prob_drunk<jupyter_output><empty_output><jupyter_text>Hmm. Okay, so my recursion is working as intended. It's just breaking statistics now. Great.<jupyter_code>def non_recursive_func(prob_drunk_prior, 
                              prob_positive, 
                              prob_positive_given_drunk):
  return prob_positive_given_drunk * prob_drunk_prior / prob_positive
prob_drunk = non_recursive_func(1/1000, .08, 1)
prob_drunk
prob_drunk = non_recursive_func(prob_drunk, .08, 1)
prob_drunk<jupyter_output><empty_output><jupyter_text>Okay so the recursion wasn't the problem. Must be a fault in how I'm applying the formula I'm using. Or maybe the formula used in lecture for this part is incomplete, I rememberryan saying something about the version used wasn't quite right.<jupyter_code>#I'm just going to use the formula from the drug example
#on the drunk example, do it by "hand", and see if we get .0125
1 * 1/1000 / (1 * 1/1000 + .08 * 999/1000)<jupyter_output><empty_output><jupyter_text>AH HA! the formula the drunk section of lecture was usingWAS missing stuff!<jupyter_code># p_positive_g_drunk * p_drunk /
# (p_positive_g_drunk * p_drunk + p_positive_g_sober * p_sober)
def non_recursive(p_drunk, 
                       p_positive_g_sober, 
                       p_positive_g_drunk):
  p_sober = 1 - p_drunk
  return p_positive_g_drunk * p_drunk / (p_positive_g_drunk * p_drunk + p_positive_g_sober * p_sober)
non_recursive(1/1000, .08, 1)<jupyter_output><empty_output><jupyter_text>Looks good.<jupyter_code>def p_drunk_g_positive(p_drunk, 
                       p_positive_g_sober, 
                       p_positive_g_drunk, 
                       iterations):
  
  if iterations == 1:
    p_sober = 1 - p_drunk
    return p_positive_g_drunk * p_drunk / (p_positive_g_drunk * p_drunk + p_positive_g_sober * p_sober)
  
  else:
    p_sober = 1 - p_drunk
    p_drunk = p_drunk_g_positive(p_drunk, 
                                 p_positive_g_sober, 
                                 p_positive_g_drunk, 
                                 iterations - 1)
    p_sober = 1 - p_drunk
    return (p_positive_g_drunk * p_drunk) / (p_positive_g_drunk * p_drunk + p_positive_g_sober * p_sober)
p_drunk_g_positive(1/1000, .08, 1, 1)
for i in range(10):
  prob = p_drunk_g_positive(1/1000, .08, 1, i+1)
  print("Certainty of DUI after ", i+1, "Tests is: ", prob*100,'%')<jupyter_output>Certainty of DUI after  1 Tests is:  1.2357884330202669 %
Certainty of DUI after  2 Tests is:  13.525210993291495 %
Certainty of DUI after  3 Tests is:  66.15996951348605 %
Certainty of DUI after  4 Tests is:  96.06895076105054 %
Certainty of DUI after  5 Tests is:  99.67371577896735 %
Certainty of DUI after  6 Tests is:  99.97381867081508 %
Certainty of DUI after  7 Tests is:  99.99790498904363 %
Certainty of DUI after  8 Tests is:  99.99983239589308 %
Certainty of DUI after  9 Tests is:  99.99998659165077 %
Certainty of DUI after  10 Tests is:  99.99999892733193 %<jupyter_text>And there we go. Problems all seem sorted. \\Given how many false positives this test puts out, it makes sense that it takes 4-5 times before we become very, very sure that someone is drunk<jupyter_code>data = []
index = []
for i in range(10):
  prob = p_drunk_g_positive(1/1000, .08, 1, i+1)
  data.append(prob)
  index.append(i + 1)
print(data, '\n', index)
df = pd.DataFrame({'consecutive_pos_tests': index, 'dui_probability': data})
df
df.dtypes
plt.plot(df['consecutive_pos_tests'], 
         df['dui_probability']*100, 
         linestyle='--', 
         marker='o')
plt.xlabel('Number of Consective Positive Tests')
plt.ylabel('Percent Chance the Person is Drunk ')
#here we play a little with the politics data set with bayesian statistcs
politics = pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/voting-records/house-votes-84.data')
bills = {
    'republican': 'party',
    'n': 'infants',
    'y': 'cost-sharing',
    'n.1': 'budget',
    'y.1': 'fee-freeze',
    'y.2': 'el-salvador',
    'y.3': 'religion',
    'n.2': 'test-ban',
    'n.3': 'nicaraguan-contras', 
    'n.4': 'mx-missile', 
    'y.4': 'immigration', 
    '?': 'corporation-cutback', 
    'y.5': 'education', 
    'y.6': 'right-to-sue', 
    'y.7': 'crime', 
    'n.5': 'duty-free-exports', 
    'y.8': 'export-administration'
}
politics = politics.rename(columns = bills)
politics = politics.replace({'y': 1, 'n': -1, '?': 0})
politics.head()
import scipy.stats as stats
conf_int, _, _ = stats.bayes_mvs(politics['cost-sharing'],.95)
conf_int
fig = plt.figure()
ax = fig.add_subplot(111)
ax.hist(politics['cost-sharing'], width = .05)
ax.vlines(conf_int.statistic, ymin = 0, ymax = 200, colors='g')
ax.vlines(conf_int[1][0], ymin = 0, ymax = 200, colors='r')
ax.vlines(conf_int[1][1], ymin = 0, ymax = 200, colors='r')
plt.xticks([-1,0,1])

plt.show();
#frequentist statistician function
def freq_dui (p_drunk_g_positive, iterations):
  
  return 1 - (1 - p_drunk_g_positive)**iterations
freq_dui(.92, 3)
data2 = []
index2 = []
for i in range(10):
  prob = freq_dui(.92, i+1)
  data2.append(prob)
  index2.append(i + 1)
print(data2, '\n', index2)
df2 = pd.DataFrame({'consecutive_pos_tests': index2, 'dui_probability': data2})
df2
#graphing both the bayesian and the frequentist together
plt.plot(df2['consecutive_pos_tests'], 
         df2['dui_probability']*100, 
         linestyle='--', 
         marker='o',
         color = 'orange')

plt.plot(df['consecutive_pos_tests'], 
         df['dui_probability']*100, 
         linestyle='--', 
         marker='o',
         color = 'blue')

plt.show();<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09735287683220302, 'fraction_numerical': 0.056552176766571864, 'mean_word_length': 3.523998020781791, 'pattern_counts': {'":': 0, '<': 27, '<?xml version=': 0, '>': 27, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '301677', 'license': 'MIT', 'n_tokens_mistral': 3566, 'n_tokens_neox': 3089, 'n_words': 914, 'path': 'module3-introduction-to-bayesian-inference/LS_DS_133_Introduction_to_Bayesian_Inference_Assignment.ipynb', 'repo_name': 'CurtCalledBurt/DS-Unit-1-Sprint-3-Statistical-Tests-and-Experiments', 'size': '9142'}