<jupyter_start><jupyter_text>Lambda School Data Science Module 133 Introduction to Bayesian Inference  Assignment - Code it up!Most of the above was pure math - now write Python code to reproduce the results! This is purposefully open ended - you'll have to think about how you should represent probabilities and events. You can and should look things up, and as a stretch goal - refactor your code into helpful reusable functions!Specific goals/targets:1. Write a function `def prob_drunk_given_positive(prob_drunk_prior, prob_positive, prob_positive_drunk)` that reproduces the example from lecture, and use it to calculate and visualize a range of situations2. Explore `scipy.stats.bayes_mvs` - read its documentation, and experiment with it on data you've tested in other ways earlier this week3. Create a visualization comparing the results of a Bayesian approach to a traditional/frequentist approach4. In your own words, summarize the difference between Bayesian and Frequentist statisticsIf you're unsure where to start, check out [this blog post of Bayes theorem with Python](https://dataconomy.com/2015/02/introduction-to-bayes-theorem-with-python/) - you could and should create something similar!Stretch goals:- Apply a Bayesian technique to a problem you previously worked (in an assignment or project work) on from a frequentist (standard) perspective- Check out [PyMC3](https://docs.pymc.io/) (note this goes beyond hypothesis tests into modeling) - read the guides and work through some examples- Take PyMC3 further - see if you can build something with it! I need to find the probability of drunk(condition positive breath test), which equals "true positive"  * background probability that someone is drunk (prior belief), divided by the overall chance of a positive test. In the breathalyzer case, that's so close to the false positive rate as to make the difference negligible.<jupyter_code>def prob_drunk_given_positive(prob_drunk_prior, prob_positive, prob_positive_drunk):
  numerator = prob_positive * prob_drunk_prior
  print(numerator)
  denominator = (prob_positive * prob_drunk_prior) + ((1 - prob_positive) * (1-prob_drunk_prior))
  print(denominator)
  return numerator / denominator
  
def prob_positive(prior_belief, false_pos, true_pos):
  pos_pos = true_pos * prior_belief
  pos_neg = false_pos * (1 - prior_belief)
  return pos_pos + pos_neg
  
def bayes_theorem(prior_belief, true_pos, false_pos):
  numerator = prior_belief * true_pos
  denominator = (prior_belief * true_pos) + (false_pos * (1-prior_belief))
  return numerator/denominator
#Let's say true positive rate is 95%, false positive rate is 10% (based on manufacturer's stats)
#research says there are 111 million self-reported incidences of drunk driving in a year, and there are 225 million licensed drivers in the US. 
#If i work from the very basic assumption that everyone with a license drives twice per day 
#(which balances out people who hardly drive against people who make several trips)
#that would mean my prior belief should be 0.00067 (total drives per year divided by drunk driving reports)
bayes_theorem(0.00067, 0.95, 0.1)
prob_pos = prob_positive(0.00067, 0.10, 0.95)
prob_drunk_given_positive(0.00067,0.10, 0.95)<jupyter_output><empty_output><jupyter_text>That would mean only a 0.6% chance that a driver is drunk given a positive breath test!But that's assuming total randomness of drunk driving vs total driving, which is of course not true.<jupyter_code>#lets use the same numbers for true and false positives, and assume 5% of our drivers are drunk (it's 1AM on a Saturday)
bayes_theorem(0.05,0.95, 0.1)
#let's look at a re-test scenario - the chance of a false positive TWICE is only 10% * 10%, or 1%

bayes_theorem(0.05, 0.95, 0.01)
bayes_theorem(0.05, 1, 0.08)<jupyter_output><empty_output><jupyter_text>Iterate bayes theorem<jupyter_code>def iterate_theorem(prior_belief, true_pos, false_pos, iterations):
  new_prior = prior_belief
  for i in range(iterations):
    numerator = new_prior * true_pos
    denominator = (new_prior * true_pos) + (false_pos * (1-prior_belief))
    new_prior = numerator/denominator
    print(new_prior)
iterate_theorem(0.01, 0.95, 0.10, 2)
iterate_theorem(0.05, 0.95, 0.1, 3)
bayes_theorem(0.05, 0.95, 0.1)
bayes_theorem(0.3333333333333333, 0.95, 0.1)
bayes_theorem(0.8260869565217391, 0.95, 0.1)
iterate_theorem(0.001, 0.99, 0.01, 6)<jupyter_output>0.09016393442622951
0.899347415676889
0.9889042403249438
0.9898989408539074
0.9899089884847672
0.9899090898755362<jupyter_text>Apply bayesian to previous data<jupyter_code>import pandas as pd
import numpy as np

url = 'https://archive.ics.uci.edu/ml/machine-learning-databases/voting-records/house-votes-84.data'

df = pd.read_csv(url, header=None)

columns = ['Class Name', 'handicapped-infants', 
           'water-project-cost-sharing',
           'adoption-of-the-budget-resolution',
           'physician-fee-freeze',
           'el-salvador-aid',
           'religious-groups-in-schools',
           'anti-satellite-test-ban', 
           'aid-to-nicaraguan-contras',
           'mx-missile',
           'immigration',
           'synfuels-corporation-cutback',
           'education-spending',
           'superfund-right-to-sue',
           'crime',
           'duty-free-exports',
           'export-administration-act-south-africa']
df.columns = columns
df = df.replace(('y','n', '?'), (1, 0, np.NaN))
df=df.fillna(df.mean())
dem_mask = df['Class Name'] == 'democrat'
df_dems = df[dem_mask]
rep_mask = df['Class Name'] == 'republican'
df_reps = df[rep_mask]

from scipy import stats
rep_arr_crime = df_reps['crime'].values
dem_arr_crime = df_dems['crime'].values
rep_arr_crime
stats.bayes_mvs(rep_arr_crime, alpha=0.95)
stats.bayes_mvs(dem_arr_crime, alpha=0.95)
def bayesian_on_data_column(dataframe, column, alpha=0.95):
  df = dataframe
  arr = df[column].values
  return stats.bayes_mvs(arr, alpha)
bayesian_on_data_column(df_dems, 'crime', 0.95)
for i in range(len(df.columns)-1):
  col = df_dems.iloc[:, i]
  bayesian_on_data_column(df_dems, col)<jupyter_output><empty_output><jupyter_text>Bayesian Vs FrequentistThe difference between frequentist and Bayesian statistics is that frequentist statistics is based on how frequently an event has occurred over the long-term, without attaching a probability to any hypothesis or unknown value. Frequentists define "probability" as the number of times that a repeatable random result will occur (ie, a coin flip) over a very large set of events. Bayesian statistics attaches a probability to the uncertainty of any given hypothesis or event, such as an election. This is different because you can't re-run the event to see the result over a large number of chances.<jupyter_code><jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08723247232472325, 'fraction_numerical': 0.045018450184501846, 'mean_word_length': 4.74724342663274, 'pattern_counts': {'":': 0, '<': 20, '<?xml version=': 0, '>': 20, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '192141', 'license': 'MIT', 'n_tokens_mistral': 2372, 'n_tokens_neox': 2078, 'n_words': 719, 'path': 'module3-introduction-to-bayesian-inference/Cai_Nowicki_LS_DS_133_Introduction_to_Bayesian_Inference_Assignment.ipynb', 'repo_name': 'dunkelweizen/DS-Unit-1-Sprint-3-Statistical-Tests-and-Experiments', 'size': '6775'}