<jupyter_start><jupyter_text>Think BayesSecond EditionCopyright 2020 <NAME>License: [Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0)](https://creativecommons.org/licenses/by-nc-sa/4.0/)<jupyter_code># If we're running on Colab, install empiricaldist
# https://pypi.org/project/empiricaldist/

import sys
IN_COLAB = 'google.colab' in sys.modules

if IN_COLAB:
    !pip install empiricaldist
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

from empiricaldist import Pmf<jupyter_output><empty_output><jupyter_text>The Pmf classI'll start by making a Pmf that represents the outcome of a six-sided die.  Initially there are 6 values with equal probability.<jupyter_code>pmf = Pmf()
for x in [1,2,3,4,5,6]:
    pmf[x] = 1
    
pmf<jupyter_output><empty_output><jupyter_text>To be true probabilities, they have to add up to 1.  So we can normalize the Pmf:<jupyter_code>pmf.normalize()<jupyter_output><empty_output><jupyter_text>The return value from `Normalize` is the sum of the probabilities before normalizing.<jupyter_code>pmf<jupyter_output><empty_output><jupyter_text>A faster way to make a Pmf is to provide a sequence of values.  The constructor adds the values to the Pmf and then normalizes:<jupyter_code>pmf = Pmf.from_seq([1,2,3,4,5,6])
pmf
pmf[1]<jupyter_output><empty_output><jupyter_text>Or you can use the bracket operator.<jupyter_code>pmf(1)<jupyter_output><empty_output><jupyter_text>Either way, if you ask for the probability of something that's not in the Pmf, the result is 0.<jupyter_code>try:
    pmf[7]
except KeyError as e:
    print(e)
pmf(7)
pmf([1,4,7])<jupyter_output><empty_output><jupyter_text>The cookie problemHere's a Pmf that represents the prior distribution.<jupyter_code>prior = Pmf.from_seq(['Bowl 1', 'Bowl 2'])
prior<jupyter_output><empty_output><jupyter_text>And we can update it using `Mult`<jupyter_code>likelihood = [0.75, 0.5]
posterior = prior * likelihood
posterior.normalize()
posterior<jupyter_output><empty_output><jupyter_text>The Bayesian framework<jupyter_code>likelihood_dict = dict(
    vanilla = [0.75, 0.5], 
    chocolate = [0.25, 0.5]
)

def update_cookie(pmf, data):
    pmf *= likelihood_dict[data]
    pmf.normalize()<jupyter_output><empty_output><jupyter_text>We can confirm that we get the same result.<jupyter_code>cookie = Pmf.from_seq(['Bowl 1', 'Bowl 2'])
update_cookie(cookie, 'vanilla')
cookie<jupyter_output><empty_output><jupyter_text>But this implementation is more general; it can handle any sequence of data.<jupyter_code>cookie = Pmf.from_seq(['Bowl 1', 'Bowl 2'])

dataset = ['vanilla', 'chocolate', 'vanilla']
for data in dataset:
    update_cookie(cookie, 'vanilla')
    
cookie<jupyter_output><empty_output><jupyter_text>The Monty Hall problemThe Monty Hall problem might be the most contentious question inthe history of probability.  The scenario is simple, but the correctanswer is so counterintuitive that many people just can't acceptit.Here's the statement of the problem, from [Wikipedia](https://en.wikipedia.org/wiki/Monty_Hall_problem):> Suppose you're on a game show, and you're given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who knows what's behind the doors, opens another door, say No. 3, which has a goat. He then says to you, "Do you want to pick door No. 2?" Is it to your advantage to switch your choice?To avoid ambiguities, we have to make some assumptions about the behavior of the host:1. The host never opens the door you picked.2. The host never opens the door with the car.3. If you choose the door with the car, the host chooses one of the other doors at random.4. The host always offers you the option to switch.Under these assumptions, are you better off sticking or switching?The correct answer is that you are better off switching.  If you stick, you win 1/3 of the time.  If you switch, you win 2/3 of the time.Here's one of many arguments that might persuade you.> If you always stick, you win if you initially choose the door with the car, so the probability is 1/3.>> If you always switch, you win if you did _not_ choose the door with the car, so the probability is 2/3.However, many people do not find any verbal arguments persuasive. Maybe Bayes's Theorem can help.<jupyter_code>hypos = [1, 2, 3]
prior = Pmf(1, hypos)
prior
likelihood = [0.5, 1, 0]<jupyter_output><empty_output><jupyter_text>And here's how we use it.<jupyter_code>posterior = prior * likelihood
posterior.normalize()
posterior<jupyter_output><empty_output><jupyter_text>The M&M problemM&Ms are small candy-coated chocolates that come in a variety ofcolors.  Mars, Inc., which makes M&Ms, changes the mixture ofcolors from time to time.In 1995, they introduced blue M&Ms.  Before then, the color mix ina bag of plain M&Ms was 30% Brown, 20% Yellow, 20% Red, 10%Green, 10% Orange, 10% Tan.  Afterward it was 24% Blue , 20%Green, 16% Orange, 14% Yellow, 13% Red, 13% Brown.Suppose a friend of mine has two bags of M&Ms, and he tells methat one is from 1994 and one from 1996.  He won't tell me which iswhich, but he gives me one M&M from each bag.  One is yellow andone is green.  What is the probability that the yellow one camefrom the 1994 bag?Here's a solution:<jupyter_code>mix94 = dict(brown=30,
                 yellow=20,
                 red=20,
                 green=10,
                 orange=10,
                 tan=10,
                 blue=0)

mix96 = dict(blue=24,
                 green=20,
                 orange=16,
                 yellow=14,
                 red=13,
                 brown=13,
                 tan=0)

hypoA = dict(bag1=mix94, bag2=mix96)
hypoB = dict(bag1=mix96, bag2=mix94)

hypotheses = dict(A=hypoA, B=hypoB)

def likelihood_mm(data, hypo):
    """Computes the likelihood of the data under the hypothesis.

    hypo: string hypothesis (A or B)
    data: tuple of string bag, string color
    """
    bag, color = data
    like = hypotheses[hypo][bag][color]
    return like<jupyter_output><empty_output><jupyter_text>And here's an update:<jupyter_code>data = ('bag1', 'yellow')
hypos = ['A', 'B']

likelihood1 = [likelihood_mm(data, hypo) for hypo in hypos]
likelihood1
def update_mm(pmf, data):
    pmf *= [likelihood_mm(data, hypo) 
            for hypo in hypos]
    pmf.normalize()
pmf = Pmf(1, hypos)
pmf
data = ('bag1', 'yellow')
update_mm(pmf, data)
pmf
data = ('bag2', 'green')
update_mm(pmf, data)
pmf<jupyter_output><empty_output><jupyter_text>**Exercise:**  Suppose you draw another M&M from `bag1` and it's blue.  What can you conclude?  Run the update to confirm your intuition.<jupyter_code>data = ('bag1', 'blue')
update_mm(pmf, data)
pmf<jupyter_output><empty_output><jupyter_text>**Exercise:**  Now suppose you draw an M&M from `bag2` and it's blue.  Run the update to see what happens.  What does that mean?<jupyter_code># Solution

data = ('bag2', 'blue')
update_mm(pmf, data)
pmf
# The unnormalized posterior for both hypotheses is 0, so when
# we try to normalize, the total probability of the data is 0.

# This means that the data have ruled out all of our hypotheses.<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09729579035405632, 'fraction_numerical': 0.023836074714245887, 'mean_word_length': 4.423280423280423, 'pattern_counts': {'":': 0, '<': 74, '<?xml version=': 0, '>': 78, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '18', 'dir': 'jupyter-structured-clean-dedup', 'id': '299106', 'license': 'MIT', 'n_tokens_mistral': 2514, 'n_tokens_neox': 2343, 'n_words': 846, 'path': 'code/soln/chap02soln.ipynb', 'repo_name': 'francescolosterzo/ThinkBayes2', 'size': '7183'}