<jupyter_start><jupyter_text>Tutorial 1: Bayes with a binary hidden state**Week 3, Day 1: Bayesian Decisions****By Neuromatch Academy**__Content creators:__ [insert your name here]__Content reviewers:__   Tutorial ObjectivesThis is the first in a series of two core tutorials on Bayesian statistics. In these tutorials, we will explore the fundemental concepts of the Bayesian approach from two perspectives. This tutorial will work through an example of Bayesian inference and decision making using a binary hidden state. The second main tutorial extends these concepts to a continuous hidden state. In the next days, each of these basic ideas will be extended--first through time as we consider what happens when we infere a hidden state using multiple observations and when the hidden state changes across time. In the third day, we will introduce the notion of how to use inference and decisions to select actions for optimal control. For this tutorial, you will be introduced to our binary state fishing problem!This notebook will introduce the fundamental building blocks for Bayesian statistics: 1. How do we use probability distributions to represent hidden states?2. How does marginalization work and how can we use it?3. How do we combine new information with our prior knowledge?4. How do we combine the possible loss (or gain) for making a decision with our probabilitic knowledge?<jupyter_code>#@title Video 1: Introduction to Bayesian Statistics
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='JiEIn9QsrFg', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video<jupyter_output><empty_output><jupyter_text>Setup  Please execute the cells below to initialize the notebook environment.<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
from matplotlib import patches
from matplotlib import transforms
from matplotlib import gridspec
from scipy.optimize import fsolve

from collections import namedtuple
#@title Figure Settings
import ipywidgets as widgets       # interactive display
from ipywidgets import GridspecLayout
from IPython.display import clear_output
%config InlineBackend.figure_format = 'retina'
plt.style.use("https://raw.githubusercontent.com/NeuromatchAcademy/course-content/master/nma.mplstyle")

import warnings
warnings.filterwarnings("ignore")
# @title Plotting Functions

def plot_joint_probs(P, ):
    assert np.all(P >= 0), "probabilities should be >= 0"
    # normalize if not
    P = P / np.sum(P)
    marginal_y = np.sum(P,axis=1)
    marginal_x = np.sum(P,axis=0)

    # definitions for the axes
    left, width = 0.1, 0.65
    bottom, height = 0.1, 0.65
    spacing = 0.005

    # start with a square Figure
    fig = plt.figure(figsize=(5, 5))

    joint_prob = [left, bottom, width, height]
    rect_histx = [left, bottom + height + spacing, width, 0.2]
    rect_histy = [left + width + spacing, bottom, 0.2, height]

    rect_x_cmap = plt.cm.Blues
    rect_y_cmap = plt.cm.Reds

    # Show joint probs and marginals
    ax = fig.add_axes(joint_prob)
    ax_x = fig.add_axes(rect_histx, sharex=ax)
    ax_y = fig.add_axes(rect_histy, sharey=ax)

    # Show joint probs and marginals
    ax.matshow(P,vmin=0., vmax=1., cmap='Greys')
    ax_x.bar(0, marginal_x[0], facecolor=rect_x_cmap(marginal_x[0]))
    ax_x.bar(1, marginal_x[1], facecolor=rect_x_cmap(marginal_x[1]))
    ax_y.barh(0, marginal_y[0], facecolor=rect_y_cmap(marginal_y[0]))
    ax_y.barh(1, marginal_y[1], facecolor=rect_y_cmap(marginal_y[1]))
    # set limits
    ax_x.set_ylim([0,1])
    ax_y.set_xlim([0,1])

    # show values
    ind = np.arange(2)
    x,y = np.meshgrid(ind,ind)
    for i,j in zip(x.flatten(), y.flatten()):
        c = f"{P[i,j]:.2f}"
        ax.text(j,i, c, va='center', ha='center', color='black')
    for i in ind:
        v = marginal_x[i]
        c = f"{v:.2f}"
        ax_x.text(i, v +0.1, c, va='center', ha='center', color='black')
        v = marginal_y[i]
        c = f"{v:.2f}"
        ax_y.text(v+0.2, i, c, va='center', ha='center', color='black')

    # set up labels
    ax.xaxis.tick_bottom()
    ax.yaxis.tick_left()
    ax.set_xticks([0,1])
    ax.set_yticks([0,1])
    ax.set_xticklabels(['Silver','Gold'])
    ax.set_yticklabels(['Small', 'Large'])
    ax.set_xlabel('color')
    ax.set_ylabel('size')
    ax_x.axis('off')
    ax_y.axis('off')
    return fig
# test
# P = np.random.rand(2,2)
# P = np.asarray([[0.9, 0.8], [0.4, 0.1]])
# P = P / np.sum(P)
# fig = plot_joint_probs(P)
# plt.show(fig)
# plt.close(fig)


# fig = plot_prior_likelihood(0.5, 0.3)
# plt.show(fig)
# plt.close(fig)


def plot_prior_likelihood_posterior(prior, likelihood, posterior):

    # definitions for the axes
    left, width = 0.05, 0.3
    bottom, height = 0.05, 0.9
    padding = 0.1
    small_width = 0.1
    left_space = left + small_width + padding
    added_space = padding + width

    fig = plt.figure(figsize=(10, 4))

    rect_prior = [left, bottom, small_width, height]
    rect_likelihood = [left_space , bottom , width, height]
    rect_posterior = [left_space +  added_space, bottom , width, height]

    ax_prior = fig.add_axes(rect_prior)
    ax_likelihood = fig.add_axes(rect_likelihood, sharey=ax_prior)
    ax_posterior = fig.add_axes(rect_posterior, sharey = ax_prior)

    rect_colormap = plt.cm.Blues

    # Show posterior probs and marginals
    ax_prior.barh(0, prior[0], facecolor = rect_colormap(prior[0, 0]))
    ax_prior.barh(1, prior[1], facecolor = rect_colormap(prior[1, 0]))
    ax_likelihood.matshow(likelihood, vmin=0., vmax=1., cmap='Reds')
    ax_posterior.matshow(posterior, vmin=0., vmax=1., cmap='Greens')


    # Probabilities plot details
    ax_prior.set(xlim = [1, 0], yticks = [0, 1], yticklabels = ['left', 'right'],
                 ylabel = 'state (s)', title = "Prior p(s)")
    ax_prior.axis('off')

    # Likelihood plot details
    ax_likelihood.set(xticks = [0, 1], xticklabels = ['fish', 'no fish'],
                  yticks = [0, 1], yticklabels = ['left', 'right'],
                   ylabel = 'state (s)', xlabel = 'measurement (m)',
                   title = 'Likelihood p(m (right) | s)')
    ax_likelihood.xaxis.set_ticks_position('bottom')
    ax_likelihood.spines['left'].set_visible(False)
    ax_likelihood.spines['bottom'].set_visible(False)

    # Posterior plot details

    ax_posterior.set(xticks = [0, 1], xticklabels = ['fish', 'no fish'],
                  yticks = [0, 1], yticklabels = ['left', 'right'],
                   ylabel = 'state (s)', xlabel = 'measurement (m)',
                   title = 'Posterior p(s | m)')
    ax_posterior.xaxis.set_ticks_position('bottom')
    ax_posterior.spines['left'].set_visible(False)
    ax_posterior.spines['bottom'].set_visible(False)


    # show values
    ind = np.arange(2)
    x,y = np.meshgrid(ind,ind)
    for i,j in zip(x.flatten(), y.flatten()):
        c = f"{posterior[i,j]:.2f}"
        ax_posterior.text(j,i, c, va='center', ha='center', color='black')
    for i,j in zip(x.flatten(), y.flatten()):
        c = f"{likelihood[i,j]:.2f}"
        ax_likelihood.text(j,i, c, va='center', ha='center', color='black')
    for i in ind:
        v = prior[i, 0]
        c = f"{v:.2f}"
        ax_prior.text(v+0.2, i, c, va='center', ha='center', color='black')


def plot_prior_likelihood(ps, p_a_s1, p_a_s0, measurement):
    likelihood = np.asarray([[p_a_s1, 1-p_a_s1],[p_a_s0, 1-p_a_s0]])
    assert 0.0 <= ps <= 1.0
    prior = np.asarray([ps, 1 - ps])
    if measurement:
        posterior = likelihood[:, 0] * prior
    else:
        posterior = (likelihood[:, 1] * prior).reshape(-1)
    posterior /= np.sum(posterior)

    # definitions for the axes
    left, width = 0.05, 0.3
    bottom, height = 0.05, 0.9
    padding = 0.1
    small_width = 0.22
    left_space = left + small_width + padding
    small_padding = 0.05

    fig = plt.figure(figsize=(10, 4))

    rect_prior = [left, bottom, small_width, height]
    rect_likelihood = [left_space , bottom , width, height]
    rect_posterior = [left_space + width + small_padding, bottom , small_width, height]

    ax_prior = fig.add_axes(rect_prior)
    ax_likelihood = fig.add_axes(rect_likelihood, sharey=ax_prior)
    ax_posterior = fig.add_axes(rect_posterior, sharey=ax_prior)

    prior_colormap = plt.cm.Blues
    posterior_colormap = plt.cm.Greens

    # Show posterior probs and marginals
    ax_prior.barh(0, prior[0], facecolor = prior_colormap(prior[0]))
    ax_prior.barh(1, prior[1], facecolor = prior_colormap(prior[1]))
    ax_likelihood.matshow(likelihood, vmin=0., vmax=1., cmap='Reds')
    # ax_posterior.matshow(posterior, vmin=0., vmax=1., cmap='')
    ax_posterior.barh(0, posterior[0], facecolor = posterior_colormap(posterior[0]))
    ax_posterior.barh(1, posterior[1], facecolor = posterior_colormap(posterior[1]))

    # Probabilities plot details
    ax_prior.set(xlim = [1, 0], yticks = [0, 1], yticklabels = ['left', 'right'],
                 ylabel = 'state (s)', title = "Prior p(s)")
    ax_prior.axis('off')

    # Likelihood plot details
    ax_likelihood.set(xticks = [0, 1], xticklabels = ['fish', 'no fish'],
                  yticks = [0, 1], yticklabels = ['left', 'right'],
                   ylabel = 'state (s)', xlabel = 'measurement (m)',
                   title = 'Likelihood p(m | s)')
    ax_likelihood.xaxis.set_ticks_position('bottom')
    ax_likelihood.spines['left'].set_visible(False)
    ax_likelihood.spines['bottom'].set_visible(False)

    # Posterior plot details
    ax_posterior.set(xlim = [0, 1], yticks = [0, 1], yticklabels = ['left', 'right'],
                 ylabel = 'state (s)', title = "Posterior p(s | m)")
    ax_posterior.axis('off')
    # ax_posterior.set(xticks = [0, 1], xticklabels = ['fish', 'no fish'],
    #               yticks = [0, 1], yticklabels = ['left', 'right'],
    #                ylabel = 'state (s)', xlabel = 'measurement (m)',
    #                title = 'Posterior p(s | m)')
    # ax_posterior.xaxis.set_ticks_position('bottom')
    # ax_posterior.spines['left'].set_visible(False)
    # ax_posterior.spines['bottom'].set_visible(False)


    # show values
    ind = np.arange(2)
    x,y = np.meshgrid(ind,ind)
    # for i,j in zip(x.flatten(), y.flatten()):
    #     c = f"{posterior[i,j]:.2f}"
    #     ax_posterior.text(j,i, c, va='center', ha='center', color='black')
    for i in ind:
        v = posterior[i]
        c = f"{v:.2f}"
        ax_posterior.text(v+0.2, i, c, va='center', ha='center', color='black')
    for i,j in zip(x.flatten(), y.flatten()):
        c = f"{likelihood[i,j]:.2f}"
        ax_likelihood.text(j,i, c, va='center', ha='center', color='black')
    for i in ind:
        v = prior[i]
        c = f"{v:.2f}"
        ax_prior.text(v+0.2, i, c, va='center', ha='center', color='black')
    return fig

# fig = plot_prior_likelihood(0.5, 0.3)
# plt.show(fig)
# plt.close(fig)


from matplotlib import colors
def plot_utility(ps):
    prior = np.asarray([ps, 1 - ps])

    utility = np.array([[2, -3], [-2, 1]])

    expected = prior @ utility

    # definitions for the axes
    left, width = 0.05, 0.16
    bottom, height = 0.05, 0.9
    padding = 0.04
    small_width = 0.1
    left_space = left + small_width + padding
    added_space = padding + width

    fig = plt.figure(figsize=(17, 3))

    rect_prior = [left, bottom, small_width, height]
    rect_utility = [left + added_space , bottom , width, height]
    rect_expected = [left + 2* added_space, bottom , width, height]

    ax_prior = fig.add_axes(rect_prior)
    ax_utility = fig.add_axes(rect_utility, sharey=ax_prior)
    ax_expected = fig.add_axes(rect_expected)

    rect_colormap = plt.cm.Blues

    # Data of plots
    ax_prior.barh(0, prior[0], facecolor = rect_colormap(prior[0]))
    ax_prior.barh(1, prior[1], facecolor = rect_colormap(prior[1]))
    ax_utility.matshow(utility, cmap='cool')
    norm = colors.Normalize(vmin=-3, vmax=3)
    ax_expected.bar(0, expected[0], facecolor = rect_colormap(norm(expected[0])))
    ax_expected.bar(1, expected[1], facecolor = rect_colormap(norm(expected[1])))

    # Probabilities plot details
    ax_prior.set(xlim = [1, 0], yticks = [0, 1], yticklabels = ['left', 'right'],
                 ylabel = 'state (s)', title = "Probability of state")
    ax_prior.axis('off')

    # Utility plot details
    ax_utility.set(xticks = [0, 1], xticklabels = ['left', 'right'],
                  yticks = [0, 1], yticklabels = ['left', 'right'],
                   ylabel = 'state (s)', xlabel = 'action (a)',
                   title = 'Utility')
    ax_utility.xaxis.set_ticks_position('bottom')
    ax_utility.spines['left'].set_visible(False)
    ax_utility.spines['bottom'].set_visible(False)

    # Expected utility plot details
    ax_expected.set(title = 'Expected utility', ylim = [-3, 3],
                    xticks = [0, 1], xticklabels = ['left', 'right'],
                    xlabel = 'action (a)',
                    yticks = [])
    ax_expected.xaxis.set_ticks_position('bottom')
    ax_expected.spines['left'].set_visible(False)
    ax_expected.spines['bottom'].set_visible(False)

    # show values
    ind = np.arange(2)
    x,y = np.meshgrid(ind,ind)

    for i,j in zip(x.flatten(), y.flatten()):
        c = f"{utility[i,j]:.2f}"
        ax_utility.text(j,i, c, va='center', ha='center', color='black')
    for i in ind:
        v = prior[i]
        c = f"{v:.2f}"
        ax_prior.text(v+0.2, i, c, va='center', ha='center', color='black')
    for i in ind:
        v = expected[i]
        c = f"{v:.2f}"
        ax_expected.text(i, 2.5, c, va='center', ha='center', color='black')

    return fig


def plot_prior_likelihood_utility(ps, p_a_s1, p_a_s0,measurement):
    assert 0.0 <= ps <= 1.0
    assert 0.0 <= p_a_s1 <= 1.0
    assert 0.0 <= p_a_s0 <= 1.0
    prior = np.asarray([ps, 1 - ps])
    likelihood = np.asarray([[p_a_s1, 1-p_a_s1],[p_a_s0, 1-p_a_s0]])
    utility = np.array([[2.0, -3.0], [-2.0, 1.0]])
    # expected = np.zeros_like(utility)

    if measurement:
        posterior = likelihood[:, 0] * prior
    else:
        posterior = (likelihood[:, 1] * prior).reshape(-1)
    posterior /= np.sum(posterior)
    # expected[:, 0] = utility[:, 0] * posterior
    # expected[:, 1] = utility[:, 1] * posterior
    expected = posterior @ utility

    # definitions for the axes
    left, width = 0.05, 0.15
    bottom, height = 0.05, 0.9
    padding = 0.05
    small_width = 0.1
    large_padding = 0.07
    left_space = left + small_width + large_padding

    fig = plt.figure(figsize=(17, 4))

    rect_prior = [left, bottom+0.05, small_width, height-0.1]
    rect_likelihood = [left_space, bottom , width, height]
    rect_posterior = [left_space + padding + width - 0.02, bottom+0.05 , small_width, height-0.1]
    rect_utility = [left_space + padding + width + padding + small_width, bottom , width, height]
    rect_expected = [left_space + padding + width + padding + small_width + padding + width, bottom+0.05 , width, height-0.1]

    ax_likelihood = fig.add_axes(rect_likelihood)
    ax_prior = fig.add_axes(rect_prior, sharey=ax_likelihood)
    ax_posterior = fig.add_axes(rect_posterior, sharey=ax_likelihood)
    ax_utility = fig.add_axes(rect_utility, sharey=ax_posterior)
    ax_expected = fig.add_axes(rect_expected)


    prior_colormap = plt.cm.Blues
    posterior_colormap = plt.cm.Greens
    expected_colormap = plt.cm.Wistia

    # Show posterior probs and marginals
    ax_prior.barh(0, prior[0], facecolor = prior_colormap(prior[0]))
    ax_prior.barh(1, prior[1], facecolor = prior_colormap(prior[1]))
    ax_likelihood.matshow(likelihood, vmin=0., vmax=1., cmap='Reds')
    ax_posterior.barh(0, posterior[0], facecolor = posterior_colormap(posterior[0]))
    ax_posterior.barh(1, posterior[1], facecolor = posterior_colormap(posterior[1]))
    ax_utility.matshow(utility, vmin=0., vmax=1., cmap='cool')
    # ax_expected.matshow(expected, vmin=0., vmax=1., cmap='Wistia')
    ax_expected.bar(0, expected[0], facecolor = expected_colormap(expected[0]))
    ax_expected.bar(1, expected[1], facecolor = expected_colormap(expected[1]))

    # Probabilities plot details
    ax_prior.set(xlim = [1, 0], yticks = [0, 1], yticklabels = ['left', 'right'],
                 ylabel = 'state (s)', title = "Prior p(s)")
    ax_prior.axis('off')

    # Likelihood plot details
    ax_likelihood.set(xticks = [0, 1], xticklabels = ['fish', 'no fish'],
                  yticks = [0, 1], yticklabels = ['left', 'right'],
                   ylabel = 'state (s)', xlabel = 'measurement (m)',
                   title = 'Likelihood p(m | s)')
    ax_likelihood.xaxis.set_ticks_position('bottom')
    ax_likelihood.spines['left'].set_visible(False)
    ax_likelihood.spines['bottom'].set_visible(False)

    # Posterior plot details
    ax_posterior.set(xlim = [0, 1], yticks = [0, 1], yticklabels = ['left', 'right'],
                 ylabel = 'state (s)', title = "Posterior p(s | m)")
    ax_posterior.axis('off')

    # Utility plot details
    ax_utility.set(xticks = [0, 1], xticklabels = ['left', 'right'],
                   xlabel = 'action (a)',
                   title = 'Utility')
    ax_utility.xaxis.set_ticks_position('bottom')
    ax_utility.spines['left'].set_visible(False)
    ax_utility.spines['bottom'].set_visible(False)

    # Expected Utility plot details
    ax_expected.set(ylim = [-2, 2], xticks = [0, 1], xticklabels = ['left', 'right'],
                 xlabel = 'action (a)', title = 'Expected utility', yticks=[])
    # ax_expected.axis('off')
    ax_expected.spines['left'].set_visible(False)
    # ax_expected.set(xticks = [0, 1], xticklabels = ['left', 'right'],
    #                 xlabel = 'action (a)',
    #                title = 'Expected utility')
    # ax_expected.xaxis.set_ticks_position('bottom')
    # ax_expected.spines['left'].set_visible(False)
    # ax_expected.spines['bottom'].set_visible(False)

    # show values
    ind = np.arange(2)
    x,y = np.meshgrid(ind,ind)
    for i in ind:
        v = posterior[i]
        c = f"{v:.2f}"
        ax_posterior.text(v+0.2, i, c, va='center', ha='center', color='black')
    for i,j in zip(x.flatten(), y.flatten()):
        c = f"{likelihood[i,j]:.2f}"
        ax_likelihood.text(j,i, c, va='center', ha='center', color='black')
    for i,j in zip(x.flatten(), y.flatten()):
        c = f"{utility[i,j]:.2f}"
        ax_utility.text(j,i, c, va='center', ha='center', color='black')
    # for i,j in zip(x.flatten(), y.flatten()):
    #     c = f"{expected[i,j]:.2f}"
    #     ax_expected.text(j,i, c, va='center', ha='center', color='black')
    for i in ind:
        v = prior[i]
        c = f"{v:.2f}"
        ax_prior.text(v+0.2, i, c, va='center', ha='center', color='black')
    for i in ind:
        v = expected[i]
        c = f"{v:.2f}"
        ax_expected.text(i, v, c, va='center', ha='center', color='black')

    # # show values
    # ind = np.arange(2)
    # x,y = np.meshgrid(ind,ind)
    # for i,j in zip(x.flatten(), y.flatten()):
    #     c = f"{P[i,j]:.2f}"
    #     ax.text(j,i, c, va='center', ha='center', color='white')
    # for i in ind:
    #     v = marginal_x[i]
    #     c = f"{v:.2f}"
    #     ax_x.text(i, v +0.2, c, va='center', ha='center', color='black')
    #     v = marginal_y[i]
    #     c = f"{v:.2f}"
    #     ax_y.text(v+0.2, i, c, va='center', ha='center', color='black')

    return fig
# @title Helper Functions

def compute_marginal(px, py, cor):
    # calculate 2x2 joint probabilities given marginals p(x=1), p(y=1) and correlation
    p11 = px*py + cor*np.sqrt(px*py*(1-px)*(1-py))
    p01 = px - p11
    p10 = py - p11
    p00 = 1.0 - p11 - p01 - p10
    return np.asarray([[p00, p01], [p10, p11]])
# test
# print(compute_marginal(0.4, 0.6, -0.8))


def compute_cor_range(px,py):
    # Calculate the allowed range of correlation values given marginals p(x=1) and p(y=1)
    def p11(corr):
        return px*py + corr*np.sqrt(px*py*(1-px)*(1-py))
    def p01(corr):
        return px - p11(corr)
    def p10(corr):
        return py - p11(corr)
    def p00(corr):
        return 1.0 - p11(corr) - p01(corr) - p10(corr)
    Cmax = min(fsolve(p01, 0.0), fsolve(p10, 0.0))
    Cmin = max(fsolve(p11, 0.0), fsolve(p00, 0.0))
    return Cmin, Cmax<jupyter_output><empty_output><jupyter_text>--- Section 1: <NAME>'<jupyter_code>#@title Video 2: <NAME>'
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='McALsTzb494', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video<jupyter_output><empty_output><jupyter_text>You were just introduced to the **binary hidden state problem** we are going to explore. You need to decide which side to fish on. We know fish like to school together. On different days the school of fish is either on the left or right side, but we don’t know what the case is today. We will represent our knowledge probabilistically, asking how to make a decision (where to decide the fish are or where to fish) and what to expect in terms of gains or losses. In the next two sections we will consider just the probability of where the fish might be and what you gain or lose by choosing where to fish.Remember, you can either think of your self as a scientist conducting an experiment or as a brain trying to make a decision. The Bayesian approach is the same! --- Section 2: Deciding where to fish<jupyter_code>#@title Video 3: Utility
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='xvIVZrqF_5s', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video<jupyter_output><empty_output><jupyter_text>You know the probability that the school of fish is on the left side of the dock today, $P(s = left)$. You also know the probability that it is on the right, $P(s = right)$, because these two probabilities must add up to 1. You need to decide where to fish. It may seem obvious - you could just fish on the side where the probability of the fish being is higher! Unfortunately, decisions and actions are always a little more complicated. Deciding to fish may be influenced by more than just the probability of the school of fish being there as we saw by the potential issues of submarines and sunburn. We quantify these factors numerically using **utility**, which describes the consequences of your actions: how much value you gain (or if negative, lose) given the state of the world ($s$) and the action you take ($a$). In our example, our utility can be summarized as:| Utility: U(s,a)   | a = left   | a = right  || ----------------- |----------|----------|| s = Left          | 2          | -3         || s = right         | -2         |  1         |To use utility to choose an action, we calculate the **expected utility** of that action by weighing these utilities with the probability of that state occuring. This allows us to choose actions by taking probabilities of events into account: we don't care if the outcome of an action-state pair is a loss if the probability of that state is very low. We can formalize this as:$$\text{Expected utility of action a} = \sum_{s}U(s,a)P(s) $$In other words, the expected utility of an action a is the sum over possible states of the utility of that action and state times the probability of that state.  Interactive Demo 2: Exploring the decisionLet's start to get a sense of how all this works. Take a look at the interactive demo below. You can change the probability that the school of fish is on the left side ($p(s = left)$ using the slider. You will see the utility matrix and the corresponding expected utility of each action.First, make sure you understand how the expected utility of each action is being computed from the probabilities and the utility values. In the initial state: the probability of the fish being on the left is 0.9 and on the right is 0.1. The expected utility of the action of fishing on the left is then $U(s = left,a = left)p(s = left) + U(s = right,a = left)p(s = right) = 2(0.9) + -2(0.1) = 1.6$.For each of these scenarios, think and discuss first. Then use the demo to try out each and see if your action would have been correct (that is, if the expected value of that action is the highest).1.  You just arrived at the dock for the first time and have no sense of where the fish might be. So you guess that the probability of the school being on the left side is 0.5 (so the probability on the right side is also 0.5). Which side would you choose to fish on given our utility values?2.  You think that the probability of the school being on the left side is very low (0.1) and correspondingly high on the right side (0.9). Which side would you choose to fish on given our utility values?3.  What would you choose if the probability of the school being on the left side is slightly lower than on the right side (0. 4 vs 0.6)?<jupyter_code># @markdown Execute this cell to use the widget
ps_widget = widgets.FloatSlider(0.9, description='p(s = left)', min=0.0, max=1.0, step=0.01)

@widgets.interact(
    ps = ps_widget,
)
def make_utility_plot(ps):
    fig = plot_utility(ps)
    plt.show(fig)
    plt.close(fig)
    return None
# to_remove explanation

# 1)  With equal probabilities, the expected utility is higher on the left side,
#    since that is the side without submarines, so you would choose to fish there.

# 2)  If the probability that the fish is on the right side is high, you would
#     choose to fish there. The high probability of fish being on the right far outweights
#    the slightly higher utilities from fishing on the left (as you are unlikely to gain these)

# 3)  If the probability that the fish is on the right side is just slightly higher
#.    than on the left, you would choose the left side as the expected utility is still
#.    higher on the left. Note that in this situation, you are not simply choosing the
#.    side with the higher probability - the utility really matters here for the decision<jupyter_output><empty_output><jupyter_text>In this section, you have seen that both the utility of various state and action pairs and our knowledge of the probability of each state affects your decision. Importantly, we want our knowledge of the probability of each state to be as accurate as possible! So how do we know these probabilities? We may have prior knowledge from years of fishing at the same dock. Over those years, we may have learned that the fish are more likely to be on the left side for example. We want to make sure this knowledge is as accurate as possible though. To do this, we want to collect more data, or take some more measurements! For the next few sections, we will focus on making our knowledge of the probability as accurate as possible, before coming back to using utility to make decisions. --- Section 3: Likelihood of the fish being on either side<jupyter_code>#@title Video 4: Likelihood
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='l4m0JzMWGio', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video<jupyter_output><empty_output><jupyter_text>First, we'll think about what it means to take a measurement (also often called an observation or just data) and what it tells you about what the hidden state may be. Specifically, we'll be looking at the **likelihood**, which is the probability of your measurement ($m$) given the hidden state ($s$): $P(m | s)$. Remember that in this case, the hidden state is which side of the dock the school of fish is on.We will watch someone fish (for let's say 10 minutes) and our measurement is whether they catch a fish or not. We know something about what catching a fish means for the likelihood of the fish being on one side or the other.   Think! 3: Guessing the location of the fishLet's say we go to different dock from the one in the video. Here, there are different probabilities of catching fish given the state of the world. In this case, if they fish on the side of the dock where the fish are, they have a 70% chance of catching a fish. Otherwise, they catch a fish with only 20% probability. The fisherperson is fishing on the left side. 1) Figure out each of the following:- probability of catching a fish given that the school of fish is on the left side, $P(m = catch\text{ } fish | s = left )$- probability of not catching a fish given that the school of fish is on the left side, $P(m = no \text{ } fish | s = left)$- probability of catching a fish given that the school of fish is on the right side, $P(m = catch  \text{ } fish | s = right)$- probability of not catching a fish given that the school of fish is on the right side, $P(m = no \text{ } fish | s = right)$2) If the fisherperson catches a fish, which side would you guess the school is on? Why?3) If the fisherperson does not catch a fish, which side would you guess the school is on? Why?<jupyter_code>#to_remove explanation

# 1) The fisherperson is on the left side so:
#       - P(m = catch fish | s = left) = 0.7 because they have a 70% chance of catching
#         a fish when on the same side as the school
#       - P(m = no fish | s = left) = 0.3 because the probability of catching a fish
#         and not catching a fish for a given state must add up to 1 as these
#         are the only options: 1 - 0.7 = 0.3
#       - P(m = catch fish | s = right) = 0.2
#       - P(m = no fish | s = right) = 0.8

# 2) If the fisherperson catches a fish, you would guess the school of fish is on the
#     left side. This is because the probability of catching a fish given that the
#    school is on the left side (0.7) is higher than the probability given that
#    the school is on the right side (0.2).

# 3) If the fisherperson does not catch a fish, you would guess the school of fish is on the
#     right side. This is because the probability of not catching a fish given that the
#    school is on the right side (0.8) is higher than the probability given that
#    the school is on the right side (0.3).<jupyter_output><empty_output><jupyter_text>In the prior exercise, you guessed where the school of fish was based on the measurement you took (watching someone fish). You did this by choosing the state (side of school) that maximized the probability of the measurement. In other words, you estimated the state by maximizing the likelihood (had the highest probability of measurement given state $P(m|s$)). This is called maximum likelihood estimation (MLE) and you've encountered it before during this course, in W1D3!What if you had been going to this river for years and you knew that the fish were almost always on the left side? This would probably affect how you make your estimate - you would rely less on the single new measurement and more on your prior knowledge. This is the idea behind Bayesian inference, as we will see later in this tutorial! --- Section 4: Correlation and marginalization<jupyter_code>#@title Video 5: Correlation and marginalization
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='vsDjtWi-BVo', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video<jupyter_output><empty_output><jupyter_text>In this section, we are going to take a step back for a bit and think more generally about the amount of information shared between two random variables. We want to know how much information you gain when you observe one variable (take a measurement) if you know something about another. We will see that the fundamental concept is the same if we think about two attributes, for example the size and color of the fish, or the prior information and the likelihood.  Math Exercise 4: Computing marginal likelihoodsTo understand the information between two variables, let's first consider the size and color of the fish.| P(X, Y)   | Y = silver   | Y = gold  || ----------------- |----------|----------|| X = small          | 0.4          | 0.2         || X = large         | 0.1         |  0.3         |The table above shows us the **joint probabilities**: the probability of both specific attributes occuring together. For example, the probability of a fish being small and silver ($P(X = small, Y = silver$) is 0.4.We want to know what the probability of a fish being small  regardless of color. Since the fish are either silver or gold, this would be the probability of a fish being small and silver plus the probability of a fish being small and gold. This is an example of marginalizing, or averaging out, the variable we are not interested in across the rows or columns.. In math speak: $P(X = small) = \sum_y{P(X = small, Y)}$. This gives us a **marginal probability**, a probability of a variable outcome (in this case size), regardless of the other variables (in this case color).Please complete the following math problems to further practice thinking through probabilities:1. Calculate the probability of a fish being silver.2. Calculate the probability of a fish being small, large, silver, or gold.3. Calculate the probability of a fish being small OR gold. (Hint: $P(A\ \textrm{or}\ B) = P(A) + P(B) - P(A\ \textrm{and}\ B)$)<jupyter_code># to_remove explanation

# 1) The probability of a fish being silver is the joint probability of it being
#.     small and silver plus the joint probability of it being large and silver:
#
#.    P(Y = silver) = P(X = small, Y = silver) + P(X = large, Y = silver)
#.     = 0.4 + 0.1
#.     = 0.5


# 2) This is all the possibilities as in this scenario, our fish can only be small
#.   or large, silver or gold. So the probability is 1 - the fish has to be at
#.   least one of these.


#. 3) First we compute the marginal probabilities
#.  P(X = small) = P(X = small, Y = silver) + P(X = small, Y = gold) = 0.6
#.  P(Y = gold) = P(X = small, Y = gold) + P(X = large, Y = gold) = 0.5
#.   We already know the joint probability: P(X = small, Y = gold) = 0.2
#.   We can now use the given formula:
#.   P( X = small or Y = gold) = P(X = small) + P(Y = gold) - P(X = small, Y = gold)
#.   = 0.6 + 0.5 - 0.2
#.   = 0.9<jupyter_output><empty_output><jupyter_text>Think! 4: Covarying probability distributionsThe relationship between the marginal probabilities and the joint probabilities is determined by the correlation between the two random variables - a normalized measure of how much the variables covary. We can also think of this as gaining some information about one of the variables when we observe a measurement from the other. We will think about this more formally in Tutorial 2. Here, we want to think about how the correlation between size and color of these fish changes how much information we gain about one attribute based on the other. See Bonus Section 1 for the formula for correlation.Use the widget below and answer the following questions:1. When the correlation is zero, $\rho = 0$, what does the distribution of size tell you about color?2. Set $\rho$ to something small. As you change the probability of golden fish, what happens to the ratio of size probabilities? Set $\rho$ larger (can be negative). Can you explain the pattern of changes in the probabilities of size as you change the probability of golden fish?3. Set the probability of golden fish and of large fish to around 65%. As the correlation goes towards 1, how often will you see silver large fish?4. What is increasing the (absolute) correlation telling you about how likely you are to see one of the properties if you see a fish with the other?<jupyter_code># @markdown Execute this cell to enable the widget
style = {'description_width': 'initial'}
gs = GridspecLayout(2,2)

cor_widget = widgets.FloatSlider(0.0, description='ρ', min=-1, max=1, step=0.01)
px_widget = widgets.FloatSlider(0.5, description='p(color=golden)', min=0.01, max=0.99, step=0.01, style=style)
py_widget = widgets.FloatSlider(0.5, description='p(size=large)', min=0.01, max=0.99, step=0.01, style=style)
gs[0,0] = cor_widget
gs[0,1] = px_widget
gs[1,0] = py_widget


@widgets.interact(
    px=px_widget,
    py=py_widget,
    cor=cor_widget,
)
def make_corr_plot(px, py, cor):
    Cmin, Cmax = compute_cor_range(px, py) #allow correlation values
    cor_widget.min, cor_widget.max = Cmin+0.01, Cmax-0.01
    if cor_widget.value > Cmax:
        cor_widget.value = Cmax
    if cor_widget.value < Cmin:
        cor_widget.value = Cmin
    cor = cor_widget.value
    P = compute_marginal(px,py,cor)
    # print(P)
    fig = plot_joint_probs(P)
    plt.show(fig)
    plt.close(fig)
    return None

# gs[1,1] = make_corr_plot()
# to_remove explanation

#' 1. When the correlation is zero, the two properties are completely independent.
#' This means you don't gain any information about one variable from observing the other.
#' Importantly, the marginal distribution of one variable is therefore independent of the other.

#' 2. The correlation controls the distribution of probability across the joint probabilty table.
#' The higher the correlation, the more the probabilities are restricted by the fact that both rows
#' and columns need to sum to one! While the marginal probabilities show the relative weighting, the
#' absolute probabilities for one quality will become more dependent on the other as the correlation
#' goes to 1 or -1.

#' 3. The correlation will control how much probabilty mass is located on the diagonals. As the
#' correlation goes to 1 (or -1), the probability of seeing the one of the two pairings has to goes
#' towards zero!

#' 4. If we think about what information we gain by observing one quality, the intution from (3.) tells
#' us that we know more (have more information) about the other quality as a function of the correlation.<jupyter_output><empty_output><jupyter_text>We have just seen how two random variables can be more or less independent. The more correlated, the less independent, and the more shared information. We also learned that we can marginalize to determine the marginal likelihood of a hidden state or to find the marginal probability distribution of two random variables. We are going to now complete our journey towards being fully Bayesian! --- Section 5: Bayes' Rule and the Posterior Marginalization is going to be used to combine our prior knowlege, which we call the **prior**, and our new information from a measurement, the **likelihood**. Only in this case, the information we gain about the hidden state we are interested in, where the fish are, is based on the relationship between the probabilities of the measurement and our prior. We can now calculate the full posterior distribution for the hidden state ($s$) using Bayes' Rule. As we've seen, the posterior is proportional the the prior times the likelihood. This means that the posterior probability of the hidden state ($s$) given a measurement ($m$) is proportional to the likelihood of the measurement given the state times the prior probability of that state (the marginal likelihood):$$ P(s | m) \propto P(m | s) P(s)  $$We say proportional to instead of equal because we need to normalize to produce a full probability distribution:$$ P(s | m) = \frac{P(m | s) P(s)}{P(m)}  $$Normalizing by this $P(m)$ means that our posterior is a complete probability distribution that sums or integrates to 1 appropriately. We now can use this new, complete probability distribution for any future inference or decisions we like! In fact, as we will see tomorrow, we can use it as a new prior! Finally, we often call this probability distribution our beliefs over the hidden states, to emphasize that it is our subjective knowlege about the hidden state.For many complicated cases, like those we might be using to model behavioral or brain inferences, the normalization term can be intractable or extremely complex to calculate. We can be careful to choose probability distributions were we can analytically calculate the posterior probability or numerical approximation is reliable. Better yet, we sometimes don't need to bother with this normalization! The normalization term, $P(m)$, is the probability of the measurement. This does not depend on state so is essentially a constant we can often ignore. We can compare the unnormalized posterior distribution values for different states because how they relate to each other is unchanged when divided by the same constant. We will see how to do this to compare evidence for different hypotheses tomorrow. (It's also used to compare the likelihood of models fit using maximum likelihood estimation, as you did in W1D5.)In this relatively simple example, we can compute the marginal probability $P(m)$ easily by using:$$P(m) = \sum_s P(m | s) P(s)$$We can then normalize so that we deal with the full posterior distribution.  Math Exercise 5: Calculating a posterior probabilityOur prior is $p(s = left) = 0.3$ and $p(s = right) = 0.7$. In the video, we learned that the chance of catching a fish given they fish on the same side as the school was 50%. Otherwise, it was 10%. We observe a person fishing on the left side. Our likelihood is: | Likelihood: p(m \| s) | m = catch fish   | m = no fish  || ----------------- |----------|----------|| s = left          | 0.5          | 0.5         || s = right         | 0.1        |  0.9       |Calculate the posterior probability (on paper) that:1. The school is on the left if the fisherperson catches a fish: $p(s = left | m = catch fish)$ (hint: normalize by compute $p(m = catch fish)$)2. The school is on the right if the fisherperson does not catch a fish: $p(s = right | m = no fish)$<jupyter_code># to_remove explanation

# 1. Using Bayes rule, we know that P(s = left | m = catch fish) = P(m = catch fish | s = left)P(s = left) / P(m = catch fish)
#.   Let's first compute P(m = catch fish):
#.   P(m = catch fish) =  P(m = catch fish | s = left)P(s = left) +  P(m = catch fish | s = right)P(s = right)
#                      = 0.5 * 0.3 + .1*.7
#                      = 0.22
#.   Now we can plug in all parts of Bayes rule:
#    P(s = left | m = catch fish) = P(m = catch fish | s = left)P(s = left) / P(m = catch fish)
#                                 = 0.5*0.3/0.22
#                                 = 0.68

# 2. Using Bayes rule, we know that P(s = right | m = no fish) = P(m = no fish | s = right)P(s = right) / P(m = no fish)
#.   Let's first compute P(m = no fish):
#.   P(m = no fish) =  P(m = no fish | s = left)P(s = left) +  P(m = no fish | s = right)P(s = right)
#                      = 0.5 * 0.3 + .9*.7
#                      = 0.78
#.   Now we can plug in all parts of Bayes rule:
#    P(s = right | m = no fish) = P(m = no fish | s = right)P(s = right) / P(m = no fish)
#                                 = 0.9*0.7/0.78
#                                 = 0.81<jupyter_output><empty_output><jupyter_text>Coding Exercise 5: Computing PosteriorsLet's implement our above math to be able to compute posteriors for different priors and likelihood.sAs before, our prior is $p(s = left) = 0.3$ and $p(s = right) = 0.7$. In the video, we learned that the chance of catching a fish given they fish on the same side as the school was 50%. Otherwise, it was 10%. We observe a person fishing on the left side. Our likelihood is: | Likelihood: p(m \| s) | m = catch fish   | m = no fish  || ----------------- |----------|----------|| s = left          | 0.5          | 0.5         || s = right         | 0.1        |  0.9       |We want our full posterior to take the same 2 by 2 form. Make sure the outputs match your math answers!<jupyter_code>def compute_posterior(likelihood, prior):
  """ Use Bayes' Rule to compute posterior from likelihood and prior

  Args:
    likelihood (ndarray): i x j array with likelihood probabilities where i is
                    number of state options, j is number of measurement options
    prior (ndarray): i x 1 array with prior probability of each state

  Returns:
    ndarray: i x j array with posterior probabilities where i is
            number of state options, j is number of measurement options

  """

  #################################################
  ## TODO for students ##
  # Fill out function and remove
  raise NotImplementedError("Student exercise: implement compute_posterior")
  #################################################

  # Compute unnormalized posterior (likelihood times prior)
  posterior = ... # first row is s = left, second row is s = right

  # Compute p(m)
  p_m = np.sum(posterior, axis = 0)

  # Normalize posterior (divide elements by p_m)
  posterior /= ...

  return posterior

# Make prior
prior = np.array([0.3, 0.7]).reshape((2, 1)) # first row is s = left, second row is s = right

# Make likelihood
likelihood = np.array([[0.5, 0.5], [0.1, 0.9]]) # first row is s = left, second row is s = right

# Compute posterior
posterior = compute_posterior(likelihood, prior)

# Visualize
with plt.xkcd():
  plot_prior_likelihood_posterior(prior, likelihood, posterior)
# to_remove solution

def compute_posterior(likelihood, prior):
  """ Use Bayes' Rule to compute posterior from likelihood and prior

  Args:
    likelihood (ndarray): i x j array with likelihood probabilities where i is
                    number of state options, j is number of measurement options
    prior (ndarray): i x 1 array with prior probability of each state

  Returns:
    ndarray: i x j array with posterior probabilities where i is
            number of state options, j is number of measurement options

  """

  # Compute unnormalized posterior (likelihood times prior)
  posterior = likelihood * prior # first row is s = left, second row is s = right

  # Compute p(m)
  p_m = np.sum(posterior, axis = 0)

  # Normalize posterior (divide elements by p_m)
  posterior /= p_m

  return posterior

# Make prior
prior = np.array([0.3, 0.7]).reshape((2, 1)) # first row is s = left, second row is s = right

# Make likelihood
likelihood = np.array([[0.5, 0.5], [0.1, 0.9]]) # first row is s = left, second row is s = right

# Compute posterior
posterior = compute_posterior(likelihood, prior)

# Visualize
with plt.xkcd():
  plot_prior_likelihood_posterior(prior, likelihood, posterior)<jupyter_output><empty_output><jupyter_text>Interactive Demo 5: What affects the posterior?Now that we can understand the implementation of *Bayes rule*, let's vary the parameters of the prior and likelihood to see how changing the prior and likelihood affect the posterior. In the demo below, you can change the prior by playing with the slider for $p( s = left)$. You can also change the likelihood by changing the probability of catching a fish given that the school is on the left and the probability of catching a fish given that the school is on the right. The fisherperson you are observing is fishing on the left. 1.   Keeping the likelihood constant, when does the prior have the strongest influence over the posterior? Meaning, when does the posterior look most like the prior no matter whether a fish was caught or not?2.   Keeping the likelihood constant, when does the prior exert the weakest influence?  Meaning, when does the posterior look least like the prior and depend most on whether a fish was caught or not?3.  Set the prior probability of the state = left to 0.6 and play with the likelihood. When does the likelihood exert the most influence over the posterior?<jupyter_code># @markdown Execute this cell to enable the widget
style = {'description_width': 'initial'}
ps_widget = widgets.FloatSlider(0.3, description='p(s = left)',
                                min=0.01, max=0.99, step=0.01)
p_a_s1_widget = widgets.FloatSlider(0.5, description='p(fish | s = left)',
                                    min=0.01, max=0.99, step=0.01, style=style)
p_a_s0_widget = widgets.FloatSlider(0.1, description='p(fish | s = right)',
                                    min=0.01, max=0.99, step=0.01, style=style)
observed_widget = widgets.Checkbox(value=False, description='Observed fish (m)',
                                 disabled=False, indent=False, layout={'width': 'max-content'})

@widgets.interact(
    ps=ps_widget,
    p_a_s1=p_a_s1_widget,
    p_a_s0=p_a_s0_widget,
    m_right=observed_widget
)
def make_prior_likelihood_plot(ps,p_a_s1,p_a_s0,m_right):
    fig = plot_prior_likelihood(ps,p_a_s1,p_a_s0,m_right)
    plt.show(fig)
    plt.close(fig)
    return None
# to_remove explanation

# 1).  The prior exerts a strong influence over the posterior when it is very informative: when
#.   the probability of the school being on one side or the other. If the prior that the fish are
#.   on the left side is very high (like 0.9), the posterior probability of the state being left is
#.   high regardless of the measurement.

# 2).  The prior does not exert a strong influence when it is not informative: when the probabilities
#.     of the school being on the left vs right are similar (both are 0.5 for example). In this case,
#.     the posterior is more driven  by the collected data (the measurement) and more closely resembles
#.     the likelihood.


#.  3) Similarly to the prior, the likelihood exerts the most influence when it is informative: when catching
#.    a fish tells you a lot of information about which state is likely. For example, if the probability of the
#.    fisherperson catching a fish if he is fishing on the right side and the school is on the left is 0
#.    (p fish | s = left) = 0 and the probability of catching a fish if the school is on the right is 1, the
#.    prior does not affect the posterior at all. The measurement tells you the hidden state completely.<jupyter_output><empty_output><jupyter_text>Section 6: Making Bayesian fishing decisionsWe will explore how to consider the expected utility of an action based on our belief (the posterior distribution) about where we think the fish are. Now we have all the components of a Bayesian decision: our prior information, the likelihood given a measurement, the posterior distribution (belief) and our utility (the gains and losses). This allows us to consider the relationship between the true value of the hidden state, $s$, and what we *expect* to get if we take action, $a$, based on our belief!Let's use the following widget to think about the relationship between these probability distributions and utility function.  Think! 6: What is more important, the probabilities or the utilities?We are now going to put everything we've learned together to gain some intuitions for how each of the elements that goes into a Bayesian decision comes together. Remember, the common assumption in neuroscience, psychology, economics, ecology, etc. is that we (humans and animals) are tying to maximize our expected utility.1. Can you find a situation where the expected utility is the same for both actions?2. What is more important for determining the expected utility: the prior or a new measurement (the likelihood)?3. Why is this a normative model?4. Can you think of ways in which this model would need to be extended to describe human or animal behavior?<jupyter_code># @markdown Execute this cell to enable the widget
style = {'description_width': 'initial'}
ps_widget = widgets.FloatSlider(0.3, description='p(s)',
                                min=0.01, max=0.99, step=0.01)
p_a_s1_widget = widgets.FloatSlider(0.5, description='p(fish | s = left)',
                                    min=0.01, max=0.99, step=0.01, style=style)
p_a_s0_widget = widgets.FloatSlider(0.1, description='p(fish | s = right)',
                                    min=0.01, max=0.99, step=0.01, style=style)
observed_widget = widgets.Checkbox(value=False, description='Observed fish (m)',
                                 disabled=False, indent=False, layout={'width': 'max-content'})

@widgets.interact(
    ps=ps_widget,
    p_a_s1=p_a_s1_widget,
    p_a_s0=p_a_s0_widget,
    m_right=observed_widget
)
def make_prior_likelihood_utility_plot(ps, p_a_s1, p_a_s0,m_right):
    fig = plot_prior_likelihood_utility(ps, p_a_s1, p_a_s0,m_right)
    plt.show(fig)
    plt.close(fig)
    return None
# to_remove explanation

#' 1. There are actually many (infinite) combinations that can produce the same
#.    expected utility for both actions: but the posterior probabilities will always
#     have to balance out the differences in the utility function. So, what is
#     important is that for a given utility function, there will be some 'point
#     of indifference'

#' 2. What matters is the relative information: if the prior is close to 50/50,
#     then the likelihood has more infuence, if the likelihood is 50/50 given a
#     measurement (the measurement is uninformative), the prior is more important.
#     But the critical insite from Bayes Rule and the Bayesian approach is that what
#     matters is the relative information you gain from a measurement, and that
#     you can use all of this information for your decision.

#' 3. The model gives us a very precise way to think about how we *should* combine
#     information and how we *should* act, GIVEN some assumption about our goals.
#     In this case, if we assume we are trying to maximize expected utility--we can
#     state what an animal or subject should do.

#' 4. There are lots of possible extensions. Humans may not always try to maximize
#     utility; humans and animals might not be able to calculate or represent probabiltiy
#     distributions exactly; The utility function might be more complicated; etc.<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10762608777018808, 'fraction_numerical': 0.017909609806306726, 'mean_word_length': 3.416198347107438, 'pattern_counts': {'":': 0, '<': 68, '<?xml version=': 0, '>': 62, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 13, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '919', 'license': 'CC-BY-4.0', 'n_tokens_mistral': 16932, 'n_tokens_neox': 16069, 'n_words': 6473, 'path': 'tutorials/W3D1_BayesianDecisions/W3D1_Tutorial1.ipynb', 'repo_name': 'bgalbraith/course-content', 'size': '53445'}