<jupyter_start><jupyter_text>Bayes' Rule in Systems with Hidden State When a system has some form of memory, we refer to the current value of that memory as the *state* of the system. If that memory cannot be observed, then we call that *hidden state*. Systems with hidden state are different from systems with unknown inputs in that the hidden state can affect more than one output: depending on how the state evolves, it may affect one output, a few outputs, or all outputs.In fact, we have already seen a system with hidden state: the Magician's Coin problem. In this problem, the hidden state is which coin was chosen, and the chosen coin affects all future outputs of the system. Note that although the Magician's Coin does not have both state and input(s), many have both.In this section, I will revisit the Magician's Coin problem to better expose why the observations affect the probabilities of future events in the way they do. This gives us a chance to use Bayes' Rule to explore the probabilities of the hidden state, and it will also give us an opportunity to apply our knowledge about conditional independence. Recall the simple question we started with: **If the coin comes up heads on the first flip, what is the probability that it comes up heads on the second flip?**As before, let $H_i$ be the event that the chosen coin comes up heads on flip $i$. Also, we assume that the magician is equally likely to choose between a fair coin and a two-headed coin. Let $F$ be the event atht eh chosen coin is fair.We are asked to find $P(H_2 |H_1)$. In the previous section, we solved this using the definition of conditional probability and the Law of Total Probability as follows:$$ P(H_2|H_1) = \frac{P(H_1 \cap H_2)}{P(H_1)}$$where$$P(H_1) = P(H_1 |F) P(F) + P(H_1 |\overline{F}) P(\overline{F}) = 3/4$$and$$P(H_1 \cap H_2) = P(H_1 \cap H_2 |F) P(F) + P(H_1 \cap H_2 |\overline{F}) P(\overline{F}) = 5/8$$So, $P(H_2|H_1) = 5/6$. This is the easiest way to solve for $P(H_2|H_1)$. But it is non-intuitive because it tells us nothing about why knowing $H_1$ affects the probability of $H_2$. We previously argued that every time we see an outcome of heads, we should be more likely to believe that we have the two-headed coin (i.e., the probability that we have the two-headed coin should increase).  Only now do we have the tools to test this hypothesis: We are interested in $P(\overline{F}|H_1)$, the probability of having selected the two-headed coin after the first outcome is observed to be heads. We note that we do not have outcomes in this form, but we do have outcomes of the form $P(H_1|\overline{F})$ and $P(F)$. In fact, these probabilities are a likelihood and an *a priori* probability for this system. So, we can solve the for the desired probability using Bayes' Rule:$$P\left(\overline{F} \left \vert H_1 \right. \right) = \frac{P \left( H_1 \left \vert \overline{F} \right. \right) P(F)}{P(H_1)}.$$Note that I did not expand the denominator because we have already found $P(H_1)$ using the Law of Total Probability. We already know the probabilities in the numerator, so we can calculate$$P\left(\overline{F} \left \vert H_1 \right. \right) = \frac{(1)(1/2)}{P(3/4)} = \frac{2}{3}.$$Thus, after observing a single heads, the probability that the magician has selected the two-headed coin goes from 1/2 to 2/3. Now, I will show how can apply our knowledge of the updated state probabilities to find the probability of $H_2$. Because of the multiple conditioning that arises, this may be confusing at first. To help with this, I'm going to first hide the conditioning on $H_1$ by defining$$\tilde{P}(H_2) = P(H_2|H_1)$$Recalling that conditioning on $H_1$ creates a new (conditional) probability measure, $\tilde{P}$ is a different name for that probability measure.Now, we can apply the Law of Total Probability using $\{F, \overline{F} \}$ as a partition:$$\tilde{P}(H_2) = \tilde{P}(H_2|F) \tilde{P}(F) + \tilde{P}(H_2 | \overline{F} ) \tilde{P} ( \overline{F})$$Next, we substitute back in that $\tilde{P}(A) = P(A|H_1)$ for any event $A$. Recall that conditioning on two events results in a single conditioning on the intersection of the two events. Thus, the resulting expression is$$P(H_2|H_1) = P(H_2|H_1 \cap F) P(F|H_1) + P\left(H_2 \left \vert H_1 \cap \overline{F} \right. \right) P \left( \overline{F} \left \vert H_1 \right. \right)$$We know $P(\overline{F}|H_1)=2/3$, and $P(F|H_1) = 1 - P(\overline{F}|H_1) = 1/3$. But this is where things get tricky -- what are the other two probabilities? We have to apply conditional independence to find these. If we know the true state, $F$ or $\overline{F}$, then information about what happened on the first flip cannot affect the outcome of the second flip. In other words, $H_2$ is *conditionally independent* of $H_1$ given either $F$ or $\overline{F}$.  Thus,$$P(H_2|H_1) = P(H_2| F) P(F|H_1) + P\left(H_2 \left \vert  \overline{F} \right. \right) P \left( \overline{F} \left \vert H_1 \right. \right)$$Finally, we are ready to solve:$$P(H_2|H_1) = \left (\frac 1 2 \right) \left( \frac 1 3 \right) +\left( 1  \right) \left( \frac 2 3 \right) = \frac 5 6,$$which is the same answer we got above. However we have a new perspective now. Given $H_1$:* the magician will have the fair coin with probability 1/3, and another flip of that coin will yield heads with probability 1/2, and* the magician will have the two-headed coin with probability 2/3, and another flipt of that coin will yield heads with probability 1.<jupyter_code>from jupyterquiz import display_quiz
git_path="https://raw.githubusercontent.com/jmshea/Foundations-of-Data-Science-with-Python/main/"

#display_quiz("quiz/bayes-hidden-state.json")
display_quiz(git_path + "07-bayesian-methods/quiz/bayes-hidden-state.json")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10529036705152507, 'fraction_numerical': 0.017404790625538513, 'mean_word_length': 4.414179104477612, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '554916', 'license': 'FSFAP', 'n_tokens_mistral': 1811, 'n_tokens_neox': 1728, 'n_words': 862, 'path': '07-bayesian-methods/bayes-hidden-state.ipynb', 'repo_name': 'jmshea/intro-data-science-for-engineers', 'size': '5803'}