<jupyter_start><jupyter_text>Building Bayesian Networksby <NAME>March 7th, 2022In this notebook, you can follow an example of creating a Bayesian Network, by using the pgmpy Python library. The example tackled by this notebook is the Bayesian Network for Computer Failure, which we covered in detail in our lectures. Bayesian Network is composed of two components:a directed acyclic graph DAG: 𝓖(𝑉,𝐸)  - nodes  𝑉, and edges 𝐸a set of random variables represented by nodesdirected edges – connect two random variables with causal probabilistic dependencycycles not allowed in the grapha set of conditional probability distributionsconditional probability distribution is defined for each node in the graphthe conditional probability distribution of a node (random variable) is defined for every possible outcome of the preceding causal node(s)  Step 1: Create the DAG of the Bayesian Network  Step 1.1. Describe the problema computer does not start, which is considered as a computer failure (observation/evidence)possible independent causes of failure:electricity failure computer malfunctionelectricity failure and computer malfunction are ancestors/parents of computer failureinference objectives:    calculate the posterior conditional probability distribution of each of the possible unobserved causes given the observed evidence of a computer failurecalculate the prior conditional probability distribution of having a computer failure without any evidencecalculate the prior conditional probability distribution of having a computer failure considering a recent electricity failure as evidence  Step 1.2. Draw and show the DAGLet's show the GAD model from the lectures.<jupyter_code>from IPython.display import Image

Image("images/computer_failure.png")<jupyter_output><empty_output><jupyter_text>Step 1.3. Create the DAG of the Bayesian Network by using the pgmpy libraryLet's create the Computer Failure Bayesian Network following the DAG shown above. We use the pgmpy library for this task.Here, we consider three nodes, each representing a binary random variable:electricity_failure: domain = {yes, no} computer_malfunction: domain = {yes, no}computer_failure: domain = {yes, no}Additional constraints: electricity_failure ⊥ computer_malfunction (independent) To build the Bayesian Network DAG we use the BayesianNetwork class provided by the pgmpy library.<jupyter_code>from pgmpy.models import BayesianNetwork
from pgmpy.factors.discrete import TabularCPD
import networkx as nx
import pylab as plt
 
#**** define the Bayesian DAG structure
#
# we define the Bayesian structure by connecting nodes (random variables)
# the direction is determined from the order of the nodes
#
# here, electricity_failure, computer_malfunction, and computer_failure are random variables

model = BayesianNetwork([('electricity_failure', 'computer_failure'), ('computer_malfunction', 'computer_failure')])<jupyter_output><empty_output><jupyter_text>Step 1.4. Show the DAG of the Bayesian NetworkWe draw the model created in the previous step.<jupyter_code>nx.draw(model, with_labels=True)
plt.show()
plt.close()<jupyter_output><empty_output><jupyter_text>Step 2: Create the Conditional Probability Distributions of the Bayesian NetworkIn this step, we build the distribution models per a random variable. We use our believes and logic to come up with these models. Let's say this is our priori knowledge. Step 2.1: Build the probability distribution tables with TabularCPDIn this exercise we are going to use the models provided in the lectures:P (electricity_failure=yes) = 0.1; P (electricity_failure=no) = 0.9P (computer_malfunction=yes) = 0.2; P (computer_malfunction=no) = 0.8P (computer_failure=yes | electricity_failure=no ∩ computer_malfunction=no) = 0P (computer_failure=yes | electricity_failure=no ∩ computer_malfunction=yes) = 0.5 P (computer_failure=yes | electricity_failure=yes ∩ computer_malfunction=no) = 1P (computer_failure=yes | electricity_failure=yes ∩ computer_malfunction=yes) = 1We use the TabularCPD class provided by the pgmpy library to build the distribution model of each random variable.<jupyter_code>#**** define the Conditional Probability Distributions (CPDs)

# define CPD for electricity_failure
#
# the TabularCPD class associates the ditribution model to be constructed with a random variable, 
# e.g., electricity_failure  
cpd_electricity_failure = TabularCPD (
    # associate the distribution table with a random variable - must be included in the Bayesian model
    variable="electricity_failure",

    # define the cardinality of the variable domain, i.e., binary = 2 -> domain = {yes, no} 
    variable_card=2,
    
    # define the distribution table 
    # values are ordered following the domain {yes, no}
    # e.g., P (electricity_failure=yes) = 0.1; P (electricity_failure=no) = 0.9 
    values=[[0.1], [0.9]] 
)    

# define CPD for computer_malfunction
cpd_computer_malfunction = TabularCPD (
    variable="computer_malfunction",
    variable_card=2,
    values=[[0.2], [0.8]]
)    

# define CPD for computer_failure
cpd_computer_failure = TabularCPD (
    variable="computer_failure",
    variable_card=2,

    # define the distribution table 
    # 
    # this distrbution table includes values for all the combinations of the values 
    # held by the variables that condition this variable
    #
    # electricity_failure = A, computer_malfunction = B, computer_failure = C
    # C=yes = {[A=yes,B=yes],[A=yes,B=no],[A=no,B=yes],[A=no,B=no]} = {1, 1, 0.5, 0} 
    # C=no = {[A=yes,B=yes],[A=yes,B=no],[A=no,B=yes],[A=no,B=no]} = {0, 0, 0.5, 1]}
    # C=yes + C=no = 1 
    #    
    values=[[1, 1, 0.5, 0],[0, 0, 0.5, 1]],

    # define the evidences - one of the causes must be true 
    evidence=["electricity_failure", "computer_malfunction"],
    evidence_card=[2,2]
)<jupyter_output><empty_output><jupyter_text>Step 2.2: Associate the distribution models with the network structure<jupyter_code># we associate the distribution models with the network structure, i.e., with the random variables
model.add_cpds(cpd_electricity_failure, cpd_computer_malfunction, cpd_computer_failure)<jupyter_output><empty_output><jupyter_text>Step 2.3: Check the modelWe use the built-in mechanism (the check_model() method) of the BayesianNetwork class to check the distribution models. The following checks are performed:consistency in terms of domain cardinality vs. number of valuesGaussian distribution for values of the same series, e.g., P(no) + P(yes) = 1<jupyter_code>model.check_model()<jupyter_output><empty_output><jupyter_text>Step 3: Inference with the Bayesian NetworkIn this step, we use the inference mechanisms of the Bayesian Network to:     calculate the prior conditional probability distribution of having a computer failure without any evidencecalculate the prior conditional probability distribution of having a computer failure considering a recent electricity failure as an evidencecalculate the posterior conditional probability distribution of each of the possible unobserved causes given the observed evidence of a computer failureTo run inference tasks on this network, we use the VariableElimination class provided by the pgmpy.inference iibrary.<jupyter_code>from pgmpy.inference import VariableElimination
 
infer = VariableElimination(model)<jupyter_output><empty_output><jupyter_text>Step 3.1: Infer Posterior Probability of having electricity failureRecall that we use the Bayes' Theorem to calculate posterior probability.                             P(Evidence|Cause) * P(Cause)   P(Cause|Evidence) = ---------------------------------------		                         P(Evidence)                                  Please, refer to lectures on how we infer posterior probability.<jupyter_code>#**** infer posterior probability
# evidence={'computer_failure': 0} means computer_failure = yes
#
posterior_p = infer.query(['electricity_failure'], evidence={'computer_failure': 0})<jupyter_output><empty_output><jupyter_text>Show the posterior probability results.<jupyter_code>print(posterior_p)<jupyter_output>+------------------------+----------------------------+
| electricity_failure    |   phi(electricity_failure) |
+========================+============================+
| electricity_failure(0) |                     0.5263 |
+------------------------+----------------------------+
| electricity_failure(1) |                     0.4737 |
+------------------------+----------------------------+<jupyter_text>Step 3.2: Infer Prior Probability of having computer failure with no evidenceRecall that we use the following formula to infer prior probability.𝑷(𝒄𝟏)=∑(𝒂∈𝑨)∑(𝒃∈𝑩) 〖𝑷(𝑨=𝒂∩𝑩=𝒃∩𝑪=𝒄𝟏)〗Please, refer to lectures on how we infer posterior probability.<jupyter_code>prior_p = infer.query(['computer_failure'], evidence={})<jupyter_output><empty_output><jupyter_text>Show the prior probability results.<jupyter_code>print(prior_p)<jupyter_output>+---------------------+-------------------------+
| computer_failure    |   phi(computer_failure) |
+=====================+=========================+
| computer_failure(0) |                  0.1900 |
+---------------------+-------------------------+
| computer_failure(1) |                  0.8100 |
+---------------------+-------------------------+<jupyter_text>Step 3.3: Infer Prior Probability of having computer failure with electricity failure as a prior evidence<jupyter_code>prior_p = infer.query(['computer_failure'], evidence={'electricity_failure': 0, 'computer_malfunction': 1})<jupyter_output><empty_output><jupyter_text>Show the prior probability results.<jupyter_code>print(prior_p)<jupyter_output>+---------------------+-------------------------+
| computer_failure    |   phi(computer_failure) |
+=====================+=========================+
| computer_failure(0) |                  1.0000 |
+---------------------+-------------------------+
| computer_failure(1) |                  0.0000 |
+---------------------+-------------------------+
{'fraction_non_alphanumeric': 0.13667768098528085, 'fraction_numerical': 0.012316010814058276, 'mean_word_length': 4.021618903971845, 'pattern_counts': {'":': 0, '<': 51, '<?xml version=': 0, '>': 52, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '42371', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 2860, 'n_tokens_neox': 2625, 'n_words': 940, 'path': 'Weeks_7-9/Lessons/bn_comp_failure.ipynb', 'repo_name': 'evassev/CS6462_SEM2_2021-2', 'size': '9992'}