<jupyter_start><jupyter_text>Figure 3.1:    Several sets of $(x, y)$ points, with the correlation coefficient of $x$ and $y$ for each set. Note that the correlation reflects the noisiness and direction of a linear relationship (top row), but not the slope of that relationship (middle), nor many aspects of nonlinear relationships (bottom). (Note: the figure in the center has a slope of 0 but in that case the correlation coefficient is undefined because the variance of $Y$ is zero.) From  https://en.wikipedia.org/wiki/Pearson_correlation_coefficient . Used with kind permission of Wikipedia author Imagecreator<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')<jupyter_output><empty_output><jupyter_text>Figure 3.2:    Examples of spurious correlation between causally unrelated time series. Consumption of ice cream (red) and violent crime rate (yellow). over time. From  http://icbseverywhere.com/blog/2014/10/the-logic-of-causal-conclusions/ . Used with kind permission of <NAME><jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')<jupyter_output><empty_output><jupyter_text>Figure 3.3:    Illustration of Simpson's paradox. (a) Overall, $y$ decreases $y$ with $x$. (b) Within each group, $y$ increases with $x$. From  https://en.wikipedia.org/wiki/Simpson's_paradox . Used with kind permission of Wikipedia author Pace\nobreakspace  svwiki<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')<jupyter_output><empty_output><jupyter_text>Figure 3.4:    Illustration of Simpson's paradox using COVID-19\xspace , (a) Case fatality rates (CFRs) in Italy and China by age group, and in aggregated form (``Total'', last pair of bars), up to the time of reporting (see legend). (b) Proportion of all confirmed cases included in (a) within each age group by country. From Figure 1 of [JLB20] . Used with kind permission of <NAME><jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')<jupyter_output><empty_output><jupyter_text>Figure 3.5:    Visualization of a 2d Gaussian density as a surface plot. (a) Distribution using a full covariance matrix can be oriented at any angle. (b) Distribution using a diagonal covariance matrix must be parallel to the axis. (c) Distribution using a spherical covariance matrix must have a symmetric shape.  Figure(s) generated by [gauss_plot_2d.py](https://github.com/probml/pyprobml/blob/master/scripts/gauss_plot_2d.py)<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')
deimport(superimport)
%run gauss_plot_2d.py<jupyter_output><empty_output><jupyter_text>Figure 3.6:    Visualization of a 2d Gaussian density in terms of level sets of constant probability density. (a) A full covariance matrix has elliptical contours. (b) A diagonal covariance matrix is an  \bf axis aligned  ellipse. (c) A spherical covariance matrix has a circular shape.  Figure(s) generated by [gauss_plot_2d.py](https://github.com/probml/pyprobml/blob/master/scripts/gauss_plot_2d.py)<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')
deimport(superimport)
%run gauss_plot_2d.py<jupyter_output><empty_output><jupyter_text>Figure 3.7:    Illustration of data imputation using an MVN. (a) Visualization of the data matrix. Blank entries are missing (not observed). Blue are positive, green are negative. Area of the square is proportional to the value. (This is known as a  \bf Hinton diagram , named after <NAME>, a famous ML researcher.) (b) True data matrix (hidden). (c) Mean of the posterior predictive distribution, based on partially observed data in that row, using the true model parameters.  Figure(s) generated by [gauss_imputation_known_params_demo.py](https://github.com/probml/pyprobml/blob/master/scripts/gauss_imputation_known_params_demo.py)<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')
deimport(superimport)
%run gauss_imputation_known_params_demo.py<jupyter_output><empty_output><jupyter_text>Figure 3.8:    Inference about $z$ given a noisy observation $y=3$. (a) Strong prior $\mathcal N (0,1)$. The posterior mean is ``shrunk'' towards the prior mean, which is 0. (b) Weak prior $\mathcal N (0,5)$. The posterior mean is similar to the MLE.  Figure(s) generated by [gauss_infer_1d.py](https://github.com/probml/pyprobml/blob/master/scripts/gauss_infer_1d.py)<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')
deimport(superimport)
%run gauss_infer_1d.py<jupyter_output><empty_output><jupyter_text>Figure 3.9:    Illustration of Bayesian inference for a 2d Gaussian random vector $ \bm z  $. (a) The data is generated from $ \bm y  _n \sim \mathcal N ( \bm z  ,\boldsymbol \Sigma  _y)$, where $ \bm z  =[0.5, 0.5]^  \mkern -1.5mu\mathsf T   $ and $\boldsymbol \Sigma  _y=0.1 [2, 1; 1, 1])$. We assume the sensor noise covariance $\boldsymbol \Sigma  _y$ is known but $ \bm z  $ is unknown. The black cross represents $ \bm z  $. (b) The prior is $p( \bm z  ) = \mathcal N ( \bm z  | \bm 0  ,0.1 \mathbf I _2)$. (c) We show the posterior after 10 data points have been observed.  Figure(s) generated by [gauss_infer_2d.py](https://github.com/probml/pyprobml/blob/master/scripts/gauss_infer_2d.py)<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')
deimport(superimport)
%run gauss_infer_2d.py<jupyter_output><empty_output><jupyter_text>Figure 3.10:    We observe $ \bm y  _1=(0,-1)$(red cross) and $ \bm y  _2=(1,0)$(green cross) and estimate $\mathbb E \left[  \bm z  | \bm y  _1, \bm y  _2 \right ]$(black cross). (a) Equally reliable sensors, so the posterior mean estimate is in between the two circles. (b) Sensor 2 is more reliable, so the estimate shifts more towards the green circle. (c) Sensor 1 is more reliable in the vertical direction, Sensor 2 is more reliable in the horizontal direction. The estimate is an appropriate combination of the two measurements.  Figure(s) generated by [sensor_fusion_2d.py](https://github.com/probml/pyprobml/blob/master/scripts/sensor_fusion_2d.py)<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')
deimport(superimport)
%run sensor_fusion_2d.py<jupyter_output><empty_output><jupyter_text>Figure 3.11:    A mixture of 3 Gaussians in 2d. (a) We show the contours of constant probability for each component in the mixture. (b) A surface plot of the overall density. Adapted from Figure 2.23 of [Bis06] .  Figure(s) generated by [gmm_plot_demo.py](https://github.com/probml/pyprobml/blob/master/scripts/gmm_plot_demo.py)<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')
deimport(superimport)
%run gmm_plot_demo.py<jupyter_output><empty_output><jupyter_text>Figure 3.12:   (a) Some data in 2d. (b) A possible clustering using $K=5$ clusters computed using a GMM.  Figure(s) generated by [gmm_2d.py](https://github.com/probml/pyprobml/blob/master/scripts/gmm_2d.py)<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')
deimport(superimport)
%run gmm_2d.py<jupyter_output><empty_output><jupyter_text>Figure 3.13:    We fit a mixture of 20 Bernoullis to the binarized MNIST digit data. We visualize the estimated cluster means $  \bm \mu    _k$. The numbers on top of each image represent the estimated mixing weights $ \pi  _k$. No labels were used when training the model.  Figure(s) generated by [mix_ber_em_mnist.py](https://github.com/probml/pyprobml/blob/master/scripts/mix_ber_em_mnist.py)<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')
deimport(superimport)
%run mix_ber_em_mnist.py<jupyter_output><empty_output><jupyter_text>Figure 3.14:    Water sprinkler PGM with corresponding binary CPTs. T and F stand for true and false<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')<jupyter_output><empty_output><jupyter_text>Figure 3.15:    Illustration of first and second order autoregressive (Markov) models<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')<jupyter_output><empty_output><jupyter_text>Figure 3.16:    Left: data points $ \bm y  _n$ are conditionally independent given $ \bm \theta   $. Right: Same model, using plate notation. This represents the same model as the one on the left, except the repeated $ \bm y  _n$ nodes are inside a box, known as a plate; the number in the lower right hand corner, $N$, specifies the number of repetitions of the $ \bm y  _n$ node<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')<jupyter_output><empty_output><jupyter_text>Figure 3.17:    A Gaussian mixture model represented as a graphical model<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10343284536214281, 'fraction_numerical': 0.0099085853097232, 'mean_word_length': 4.454672245467225, 'pattern_counts': {'":': 0, '<': 72, '<?xml version=': 0, '>': 89, 'https://': 28, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '17', 'dir': 'jupyter-structured-clean-dedup', 'id': '408768', 'license': 'MIT', 'n_tokens_mistral': 5538, 'n_tokens_neox': 5265, 'n_words': 1595, 'path': 'pml1/figure_notebooks/chapter3_probability_multivariate_models_figures.ipynb', 'repo_name': 'hanchenresearch/pml-book', 'size': '15676'}