<jupyter_start><jupyter_text>Figure 4.1:     (a) Covariance matrix for the features in the iris dataset from \cref  sec:iris . (b) Correlation matrix. We only show the lower triangle, since the matrix is symmetric and has a unit diagonal. Compare this to \cref  fig:irisPairs .  Figure(s) generated by [iris_cov_mat.py](https://github.com/probml/pyprobml/blob/master/scripts/iris_cov_mat.py)<jupyter_code>#@title Setup { display-mode: "form" }
%%time
# If you run this for the first time it would take ~25/30 seconds
!git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null && git clone https://github.com/probml/colab_powertoys.git &> /dev/null 
!pip3 install nbimporter -qqq 
%cd -q /content/colab_powertoys 
from colab_powertoys.probml_toys import probml_toys as pmlt
%cd -q /pyprobml/scripts

pmlt.show_and_run("/pyprobml/scripts/iris_cov_mat.py")<jupyter_output><empty_output><jupyter_text>Figure 4.2:     Illustration of various loss functions for binary classification. The horizontal axis is the margin $z=\cc@accent  "707E  y \eta $, the vertical axis is the loss. The log loss uses log base 2.  Figure(s) generated by [hinge_loss_plot.py](https://github.com/probml/pyprobml/blob/master/scripts/hinge_loss_plot.py)<jupyter_code>#@title Setup { display-mode: "form" }
%%time
# If you run this for the first time it would take ~25/30 seconds
!git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null && git clone https://github.com/probml/colab_powertoys.git &> /dev/null 
!pip3 install nbimporter -qqq 
%cd -q /content/colab_powertoys 
from colab_powertoys.probml_toys import probml_toys as pmlt
%cd -q /pyprobml/scripts

pmlt.show_and_run("/pyprobml/scripts/hinge_loss_plot.py")<jupyter_output><empty_output><jupyter_text>Figure 4.3:     Illustration of exponentially-weighted moving average with and without bias correction. (a) Short memory: $\beta =0.9$. (a) Long memory: $\beta =0.99$.  Figure(s) generated by [ema_demo.py](https://github.com/probml/pyprobml/blob/master/scripts/ema_demo.py)<jupyter_code>#@title Setup { display-mode: "form" }
%%time
# If you run this for the first time it would take ~25/30 seconds
!git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null && git clone https://github.com/probml/colab_powertoys.git &> /dev/null 
!pip3 install nbimporter -qqq 
%cd -q /content/colab_powertoys 
from colab_powertoys.probml_toys import probml_toys as pmlt
%cd -q /pyprobml/scripts

pmlt.show_and_run("/pyprobml/scripts/ema_demo.py")<jupyter_output><empty_output><jupyter_text>Figure 4.4:     Estimating a covariance matrix in $D=50$ dimensions using $N \in \ 100, 50, 25\ $ samples. We plot the eigenvalues in descending order for the true covariance matrix (solid black), the MLE (dotted blue) and the MAP estimate (dashed red), using \cref  eqn:covShrinkLedoit  with $\lambda =0.9$. We also list the condition number of each matrix in the legend. We see that the MLE is often poorly conditioned, but the MAP estimate is numerically well behaved. Adapted from Figure 1 of [SS05] .  Figure(s) generated by [shrinkcov_plots.py](https://github.com/probml/pyprobml/blob/master/scripts/shrinkcov_plots.py)<jupyter_code>#@title Setup { display-mode: "form" }
%%time
# If you run this for the first time it would take ~25/30 seconds
!git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null && git clone https://github.com/probml/colab_powertoys.git &> /dev/null 
!pip3 install nbimporter -qqq 
%cd -q /content/colab_powertoys 
from colab_powertoys.probml_toys import probml_toys as pmlt
%cd -q /pyprobml/scripts

pmlt.show_and_run("/pyprobml/scripts/shrinkcov_plots.py")<jupyter_output><empty_output><jupyter_text>Figure 4.5:     (a-c) Ridge regression applied to a degree 14 polynomial fit to 21 datapoints. (d) MSE vs strength of regularizer. The degree of regularization increases from left to right, so model complexity decreases from left to right.  Figure(s) generated by [linreg_poly_ridge.py](https://github.com/probml/pyprobml/blob/master/scripts/linreg_poly_ridge.py)<jupyter_code>#@title Setup { display-mode: "form" }
%%time
# If you run this for the first time it would take ~25/30 seconds
!git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null && git clone https://github.com/probml/colab_powertoys.git &> /dev/null 
!pip3 install nbimporter -qqq 
%cd -q /content/colab_powertoys 
from colab_powertoys.probml_toys import probml_toys as pmlt
%cd -q /pyprobml/scripts

pmlt.show_and_run("/pyprobml/scripts/linreg_poly_ridge.py")<jupyter_output><empty_output><jupyter_text>Figure 4.6:     Schematic of 5-fold cross validation.<jupyter_code>#@title Setup { display-mode: "form" }
%%time
# If you run this for the first time it would take ~25/30 seconds
!git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null && git clone https://github.com/probml/colab_powertoys.git &> /dev/null 
!pip3 install nbimporter -qqq 
%cd -q /content/colab_powertoys 
from colab_powertoys.probml_toys import probml_toys as pmlt
%cd -q /pyprobml/scripts

pmlt.show_image("/pyprobml/book1/figures/images/Figure_4.6.png")<jupyter_output><empty_output><jupyter_text>Figure 4.7:     Ridge regression is applied to a degree 14 polynomial fit to 21 datapoints shown in \cref  fig:polyfitRidge  for different values of the regularizer $\lambda $. The degree of regularization increases from left to right, so model complexity decreases from left to right. (a) MSE on train (blue) and test (red) vs $\qopname o log (\lambda )$. (b) 5-fold cross-validation estimate of test MSE in red; error bars are standard error of the mean. In black we plot the negative log evidence $-\qopname o log p( \mathcal  D  |\lambda )$. Both curves are scaled to lie in $[0,1]$.  Figure(s) generated by [polyfitRidgeModelSel.m](https://github.com/probml/pmtk3/blob/master/demos/polyfitRidgeModelSel.m)<jupyter_code>#@title Setup { display-mode: "form" }
%%time
# If you run this for the first time it would take ~25/30 seconds
!git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null && git clone https://github.com/probml/colab_powertoys.git &> /dev/null 
!pip3 install nbimporter -qqq 
%cd -q /content/colab_powertoys 
from colab_powertoys.probml_toys import probml_toys as pmlt
%cd -q /pyprobml/scripts

pmlt.show_image("/pyprobml/book1/figures/images/Figure_4.7.png")
pmlt.show_image("/pyprobml/book1/figures/images/Figure_4.7.png")<jupyter_output><empty_output><jupyter_text>Figure 4.8:     Performance of a text classifier (a neural network applied to a bag of word embeddings using average pooling) vs number of training epochs on the IMDB movie sentiment dataset. Blue = train, red = validation. (a) Cross entropy loss. Early stopping is triggered at about epoch 25. (b) Classification accuracy.  Figure(s) generated by [imdb_mlp_bow_tf.py](https://github.com/probml/pyprobml/blob/master/scripts/imdb_mlp_bow_tf.py)<jupyter_code>#@title Setup { display-mode: "form" }
%%time
# If you run this for the first time it would take ~25/30 seconds
!git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null && git clone https://github.com/probml/colab_powertoys.git &> /dev/null 
!pip3 install nbimporter -qqq 
%cd -q /content/colab_powertoys 
from colab_powertoys.probml_toys import probml_toys as pmlt
%cd -q /pyprobml/scripts

pmlt.show_and_run("/pyprobml/scripts/imdb_mlp_bow_tf.py")<jupyter_output><empty_output><jupyter_text>Figure 4.9:     MSE on training and test sets vs size of training set, for data generated from a degree 2 polynomial with Gaussian noise of variance $\sigma ^2=4$. We fit polynomial models of varying degree to this data.  Figure(s) generated by [linreg_poly_vs_n.py](https://github.com/probml/pyprobml/blob/master/scripts/linreg_poly_vs_n.py)<jupyter_code>#@title Setup { display-mode: "form" }
%%time
# If you run this for the first time it would take ~25/30 seconds
!git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null && git clone https://github.com/probml/colab_powertoys.git &> /dev/null 
!pip3 install nbimporter -qqq 
%cd -q /content/colab_powertoys 
from colab_powertoys.probml_toys import probml_toys as pmlt
%cd -q /pyprobml/scripts

pmlt.show_and_run("/pyprobml/scripts/linreg_poly_vs_n.py")<jupyter_output><empty_output><jupyter_text>Figure 4.10:     Updating a Beta prior with a Bernoulli likelihood with sufficient statistics $N_1=4,N_0=1$. (a) Beta(2,2) prior. (b) Uniform Beta(1,1) prior.  Figure(s) generated by [beta_binom_post_plot.py](https://github.com/probml/pyprobml/blob/master/scripts/beta_binom_post_plot.py)<jupyter_code>#@title Setup { display-mode: "form" }
%%time
# If you run this for the first time it would take ~25/30 seconds
!git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null && git clone https://github.com/probml/colab_powertoys.git &> /dev/null 
!pip3 install nbimporter -qqq 
%cd -q /content/colab_powertoys 
from colab_powertoys.probml_toys import probml_toys as pmlt
%cd -q /pyprobml/scripts

pmlt.show_and_run("/pyprobml/scripts/beta_binom_post_plot.py")<jupyter_output><empty_output><jupyter_text>Figure 4.11:     Illustration of sequential Bayesian updating for the beta-Bernoulli model. Each colored box represents the predicted distribution $p(x_t|\mathbf  h _t)$, where $\mathbf  h _t=(N_ 1,t ,N_ 0,t )$ is the sufficient statistic derived from history of observations up until time $t$, namely the total number of heads and tails. The probability of heads (blue bar) is given by $p(x_t=1|\mathbf  h _t) = (N_ t,1  + 1)/(t+2)$, assuming we start with a uniform $\mathrm  Beta (\theta |1,1)$ prior. From Figure 3 of [Ped+19] . Used with kind permission of <NAME>.<jupyter_code>#@title Setup { display-mode: "form" }
%%time
# If you run this for the first time it would take ~25/30 seconds
!git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null && git clone https://github.com/probml/colab_powertoys.git &> /dev/null 
!pip3 install nbimporter -qqq 
%cd -q /content/colab_powertoys 
from colab_powertoys.probml_toys import probml_toys as pmlt
%cd -q /pyprobml/scripts

pmlt.show_image("/pyprobml/book1/figures/images/Figure_4.11.png")<jupyter_output><empty_output><jupyter_text>Figure 4.12:     (a) Posterior predictive distributions for 10 future trials after seeing $N_1=4$ heads and $N_0=1$ tails. (b) Plug-in approximation based on the same data. In both cases, we use a uniform prior.  Figure(s) generated by [beta_binom_post_pred_plot.py](https://github.com/probml/pyprobml/blob/master/scripts/beta_binom_post_pred_plot.py)<jupyter_code>#@title Setup { display-mode: "form" }
%%time
# If you run this for the first time it would take ~25/30 seconds
!git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null && git clone https://github.com/probml/colab_powertoys.git &> /dev/null 
!pip3 install nbimporter -qqq 
%cd -q /content/colab_powertoys 
from colab_powertoys.probml_toys import probml_toys as pmlt
%cd -q /pyprobml/scripts

pmlt.show_and_run("/pyprobml/scripts/beta_binom_post_pred_plot.py")<jupyter_output><empty_output><jupyter_text>Figure 4.13:     A mixture of two Beta distributions.  Figure(s) generated by [mixbetademo.py](https://github.com/probml/pyprobml/blob/master/scripts/mixbetademo.py)<jupyter_code>#@title Setup { display-mode: "form" }
%%time
# If you run this for the first time it would take ~25/30 seconds
!git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null && git clone https://github.com/probml/colab_powertoys.git &> /dev/null 
!pip3 install nbimporter -qqq 
%cd -q /content/colab_powertoys 
from colab_powertoys.probml_toys import probml_toys as pmlt
%cd -q /pyprobml/scripts

pmlt.show_and_run("/pyprobml/scripts/mixbetademo.py")<jupyter_output><empty_output><jupyter_text>Figure 4.14:     (a) The Dirichlet distribution when $K=3$ defines a distribution over the simplex, which can be represented by the triangular surface. Points on this surface satisfy $0 \leq \theta _k \leq 1$ and $\DOTSB \sum@ \slimits@ _ k=1 ^3 \theta _k = 1$.  Figure(s) generated by [dirichlet_3d_triangle_plot.py](https://github.com/probml/pyprobml/blob/master/scripts/dirichlet_3d_triangle_plot.py) [dirichlet_3d_spiky_plot.py](https://github.com/probml/pyprobml/blob/master/scripts/dirichlet_3d_spiky_plot.py)<jupyter_code>#@title Setup { display-mode: "form" }
%%time
# If you run this for the first time it would take ~25/30 seconds
!git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null && git clone https://github.com/probml/colab_powertoys.git &> /dev/null 
!pip3 install nbimporter -qqq 
%cd -q /content/colab_powertoys 
from colab_powertoys.probml_toys import probml_toys as pmlt
%cd -q /pyprobml/scripts

pmlt.show_and_run("/pyprobml/scripts/dirichlet_3d_triangle_plot.py")
pmlt.show_and_run("/pyprobml/scripts/dirichlet_3d_spiky_plot.py")<jupyter_output><empty_output><jupyter_text>Figure 4.15:     Samples from a 5-dimensional symmetric Dirichlet distribution for different parameter values. (a) $\oset  \smallsmile   \boldsymbol  \alpha    = (0.1,\ldots ,0.1)$. This results in very sparse distributions, with many 0s. (b) $\oset  \smallsmile   \boldsymbol  \alpha    = (1,\ldots ,1)$. This results in more uniform (and dense) distributions.  Figure(s) generated by [dirichlet_samples_plot.py](https://github.com/probml/pyprobml/blob/master/scripts/dirichlet_samples_plot.py)<jupyter_code>#@title Setup { display-mode: "form" }
%%time
# If you run this for the first time it would take ~25/30 seconds
!git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null && git clone https://github.com/probml/colab_powertoys.git &> /dev/null 
!pip3 install nbimporter -qqq 
%cd -q /content/colab_powertoys 
from colab_powertoys.probml_toys import probml_toys as pmlt
%cd -q /pyprobml/scripts

pmlt.show_and_run("/pyprobml/scripts/dirichlet_samples_plot.py")<jupyter_output><empty_output><jupyter_text>Figure 4.16:     Inferring the mean of a univariate Gaussian with known $\sigma ^2$. (a) Using strong prior, $p(\mu ) = \mathcal  N (\mu |0,1)$. (b) Using weak prior, $p(\mu ) = \mathcal  N (\mu |0,5)$.  Figure(s) generated by [gaussInferParamsMean1d.py](https://github.com/probml/pyprobml/blob/master/scripts/gaussInferParamsMean1d.py)<jupyter_code>#@title Setup { display-mode: "form" }
%%time
# If you run this for the first time it would take ~25/30 seconds
!git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null && git clone https://github.com/probml/colab_powertoys.git &> /dev/null 
!pip3 install nbimporter -qqq 
%cd -q /content/colab_powertoys 
from colab_powertoys.probml_toys import probml_toys as pmlt
%cd -q /pyprobml/scripts

pmlt.show_and_run("/pyprobml/scripts/gaussInferParamsMean1d.py")<jupyter_output><empty_output><jupyter_text>Figure 4.17:     Illustration of Bayesian inference for the mean of a 2d Gaussian. (a) The data is generated from $\mathbf  y _n \sim \mathcal  N (\boldsymbol  \mu  ,\boldsymbol  \Sigma  )$, where $\boldsymbol  \mu  =[0.5, 0.5]^ \top  $ and $\boldsymbol  \Sigma  =0.1 [2, 1; 1, 1])$. (b) The prior is $p(\boldsymbol  \mu  ) = \mathcal  N (\boldsymbol  \mu  |\boldsymbol  0 ,0.1 \mathbf  I _2)$. (c) We show the posterior after 10 data points have been observed.  Figure(s) generated by [gaussInferParamsMean2d.py](https://github.com/probml/pyprobml/blob/master/scripts/gaussInferParamsMean2d.py)<jupyter_code>#@title Setup { display-mode: "form" }
%%time
# If you run this for the first time it would take ~25/30 seconds
!git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null && git clone https://github.com/probml/colab_powertoys.git &> /dev/null 
!pip3 install nbimporter -qqq 
%cd -q /content/colab_powertoys 
from colab_powertoys.probml_toys import probml_toys as pmlt
%cd -q /pyprobml/scripts

pmlt.show_and_run("/pyprobml/scripts/gaussInferParamsMean2d.py")<jupyter_output><empty_output><jupyter_text>Figure 4.18:     (a) Central interval and (b) HPD region for a Beta(3,9) posterior. The CI is (0.06, 0.52) and the HPD is (0.04, 0.48). Adapted from Figure 3.6 of [Hof09] .  Figure(s) generated by [betaHPD.m](https://github.com/probml/pmtk3/blob/master/demos/betaHPD.m)<jupyter_code>#@title Setup { display-mode: "form" }
%%time
# If you run this for the first time it would take ~25/30 seconds
!git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null && git clone https://github.com/probml/colab_powertoys.git &> /dev/null 
!pip3 install nbimporter -qqq 
%cd -q /content/colab_powertoys 
from colab_powertoys.probml_toys import probml_toys as pmlt
%cd -q /pyprobml/scripts

pmlt.show_image("/pyprobml/book1/figures/images/Figure_4.18.png")
pmlt.show_image("/pyprobml/book1/figures/images/Figure_4.18.png")<jupyter_output><empty_output><jupyter_text>Figure 4.19:     (a) Central interval and (b) HPD region for a hypothetical multimodal posterior. Adapted from Figure 2.2 of [Gel+04] .  Figure(s) generated by [postDensityIntervals.py](https://github.com/probml/pyprobml/blob/master/scripts/postDensityIntervals.py)<jupyter_code>#@title Setup { display-mode: "form" }
%%time
# If you run this for the first time it would take ~25/30 seconds
!git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null && git clone https://github.com/probml/colab_powertoys.git &> /dev/null 
!pip3 install nbimporter -qqq 
%cd -q /content/colab_powertoys 
from colab_powertoys.probml_toys import probml_toys as pmlt
%cd -q /pyprobml/scripts

pmlt.show_and_run("/pyprobml/scripts/postDensityIntervals.py")<jupyter_output><empty_output><jupyter_text>Figure 4.20:     (a) Logistic regression for classifying if an Iris flower is versicolor ($y=1$) or setosa ($y=0$) using a single input feature $x$ corresponding to sepal length. Labeled points have been (vertically) jittered to avoid overlapping too much. Vertical line is the decision boundary.  Figure(s) generated by [logreg_iris_1d.py](https://github.com/probml/pyprobml/blob/master/scripts/logreg_iris_1d.py) [logreg_iris_bayes_1d_pymc3.py](https://github.com/probml/pyprobml/blob/master/scripts/logreg_iris_bayes_1d_pymc3.py)<jupyter_code>#@title Setup { display-mode: "form" }
%%time
# If you run this for the first time it would take ~25/30 seconds
!git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null && git clone https://github.com/probml/colab_powertoys.git &> /dev/null 
!pip3 install nbimporter -qqq 
%cd -q /content/colab_powertoys 
from colab_powertoys.probml_toys import probml_toys as pmlt
%cd -q /pyprobml/scripts

pmlt.show_and_run("/pyprobml/scripts/logreg_iris_1d.py")
pmlt.show_and_run("/pyprobml/scripts/logreg_iris_bayes_1d_pymc3.py")<jupyter_output><empty_output><jupyter_text>Figure 4.21:     Distribution of arrival times for two different shipping companies. ETA is the expected time of arrival. A's distribution has greater uncertainty, and may be too risky. From   https://bit.ly/39bc4XL . Used with kind permission of <NAME>.<jupyter_code>#@title Setup { display-mode: "form" }
%%time
# If you run this for the first time it would take ~25/30 seconds
!git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null && git clone https://github.com/probml/colab_powertoys.git &> /dev/null 
!pip3 install nbimporter -qqq 
%cd -q /content/colab_powertoys 
from colab_powertoys.probml_toys import probml_toys as pmlt
%cd -q /pyprobml/scripts

pmlt.show_image("/pyprobml/book1/figures/images/Figure_4.21.png")<jupyter_output><empty_output><jupyter_text>Figure 4.22:     Approximating the posterior of a beta-Bernoulli model. (a) Grid approximation using 20 grid points. (b) Laplace approximation.  Figure(s) generated by [beta_binom_approx_post_pymc3.py](https://github.com/probml/pyprobml/blob/master/scripts/beta_binom_approx_post_pymc3.py)<jupyter_code>#@title Setup { display-mode: "form" }
%%time
# If you run this for the first time it would take ~25/30 seconds
!git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null && git clone https://github.com/probml/colab_powertoys.git &> /dev/null 
!pip3 install nbimporter -qqq 
%cd -q /content/colab_powertoys 
from colab_powertoys.probml_toys import probml_toys as pmlt
%cd -q /pyprobml/scripts

pmlt.show_and_run("/pyprobml/scripts/beta_binom_approx_post_pymc3.py")<jupyter_output><empty_output><jupyter_text>Figure 4.23:     Bootstrap (top row) vs Bayes (bottom row). The $N$ data cases were generated from $\mathrm  Ber (\theta =0.7)$. Left column: $N=10$. Right column: $N=100$. (a-b) A bootstrap approximation to the sampling distribution of the MLE for a Bernoulli distribution. We show the histogram derived from $B=10,000$ bootstrap samples. (c-d) Histogram of 10,000 samples from the posterior distribution using a uniform prior.  Figure(s) generated by [bootstrapDemoBer.py](https://github.com/probml/pyprobml/blob/master/scripts/bootstrapDemoBer.py)<jupyter_code>#@title Setup { display-mode: "form" }
%%time
# If you run this for the first time it would take ~25/30 seconds
!git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null && git clone https://github.com/probml/colab_powertoys.git &> /dev/null 
!pip3 install nbimporter -qqq 
%cd -q /content/colab_powertoys 
from colab_powertoys.probml_toys import probml_toys as pmlt
%cd -q /pyprobml/scripts

pmlt.show_and_run("/pyprobml/scripts/bootstrapDemoBer.py")<jupyter_output><empty_output><jupyter_text>Figure 4.24:     Left: Sampling distribution of the MAP estimate (equivalent to the posterior mean) under a $\mathcal  N (\theta _0=0,\sigma ^2/\kappa _0)$ prior with different prior strengths $\kappa _0$. (If we set $\kappa =0$, the MAP estimate reduces to the MLE.) The data is $n=5$ samples drawn from $\mathcal  N (\theta ^*=1,\sigma ^2=1)$. Right: MSE relative to that of the MLE versus sample size. Adapted from Figure 5.6 of [Hof09] .  Figure(s) generated by [samplingDistributionGaussianShrinkage.py](https://github.com/probml/pyprobml/blob/master/scripts/samplingDistributionGaussianShrinkage.py)<jupyter_code>#@title Setup { display-mode: "form" }
%%time
# If you run this for the first time it would take ~25/30 seconds
!git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null && git clone https://github.com/probml/colab_powertoys.git &> /dev/null 
!pip3 install nbimporter -qqq 
%cd -q /content/colab_powertoys 
from colab_powertoys.probml_toys import probml_toys as pmlt
%cd -q /pyprobml/scripts

pmlt.show_and_run("/pyprobml/scripts/samplingDistributionGaussianShrinkage.py")<jupyter_output><empty_output><jupyter_text>Figure 4.25:     Illustration of bias-variance tradeoff for ridge regression. We generate 100 data sets from the true function, shown in solid green. Left: we plot the regularized fit for 20 different data sets. We use linear regression with a Gaussian RBF expansion, with 25 centers evenly spread over the $[0,1]$ interval. Right: we plot the average of the fits, averaged over all 100 datasets. Top row: strongly regularized: we see that the individual fits are similar to each other (low variance), but the average is far from the truth (high bias). Bottom row: lightly regularized: we see that the individual fits are quite different from each other (high variance), but the average is close to the truth (low bias). Adapted from [Bis06]  Figure 3.5.  Figure(s) generated by [biasVarModelComplexity3.py](https://github.com/probml/pyprobml/blob/master/scripts/biasVarModelComplexity3.py)<jupyter_code>#@title Setup { display-mode: "form" }
%%time
# If you run this for the first time it would take ~25/30 seconds
!git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null && git clone https://github.com/probml/colab_powertoys.git &> /dev/null 
!pip3 install nbimporter -qqq 
%cd -q /content/colab_powertoys 
from colab_powertoys.probml_toys import probml_toys as pmlt
%cd -q /pyprobml/scripts

pmlt.show_and_run("/pyprobml/scripts/biasVarModelComplexity3.py")<jupyter_output><empty_output><jupyter_text>Figure 4.26:     Cartoon illustration of the bias variance tradeoff. From   http://scott.fortmann-roe.com/docs/BiasVariance.html . Used with kind permission of <NAME>.<jupyter_code>#@title Setup { display-mode: "form" }
%%time
# If you run this for the first time it would take ~25/30 seconds
!git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null && git clone https://github.com/probml/colab_powertoys.git &> /dev/null 
!pip3 install nbimporter -qqq 
%cd -q /content/colab_powertoys 
from colab_powertoys.probml_toys import probml_toys as pmlt
%cd -q /pyprobml/scripts

pmlt.show_image("/pyprobml/book1/figures/images/Figure_4.26.png")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11736419576999878, 'fraction_numerical': 0.017686132279228984, 'mean_word_length': 4.863799283154122, 'pattern_counts': {'":': 0, '<': 108, '<?xml version=': 0, '>': 160, 'https://': 77, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '26', 'dir': 'jupyter-structured-clean-dedup', 'id': '251433', 'license': 'MIT', 'n_tokens_mistral': 9025, 'n_tokens_neox': 8627, 'n_words': 2350, 'path': 'book1/figures/chapter4_statistics_figures.ipynb', 'repo_name': 'ashishpapanai/pyprobml', 'size': '24563'}