<jupyter_start><jupyter_text>Figure 8.1:   (a) Illustration of local and global minimum in 1d.  Figure(s) generated by [extrema_fig_1d.py](https://github.com/probml/pyprobml/blob/master/scripts/extrema_fig_1d.py) [saddle.py](https://github.com/probml/pyprobml/blob/master/scripts/saddle.py)<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')
deimport(superimport)
%run extrema_fig_1d.py
deimport(superimport)
%run saddle.py<jupyter_output><empty_output><jupyter_text>Figure 8.2:    Illustration of constrained maximization of a nonconvex 1d function. The area between the dotted vertical lines represents the feasible set. (a) There is a unique global maximum since the function is concave within the support of the feasible set. (b) There are two global maxima, both occuring at the boundary of the feasible set. (c) In the unconstrained case, this function has no global maximum, since it is unbounded<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')<jupyter_output><empty_output><jupyter_text>Figure 8.3:    Illustration of some convex and non-convex sets<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')<jupyter_output><empty_output><jupyter_text>Figure 8.4:   (a) Illustration of the epigraph of a function. (b) For a convex function $f(x)$, its epipgraph can be represented as the intersection of half-spaces defined by linear lower bounds derived from the  \bf conjugate function $f^*(\lambda ) = \qopname m max _x \lambda x - f(x)$<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')<jupyter_output><empty_output><jupyter_text>Figure 8.5:   (a) Illustration of a convex function. We see that the chord joining $(x, f(x))$ to $(y, f(y))$ lies above the function. (b) A function that is neither convex nor concave.  \bf A  is a local minimum,  \bf B  is a global minimum<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')<jupyter_output><empty_output><jupyter_text>Figure 8.6:    The quadratic form $f( \bm x  ) =  \bm x  ^  \mkern -1.5mu\mathsf T    \mathbf A  \bm x  $ in 2d. (a) $\mathbf A $ is positive definite, so $f$ is convex. (b) $\mathbf A $ is negative definite, so $f$ is concave. (c) $\mathbf A $ is positive semidefinite but singular, so $f$ is convex, but not strictly. Notice the valley of constant height in the middle. (d) $\mathbf A $ is indefinite, so $f$ is neither convex nor concave. The stationary point in the middle of the surface is a saddle point. From Figure 5 of [Jon94]<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')<jupyter_output><empty_output><jupyter_text>Figure 8.7:   (a) Smooth 1d function. (b) Non-smooth 1d function. (There is a discontinuity at the origin.)  Figure(s) generated by [smooth-vs-nonsmooth-1d.py](https://github.com/probml/pyprobml/blob/master/scripts/smooth-vs-nonsmooth-1d.py)<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')
deimport(superimport)
%run smooth-vs-nonsmooth-1d.py<jupyter_output><empty_output><jupyter_text>Figure 8.8:    For a Lipschitz continuous function $f$, there exists a double cone (white) whose origin can be moved along the graph of $f$ so that the whole graph always stays outside the double cone. From  https://en.wikipedia.org/wiki/Lipschitz\_continuity . Used with kind permission of Wikipedia author Taschee<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')<jupyter_output><empty_output><jupyter_text>Figure 8.9:    Illustration of subgradients. At $ \bm x  _1$, the convex function $f$ is differentiable, and $ \bm g  _1$(which is the derivative of $f$ at $ \bm x  _1$) is the unique subgradient at $ \bm x  _1$. At the point $ \bm x  _2$, $f$ is not differentiable, because of the ``kink''. However, there are many subgradients at this point, of which two are shown. From  https://web.stanford.edu/class/ee364b/lectures/subgradients_slides.pdf . Used with kind permission of <NAME><jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')<jupyter_output><empty_output><jupyter_text>Figure 8.10:    The absolute value function (left) and its subdifferential (right). From  https://web.stanford.edu/class/ee364b/lectures/subgradients_slides.pdf . Used with kind permission of <NAME><jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')<jupyter_output><empty_output><jupyter_text>Figure 8.11:    Steepest descent on a simple convex function, starting from $(0,0)$, for 20 steps, using a fixed step size. The global minimum is at $(1,1)$. (a) $\rho =0.1$. (b) $\rho =0.6$.  Figure(s) generated by [steepestDescentDemo.py](https://github.com/probml/pyprobml/blob/master/scripts/steepestDescentDemo.py)<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')
deimport(superimport)
%run steepestDescentDemo.py<jupyter_output><empty_output><jupyter_text>Figure 8.12:    Illustration of the effect of condition number $\kappa $ on the convergence speed of steepest descent with exact line searches. (a) Large $\kappa $. (b) Small $\kappa $.  Figure(s) generated by [lineSearchConditionNum.py](https://github.com/probml/pyprobml/blob/master/scripts/lineSearchConditionNum.py)<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')
deimport(superimport)
%run lineSearchConditionNum.py<jupyter_output><empty_output><jupyter_text>Figure 8.13:    Illustration of the Nesterov update. Adapted from Figure 11.6 of [Aur19]<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')<jupyter_output><empty_output><jupyter_text>Figure 8.14:    Illustration of Newton's method for minimizing a 1d function. (a) The solid curve is the function $\mathcal L (x)$. The dotted line $\mathcal L _ \mathrm quad  (\theta )$ is its second order approximation at $\theta _t$. The Newton step $d_t$ is what must be added to $\theta _t$ to get to the minimum of $\mathcal L _ \mathrm quad  (\theta )$. Adapted from Figure 13.4 of [Van06] .  Figure(s) generated by [newtonsMethodMinQuad.py](https://github.com/probml/pyprobml/blob/master/scripts/newtonsMethodMinQuad.py) [newtonsMethodNonConvex.py](https://github.com/probml/pyprobml/blob/master/scripts/newtonsMethodNonConvex.py)<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')
deimport(superimport)
%run newtonsMethodMinQuad.py
deimport(superimport)
%run newtonsMethodNonConvex.py<jupyter_output><empty_output><jupyter_text>Figure 8.15:    Illustration of the trust region approach. The dashed lines represents contours of the original nonconvex objective. The circles represent successive quadratic approximations. From Figure 4.2 of [Raz14] . Used with kind permission of <NAME><jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')<jupyter_output><empty_output><jupyter_text>Figure 8.16:    Illustration of the LMS algorithm. Left: we start from $ \bm \theta   =(-0.5,2)$ and slowly converging to the least squares solution of $  \bm \theta    =(1.45, 0.93)$(red cross). Right: plot of objective function over time. Note that it does not decrease monotonically.  Figure(s) generated by [lms_demo.py](https://github.com/probml/pyprobml/blob/master/scripts/lms_demo.py)<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')
deimport(superimport)
%run lms_demo.py<jupyter_output><empty_output><jupyter_text>Figure 8.17:    Loss vs learning rate (horizontal axis). Training loss vs learning rate for a small MLP fit to FashionMNIST using vanilla SGD. (Raw loss in blue, EWMA smoothed version in orange).  To reproduce this figure, click the open in colab button:<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')<jupyter_output><empty_output><jupyter_text>Figure 8.18:    Illustration of some common learning rate schedules. (a) Piecewise constant. (b) Exponential decay. (c) Polynomial decay.  Figure(s) generated by [learning_rate_plot.py](https://github.com/probml/pyprobml/blob/master/scripts/learning_rate_plot.py)<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')
deimport(superimport)
%run learning_rate_plot.py<jupyter_output><empty_output><jupyter_text>Figure 8.19:   (a) Linear warm-up followed by cosine cool-down. (b) Cyclical learning rate schedule<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')<jupyter_output><empty_output><jupyter_text>Figure 8.20:    Illustration of some constrained optimization problems. Red contours are the level sets of the objective function $\mathcal L ( \bm \theta   )$. Optimal constrained solution is the black dot, (a) Blue line is the equality constraint $h( \bm \theta   )=0$. (b) Blue lines denote the inequality constraints $|\theta _1| + |\theta _2| \leq 1$. (Compare to \cref fig:L2L1contours (left).<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')<jupyter_output><empty_output><jupyter_text>Figure 8.21:   (a) A convex polytope in 2d defined by the intersection of linear constraints. (b) Depiction of the feasible set as well as the linear objective function. The red line is a level set of the objective, and the arrow indicates the direction in which it is improving. We see that the optimal solution lies at a vertex of the polytope<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')<jupyter_output><empty_output><jupyter_text>Figure 8.22:    Illustration of projected gradient descent. $ \bm w  $ is the current parameter estimate, $ \bm w  '$ is the update after a gradient step, and $P_ \mathcal C  ( \bm w  ')$ projects this onto the constraint set $\mathcal C $. From  https://bit.ly/3eJ3BhZ  Used with kind permission of <NAME><jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')<jupyter_output><empty_output><jupyter_text>Figure 8.23:    Illustration of a bound optimization algorithm. Adapted from Figure 9.14 of [Bis06] .  Figure(s) generated by [emLogLikelihoodMax.py](https://github.com/probml/pyprobml/blob/master/scripts/emLogLikelihoodMax.py)<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')
deimport(superimport)
%run emLogLikelihoodMax.py<jupyter_output><empty_output><jupyter_text>Figure 8.24:    The quadratic lower bound of an MM algorithm (solid) and the quadratic approximation of Newton's method (dashed) superimposed on an empirical density esitmate (dotted). The starting point of both algorithms is the circle. The square denotes the outcome of one MM update. The diamond denotes the outcome of one Newton update. (a) Newton's method overshoots the-se, global maximum. (b) Newton's method results in a reduction of the objective. From Figure 4 of [MC05] . Used with kind permission of <NAME><jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')<jupyter_output><empty_output><jupyter_text>Figure 8.25:    Illustration of the EM for a GMM applied to the Old Faithful data. The degree of redness indicates the degree to which the point belongs to the red cluster, and similarly for blue; thus purple points have a roughly 50/50 split in their responsibilities to the two clusters. Adapted from [Bis06]  Figure 9.8.  Figure(s) generated by [mix_gauss_demo_faithful.py](https://github.com/probml/pyprobml/blob/master/scripts/mix_gauss_demo_faithful.py)<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')
deimport(superimport)
%run mix_gauss_demo_faithful.py<jupyter_output><empty_output><jupyter_text>Figure 8.26:   (a) Illustration of how singularities can arise in the likelihood function of GMMs. Here $K=2$, but the first mixture component is a narrow spike (with $\sigma _1 \approx 0$) centered on a single data point $x_1$. Adapted from Figure 9.7 of [Bis06] .  Figure(s) generated by [mix_gauss_singularity.py](https://github.com/probml/pyprobml/blob/master/scripts/mix_gauss_singularity.py) [mix_gauss_mle_vs_map.py](https://github.com/probml/pyprobml/blob/master/scripts/mix_gauss_mle_vs_map.py)<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')
deimport(superimport)
%run mix_gauss_singularity.py
deimport(superimport)
%run mix_gauss_mle_vs_map.py<jupyter_output><empty_output><jupyter_text>Figure 8.27:    Left: $N=200$ data points sampled from a mixture of 2 Gaussians in 1d, with $\pi _k=0.5$, $\sigma _k=5$, $\mu _1=-10$ and $\mu _2=10$. Right: Likelihood surface $p( \mathcal D  |\mu _1,\mu _2)$, with all other parameters set to their true values. We see the two symmetric modes, reflecting the unidentifiability of the parameters.  Figure(s) generated by [gmm_lik_surface_plot.py](https://github.com/probml/pyprobml/blob/master/scripts/gmm_lik_surface_plot.py)<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone --depth 1 https://github.com/probml/pyprobml  /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  %reload_ext autoreload 
  %autoreload 2
  !pip install superimport deimport -qqq
  import superimport
  from deimport.deimport import deimport
  print('finished!')
deimport(superimport)
%run gmm_lik_surface_plot.py<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10514607118871726, 'fraction_numerical': 0.009905977165883144, 'mean_word_length': 4.501039021011314, 'pattern_counts': {'":': 0, '<': 114, '<?xml version=': 0, '>': 141, 'https://': 45, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '27', 'dir': 'jupyter-structured-clean-dedup', 'id': '109343', 'license': 'MIT', 'n_tokens_mistral': 8484, 'n_tokens_neox': 8049, 'n_words': 2352, 'path': 'pml1/figure_notebooks/chapter8_optimization_figures.ipynb', 'repo_name': 'hanchenresearch/pml-book', 'size': '23856'}