<jupyter_start><jupyter_text>Figure 10.1:     (a) Visualization of a 2d plane in a 3d space with surface normal $\mathbf  w $ going through point $\mathbf  x _0=(x_0,y_0,z_0)$. See text for details. (b) Visualization of optimal linear decision boundary induced by logistic regression on a 2-class, 2-feature version of the iris dataset.  Figure(s) generated by [iris_logreg.py](https://github.com/probml/pyprobml/blob/master/scripts/iris_logreg.py)<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  import pyprobml_utils as pml
  import colab_utils
  import os
  os.environ["PYPROBML"] = ".." # one above current scripts directory
  import google.colab 
  from google.colab.patches import cv2_imshow
  %reload_ext autoreload 
  %autoreload 2
  def show_image(img_path,size=None,ratio=None):
      img = colab_utils.image_resize(img_path, size)
      cv2_imshow(img)
  print('finished!')
google.colab.files.view("./iris_logreg.py")
%run iris_logreg.py<jupyter_output><empty_output><jupyter_text>Figure 10.2:     Plots of $\sigma (w_1 x_1 + w_2 x_2)$. Here $\mathbf  w = (w_1,w_2)$ defines the normal to the decision boundary. Points to the right of this have $\sigma (\mathbf  w ^\top \mathbf  x )>0.5$, and points to the left have $\sigma (\mathbf  w ^\top \mathbf  x ) [Mac03] .  Figure(s) generated by [sigmoid_2d_plot.py](https://github.com/probml/pyprobml/blob/master/scripts/sigmoid_2d_plot.py)<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  import pyprobml_utils as pml
  import colab_utils
  import os
  os.environ["PYPROBML"] = ".." # one above current scripts directory
  import google.colab 
  from google.colab.patches import cv2_imshow
  %reload_ext autoreload 
  %autoreload 2
  def show_image(img_path,size=None,ratio=None):
      img = colab_utils.image_resize(img_path, size)
      cv2_imshow(img)
  print('finished!')
google.colab.files.view("./sigmoid_2d_plot.py")
%run sigmoid_2d_plot.py<jupyter_output><empty_output><jupyter_text>Figure 10.3:     Illustration of how we can transform a quadratic decision boundary into a linear one by transforming the features from $\mathbf  x =(x_1,x_2)$ to $\boldsymbol  \phi  (\mathbf  x )=(x_1^2,x_2^2)$. Used with kind permission of <NAME><jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  import pyprobml_utils as pml
  import colab_utils
  import os
  os.environ["PYPROBML"] = ".." # one above current scripts directory
  import google.colab 
  from google.colab.patches import cv2_imshow
  %reload_ext autoreload 
  %autoreload 2
  def show_image(img_path,size=None,ratio=None):
      img = colab_utils.image_resize(img_path, size)
      cv2_imshow(img)
  print('finished!')
show_image("/pyprobml/book1/figures/images/Figure_10.3.png")<jupyter_output><empty_output><jupyter_text>Figure 10.4:     Polynomial feature expansion applied to a two-class, two-dimensional logistic regression problem. (a) Degree $K=1$. (b) Degree $K=2$. (c) Degree $K=4$. (d) Train and test error vs degree.  Figure(s) generated by [logreg_poly_demo.py](https://github.com/probml/pyprobml/blob/master/scripts/logreg_poly_demo.py)<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  import pyprobml_utils as pml
  import colab_utils
  import os
  os.environ["PYPROBML"] = ".." # one above current scripts directory
  import google.colab 
  from google.colab.patches import cv2_imshow
  %reload_ext autoreload 
  %autoreload 2
  def show_image(img_path,size=None,ratio=None):
      img = colab_utils.image_resize(img_path, size)
      cv2_imshow(img)
  print('finished!')
google.colab.files.view("./logreg_poly_demo.py")
%run logreg_poly_demo.py<jupyter_output><empty_output><jupyter_text>Figure 10.5:     NLL loss surface for binary logistic regression applied to Iris dataset with 1 feature and 1 bias term. The goal is to minimize the function.  Figure(s) generated by [iris_logreg_loss_surface.py](https://github.com/probml/pyprobml/blob/master/scripts/iris_logreg_loss_surface.py)<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  import pyprobml_utils as pml
  import colab_utils
  import os
  os.environ["PYPROBML"] = ".." # one above current scripts directory
  import google.colab 
  from google.colab.patches import cv2_imshow
  %reload_ext autoreload 
  %autoreload 2
  def show_image(img_path,size=None,ratio=None):
      img = colab_utils.image_resize(img_path, size)
      cv2_imshow(img)
  print('finished!')
google.colab.files.view("./iris_logreg_loss_surface.py")
%run iris_logreg_loss_surface.py<jupyter_output><empty_output><jupyter_text>Figure 10.6:     Weight decay with variance $C$ applied to two-class, two-dimensional logistic regression problem with a degree 4 polynomial. (a) $C=1$. (b) $C=316$. (c) $C=100,000$. (d) Train and test error vs $C$.  Figure(s) generated by [logreg_poly_demo.py](https://github.com/probml/pyprobml/blob/master/scripts/logreg_poly_demo.py)<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  import pyprobml_utils as pml
  import colab_utils
  import os
  os.environ["PYPROBML"] = ".." # one above current scripts directory
  import google.colab 
  from google.colab.patches import cv2_imshow
  %reload_ext autoreload 
  %autoreload 2
  def show_image(img_path,size=None,ratio=None):
      img = colab_utils.image_resize(img_path, size)
      cv2_imshow(img)
  print('finished!')
google.colab.files.view("./logreg_poly_demo.py")
%run logreg_poly_demo.py<jupyter_output><empty_output><jupyter_text>Figure 10.7:     Example of 3-class logistic regression with 2d inputs. (a) Original features. (b) Quadratic features.  Figure(s) generated by [logreg_multiclass_demo.py](https://github.com/probml/pyprobml/blob/master/scripts/logreg_multiclass_demo.py)<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  import pyprobml_utils as pml
  import colab_utils
  import os
  os.environ["PYPROBML"] = ".." # one above current scripts directory
  import google.colab 
  from google.colab.patches import cv2_imshow
  %reload_ext autoreload 
  %autoreload 2
  def show_image(img_path,size=None,ratio=None):
      img = colab_utils.image_resize(img_path, size)
      cv2_imshow(img)
  print('finished!')
google.colab.files.view("./logreg_multiclass_demo.py")
%run logreg_multiclass_demo.py<jupyter_output><empty_output><jupyter_text>Figure 10.8:     A simple example of a label hierarchy. Nodes within the same ellipse have a mutual exclusion relationship between them.<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  import pyprobml_utils as pml
  import colab_utils
  import os
  os.environ["PYPROBML"] = ".." # one above current scripts directory
  import google.colab 
  from google.colab.patches import cv2_imshow
  %reload_ext autoreload 
  %autoreload 2
  def show_image(img_path,size=None,ratio=None):
      img = colab_utils.image_resize(img_path, size)
      cv2_imshow(img)
  print('finished!')
show_image("/pyprobml/book1/figures/images/Figure_10.8.png")<jupyter_output><empty_output><jupyter_text>Figure 10.9:     A flat and hierarchical softmax model $p(w|C)$, where $C$ are the input features (context) and $w$ is the output label (word). Adapted from   https://www.quora.com/What-is-hierarchical-softmax .<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  import pyprobml_utils as pml
  import colab_utils
  import os
  os.environ["PYPROBML"] = ".." # one above current scripts directory
  import google.colab 
  from google.colab.patches import cv2_imshow
  %reload_ext autoreload 
  %autoreload 2
  def show_image(img_path,size=None,ratio=None):
      img = colab_utils.image_resize(img_path, size)
      cv2_imshow(img)
  print('finished!')
show_image("/pyprobml/book1/figures/images/Figure_10.9_A.png")
show_image("/pyprobml/book1/figures/images/Figure_10.9_B.png")<jupyter_output><empty_output><jupyter_text>Figure 10.10:     (a) Logistic regression on some data with outliers (denoted by x). Training points have been (vertically) jittered to avoid overlapping too much. Vertical line is the decision boundary, and its posterior credible interval. (b) Same as (a) but using robust model, with a mixture likelihood. Adapted from Figure 4.13 of [Mar18] .  Figure(s) generated by [logreg_iris_bayes_robust_1d_pymc3.py](https://github.com/probml/pyprobml/blob/master/scripts/logreg_iris_bayes_robust_1d_pymc3.py)<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  import pyprobml_utils as pml
  import colab_utils
  import os
  os.environ["PYPROBML"] = ".." # one above current scripts directory
  import google.colab 
  from google.colab.patches import cv2_imshow
  %reload_ext autoreload 
  %autoreload 2
  def show_image(img_path,size=None,ratio=None):
      img = colab_utils.image_resize(img_path, size)
      cv2_imshow(img)
  print('finished!')
google.colab.files.view("./logreg_iris_bayes_robust_1d_pymc3.py")
%run logreg_iris_bayes_robust_1d_pymc3.py<jupyter_output><empty_output><jupyter_text>Figure 10.11:     (a) Illustration of logistic and tempered logistic loss with $t_1=0.8$. (b) Illustration of sigmoid and tempered sigmoid transfer function with $t_2=2.0$. From   https://ai.googleblog.com/2019/08/bi-tempered-logistic-loss-for-training.html . Used with kind permission of Ehsan Amid.<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  import pyprobml_utils as pml
  import colab_utils
  import os
  os.environ["PYPROBML"] = ".." # one above current scripts directory
  import google.colab 
  from google.colab.patches import cv2_imshow
  %reload_ext autoreload 
  %autoreload 2
  def show_image(img_path,size=None,ratio=None):
      img = colab_utils.image_resize(img_path, size)
      cv2_imshow(img)
  print('finished!')
show_image("/pyprobml/book1/figures/images/Figure_10.11_A.png")
show_image("/pyprobml/book1/figures/images/Figure_10.11_B.png")<jupyter_output><empty_output><jupyter_text>Figure 10.12:     Illustration of standard and bi-tempered logistic regression on data with label noise. From   https://ai.googleblog.com/2019/08/bi-tempered-logistic-loss-for-training.html . Used with kind permission of Ehsan Amid.<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  import pyprobml_utils as pml
  import colab_utils
  import os
  os.environ["PYPROBML"] = ".." # one above current scripts directory
  import google.colab 
  from google.colab.patches import cv2_imshow
  %reload_ext autoreload 
  %autoreload 2
  def show_image(img_path,size=None,ratio=None):
      img = colab_utils.image_resize(img_path, size)
      cv2_imshow(img)
  print('finished!')
show_image("/pyprobml/book1/figures/images/Figure_10.12.png")<jupyter_output><empty_output><jupyter_text>Figure 10.13:     (a) Illustration of the data. (b) Log-likelihood for a logistic regression model. The line is drawn from the origin in the direction of the MLE (which is at infinity). The numbers correspond to 4 points in parameter space, corresponding to the lines in (a). (c) Unnormalized log posterior (assuming vague spherical prior). (d) Laplace approximation to posterior. Adapted from a figure by <NAME>.  Figure(s) generated by [logregLaplaceGirolamiDemo.py](https://github.com/probml/pyprobml/blob/master/scripts/logregLaplaceGirolamiDemo.py)<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  import pyprobml_utils as pml
  import colab_utils
  import os
  os.environ["PYPROBML"] = ".." # one above current scripts directory
  import google.colab 
  from google.colab.patches import cv2_imshow
  %reload_ext autoreload 
  %autoreload 2
  def show_image(img_path,size=None,ratio=None):
      img = colab_utils.image_resize(img_path, size)
      cv2_imshow(img)
  print('finished!')
google.colab.files.view("./logregLaplaceGirolamiDemo.py")
%run logregLaplaceGirolamiDemo.py<jupyter_output><empty_output><jupyter_text>Figure 10.14:     Posterior predictive distribution for a logistic regression model in 2d. Top left: contours of $p(y=1|\mathbf  x , \mathbf  w  _ map )$. Top right: samples from the posterior predictive distribution. Bottom left: Averaging over these samples. Bottom right: moderated output (probit approximation). Adapted from a figure by <NAME>.  Figure(s) generated by [logregLaplaceGirolamiDemo.py](https://github.com/probml/pyprobml/blob/master/scripts/logregLaplaceGirolamiDemo.py)<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  import pyprobml_utils as pml
  import colab_utils
  import os
  os.environ["PYPROBML"] = ".." # one above current scripts directory
  import google.colab 
  from google.colab.patches import cv2_imshow
  %reload_ext autoreload 
  %autoreload 2
  def show_image(img_path,size=None,ratio=None):
      img = colab_utils.image_resize(img_path, size)
      cv2_imshow(img)
  print('finished!')
google.colab.files.view("./logregLaplaceGirolamiDemo.py")
%run logregLaplaceGirolamiDemo.py<jupyter_output><empty_output><jupyter_text>Figure 10.15:     (a) Data for logistic regression question. (b) Plot of $ w _k$ vs amount of correlation $c_k$ for three different estimators.<jupyter_code>#@title Click me to run setup { display-mode: "form" }
try:
  if PYPROBML_SETUP_ALREADY_RUN:
    print('skipping setup')
except:
  PYPROBML_SETUP_ALREADY_RUN = True
  print('running setup...')
  !git clone https://github.com/probml/pyprobml /pyprobml &> /dev/null 
  %cd -q /pyprobml/scripts
  import pyprobml_utils as pml
  import colab_utils
  import os
  os.environ["PYPROBML"] = ".." # one above current scripts directory
  import google.colab 
  from google.colab.patches import cv2_imshow
  %reload_ext autoreload 
  %autoreload 2
  def show_image(img_path,size=None,ratio=None):
      img = colab_utils.image_resize(img_path, size)
      cv2_imshow(img)
  print('finished!')
show_image("/pyprobml/book1/figures/images/Figure_10.15_A.png")
show_image("/pyprobml/book1/figures/images/Figure_10.15_B.png")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1118447726614538, 'fraction_numerical': 0.012568780770344628, 'mean_word_length': 4.631441617742988, 'pattern_counts': {'":': 0, '<': 64, '<?xml version=': 0, '>': 80, 'https://': 27, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '33521', 'license': 'MIT', 'n_tokens_mistral': 6430, 'n_tokens_neox': 6153, 'n_words': 1422, 'path': 'book1/figures/chapter10_logistic_regression_figures.ipynb', 'repo_name': 'animesh-007/pyprobml', 'size': '17291'}