<jupyter_start><jupyter_text>Cloning the pyprobml repo<jupyter_code>!git clone https://github.com/probml/pyprobml 
%cd pyprobml/scripts<jupyter_output><empty_output><jupyter_text>Installing required software (This may take few minutes)<jupyter_code>!apt install octave  -qq > /dev/null
!apt-get install liboctave-dev -qq > /dev/null<jupyter_output><empty_output><jupyter_text>Figure 20.1:   An illustration of PCA where we project from 2d to 1d. Circles are the original data points, crosses are the reconstructions. The red star is the data mean.  Figure(s) generated by [pcaDemo2d.py](https://github.com/probml/pyprobml/blob/master/scripts/pcaDemo2d.py)<jupyter_code>%run ./pcaDemo2d.py<jupyter_output><empty_output><jupyter_text>Figure 20.2:   An illustration of PCA applied to MNIST digits from class 9. Grid points are at the 5, 25, 50, 75, 95 \% quantiles of the data distribution along each dimension. The circled points are the closest projected images to the vertices of the grid. Adapted from Figure 14.23 of \citep  HastieBook .  Figure(s) generated by [pca_digits.py](https://github.com/probml/pyprobml/blob/master/scripts/pca_digits.py)<jupyter_code>%run ./pca_digits.py<jupyter_output><empty_output><jupyter_text>Figure 20.3:   a) Some randomly chosen $64 \times 64$ pixel images from the Olivetti face database. (b) The mean and the first three PCA components represented as images.  Figure(s) generated by [pcaImageDemo.m](https://github.com/probml/pmtk3/blob/master/demos/pcaImageDemo.m)<jupyter_code>!octave -W pcaImageDemo.m >> _<jupyter_output><empty_output><jupyter_text>Figure 20.4:   Illustration of the variance of the points projected onto different 1d vectors. $v_1$ is the first principal component, which maximizes the variance of the projection. $v_2$ is the second principal component which is direction orthogonal to $v_1$. Finally $v'$ is some other vector in between $v_1$ and $v_2$. Adapted from Figure 8.7 of \citep  Geron2019 .  Figure(s) generated by [pca_projected_variance.py](https://github.com/probml/pyprobml/blob/master/scripts/pca_projected_variance.py)<jupyter_code>%run ./pca_projected_variance.py<jupyter_output><empty_output><jupyter_text>Figure 20.5:   Effect of standardization on PCA applied to the height/weight dataset. (Red=female, blue=male.) Left: PCA of raw data. Right: PCA of standardized data.  Figure(s) generated by [pcaStandardization.py](https://github.com/probml/pyprobml/blob/master/scripts/pcaStandardization.py)<jupyter_code>%run ./pcaStandardization.py<jupyter_output><empty_output><jupyter_text>Figure 20.6:   Reconstruction error on MNIST vs number of latent dimensions used by PCA. (a) Training set. (b) Test set.  Figure(s) generated by [pcaOverfitDemo.m](https://github.com/probml/pmtk3/blob/master/demos/pcaOverfitDemo.m)<jupyter_code>!octave -W pcaOverfitDemo.m >> _<jupyter_output><empty_output><jupyter_text>Figure 20.7:   (a) Scree plot for training set, corresponding to \cref  fig:pcaErr (a). (b) Fraction of variance explained.  Figure(s) generated by [pcaOverfitDemo.m](https://github.com/probml/pmtk3/blob/master/demos/pcaOverfitDemo.m)<jupyter_code>!octave -W pcaOverfitDemo.m >> _<jupyter_output><empty_output><jupyter_text>Figure 20.8:   Profile likelihood corresponding to PCA model in \cref  fig:pcaErr (a).  Figure(s) generated by [pcaOverfitDemo.m](https://github.com/probml/pmtk3/blob/master/demos/pcaOverfitDemo.m)<jupyter_code>!octave -W pcaOverfitDemo.m >> _<jupyter_output><empty_output><jupyter_text>Figure 20.10:   Illustration of EM for PCA when $D=2$ and $L=1$. Green stars are the original data points, black circles are their reconstructions. The weight vector $\mathbf  w $ is represented by blue line. (a) We start with a random initial guess of $\mathbf  w $. The E step is represented by the orthogonal projections. (b) We update the rod $\mathbf  w $ in the M step, keeping the projections onto the rod (black circles) fixed. (c) Another E step. The black circles can 'slide' along the rod, but the rod stays fixed. (d) Another M step. Adapted from Figure 12.12 of \citep  BishopBook .  Figure(s) generated by [pcaEmStepByStep.m](https://github.com/probml/pmtk3/blob/master/demos/pcaEmStepByStep.m)<jupyter_code>!octave -W pcaEmStepByStep.m >> _<jupyter_output><empty_output><jupyter_text>Figure 20.12:   Mixture of PPCA models fit to a 2d dataset, using $L=1$ latent dimensions and $K=1$ and $K=10$ mixture components.  Figure(s) generated by [mixPpcaDemoNetlab.m](https://github.com/probml/pmtk3/blob/master/demos/mixPpcaDemoNetlab.m)<jupyter_code>!octave -W mixPpcaDemoNetlab.m >> _<jupyter_output><empty_output><jupyter_text>Figure 20.14:   (a) 150 synthetic 16 dimensional bit vectors. (b) The 2d embedding learned by binary PCA, fit using variational EM. We have color coded points by the identity of the true ``prototype'' that generated them. (c) Predicted probability of being on. (d) Thresholded predictions.  Figure(s) generated by [binaryFaDemoTipping.m](https://github.com/probml/pmtk3/blob/master/demos/binaryFaDemoTipping.m)<jupyter_code>!octave -W binaryFaDemoTipping.m >> _<jupyter_output><empty_output><jupyter_text>Figure 20.30:   Illustration of some data generated from low-dimensional manifolds. (a) The 2d Swiss-roll manifold embedded into 3d.  Figure(s) generated by [manifold_swiss_sklearn.py](https://github.com/probml/pyprobml/blob/master/scripts/manifold_swiss_sklearn.py) [manifold_digits_sklearn.py](https://github.com/probml/pyprobml/blob/master/scripts/manifold_digits_sklearn.py)<jupyter_code>%run ./manifold_swiss_sklearn.py
%run ./manifold_digits_sklearn.py<jupyter_output><empty_output><jupyter_text>Figure 20.31:   Metric MDS applied to (a) Swiss roll.  Figure(s) generated by [manifold_swiss_sklearn.py](https://github.com/probml/pyprobml/blob/master/scripts/manifold_swiss_sklearn.py) [manifold_digits_sklearn.py](https://github.com/probml/pyprobml/blob/master/scripts/manifold_digits_sklearn.py)<jupyter_code>%run ./manifold_swiss_sklearn.py
%run ./manifold_digits_sklearn.py<jupyter_output><empty_output><jupyter_text>Figure 20.33:   Isomap applied to (a) Swiss roll.  Figure(s) generated by [manifold_swiss_sklearn.py](https://github.com/probml/pyprobml/blob/master/scripts/manifold_swiss_sklearn.py) [manifold_digits_sklearn.py](https://github.com/probml/pyprobml/blob/master/scripts/manifold_digits_sklearn.py)<jupyter_code>%run ./manifold_swiss_sklearn.py
%run ./manifold_digits_sklearn.py<jupyter_output><empty_output><jupyter_text>Figure 20.34:   (a) Noisy version of Swiss roll data. We perturb each point by adding $\mathcal  N (0, 0.5^2)$ noise. (b) Results of Isomap applied to this data.  Figure(s) generated by [manifold_swiss_sklearn.py](https://github.com/probml/pyprobml/blob/master/scripts/manifold_swiss_sklearn.py)<jupyter_code>%run ./manifold_swiss_sklearn.py<jupyter_output><empty_output><jupyter_text>Figure 20.35:   Visualization of the first 8 kernel principal component basis functions derived from some 2d data. We use an RBF kernel with $\sigma ^2=0.1$.  Figure(s) generated by [kpcaScholkopf.m](https://github.com/probml/pmtk3/blob/master/demos/kpcaScholkopf.m)<jupyter_code>!octave -W kpcaScholkopf.m >> _<jupyter_output><empty_output><jupyter_text>Figure 20.36:   Kernel PCA applied to (a) Swiss roll.  Figure(s) generated by [manifold_swiss_sklearn.py](https://github.com/probml/pyprobml/blob/master/scripts/manifold_swiss_sklearn.py) [manifold_digits_sklearn.py](https://github.com/probml/pyprobml/blob/master/scripts/manifold_digits_sklearn.py)<jupyter_code>%run ./manifold_swiss_sklearn.py
%run ./manifold_digits_sklearn.py<jupyter_output><empty_output><jupyter_text>Figure 20.37:   LLE applied to (a) Swiss roll.  Figure(s) generated by [manifold_swiss_sklearn.py](https://github.com/probml/pyprobml/blob/master/scripts/manifold_swiss_sklearn.py) [manifold_digits_sklearn.py](https://github.com/probml/pyprobml/blob/master/scripts/manifold_digits_sklearn.py)<jupyter_code>%run ./manifold_swiss_sklearn.py
%run ./manifold_digits_sklearn.py<jupyter_output><empty_output><jupyter_text>Figure 20.38:   Laplacian eigenmaps applied to (a) Swiss roll.  Figure(s) generated by [manifold_swiss_sklearn.py](https://github.com/probml/pyprobml/blob/master/scripts/manifold_swiss_sklearn.py) [manifold_digits_sklearn.py](https://github.com/probml/pyprobml/blob/master/scripts/manifold_digits_sklearn.py)<jupyter_code>%run ./manifold_swiss_sklearn.py
%run ./manifold_digits_sklearn.py<jupyter_output><empty_output><jupyter_text>Figure 20.41:   tSNE applied to (a) Swiss roll.  Figure(s) generated by [manifold_swiss_sklearn.py](https://github.com/probml/pyprobml/blob/master/scripts/manifold_swiss_sklearn.py) [manifold_digits_sklearn.py](https://github.com/probml/pyprobml/blob/master/scripts/manifold_digits_sklearn.py)<jupyter_code>%run ./manifold_swiss_sklearn.py
%run ./manifold_digits_sklearn.py<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11540622541858636, 'fraction_numerical': 0.016069221260815822, 'mean_word_length': 6.200647249190938, 'pattern_counts': {'":': 0, '<': 89, '<?xml version=': 0, '>': 107, 'https://': 28, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '22', 'dir': 'jupyter-structured-clean-dedup', 'id': '490280', 'license': 'MIT', 'n_tokens_mistral': 3341, 'n_tokens_neox': 3073, 'n_words': 671, 'path': 'notebooks/figures/chapter20_figures.ipynb', 'repo_name': 'NadzeyaKadakova/pyprobml', 'size': '8899'}