Notebooks about Bayesian methods for machine learning
## New default branch 

**Branch [master](https://github.com/krasserm/bayesian-machine-learning/tree/master) is no longer the default branch in 
this repository. Please use branch [dev](https://github.com/krasserm/bayesian-machine-learning/tree/dev) instead.** 

## Bayesian machine learning notebooks

[![DOI](https://zenodo.org/badge/125869131.svg)](https://zenodo.org/badge/latestdoi/125869131)

This repository is a collection of notebooks about *Bayesian Machine Learning*. The following links display 
the notebooks via [nbviewer](https://nbviewer.jupyter.org/) to ensure a proper rendering of formulas. **Update:** 
PyMC3 and PyMC4 implementations are now available for some notebooks (more planned).

- [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/krasserm/bayesian-machine-learning/blob/master/latent_variable_models_part_1.ipynb)
  [Latent variable models, part 1: Gaussian mixture models and the EM algorithm](https://nbviewer.jupyter.org/github/krasserm/bayesian-machine-learning/blob/master/latent_variable_models_part_1.ipynb).
  Introduction to the expectation maximization (EM) algorithm and its application to Gaussian mixture models. Example
  implementation with plain NumPy/SciPy and scikit-learn for comparison (see also 
  [PyMC3 implementation](https://nbviewer.jupyter.org/github/krasserm/bayesian-machine-learning/blob/master/latent_variable_models_part_1_pymc3.ipynb)).

- [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/krasserm/bayesian-machine-learning/blob/master/latent_variable_models_part_2.ipynb)
  [Latent variable models, part 2: Stochastic variational inference and variational autoencoders](https://nbviewer.jupyter.org/github/krasserm/bayesian-machine-learning/blob/master/latent_variable_models_part_2.ipynb). 
  Introduction to stochastic variational inference with variational autoencoder as application example. Implementation 
  with Tensorflow 2.x.

- [Variational inference in Bayesian neural networks](https://nbviewer.jupyter.org/github/krasserm/bayesian-machine-learning/blob/master/bayesian_neural_networks.ipynb). Demonstrates how to 
  implement and train a Bayesian neural network using a variational inference approach. Example implementation with Keras (see also 
  [PyMC4 implementation](https://nbviewer.jupyter.org/github/krasserm/bayesian-machine-learning/blob/wip-bnn-pymc4/bayesian_neural_networks_pymc4.ipynb)).

- [Bayesian regression with linear basis function models](https://nbviewer.jupyter.org/github/krasserm/bayesian-machine-learning/blob/master/bayesian_linear_regression.ipynb). Introduction to Bayesian
  linear regression. Implementation from scratch with plain NumPy as well as usage of scikit-learn for comparison (see also 
  [PyMC4 implementation](https://nbviewer.jupyter.org/github/krasserm/bayesian-machine-learning/blob/master/bayesian_linear_regression_pymc4.ipynb) and 
  [PyMC3 implementation](https://nbviewer.jupyter.org/github/krasserm/bayesian-machine-learning/blob/master/bayesian_linear_regression_pymc3.ipynb)).

- [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/krasserm/bayesian-machine-learning/blob/master/gaussian_processes.ipynb)
  [Gaussian processes](https://nbviewer.jupyter.org/github/krasserm/bayesian-machine-learning/blob/master/gaussian_processes.ipynb). 
  Introduction to Gaussian processes. Example implementations with plain NumPy/SciPy as well as with libraries 
  scikit-learn and GPy. 

- [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/krasserm/bayesian-machine-learning/blob/master/bayesian_optimization.ipynb)
  [Bayesian optimization](https://nbviewer.jupyter.org/github/krasserm/bayesian-machine-learning/blob/master/bayesian_optimization.ipynb). 
  Introduction to Bayesian optimization. Example implementations with plain NumPy/SciPy as well as with libraries 
  scikit-optimize and GPyOpt. Hyperparameter tuning as application example.  

- [Deep feature consistent variational autoencoder](https://nbviewer.jupyter.org/github/krasserm/bayesian-machine-learning/blob/master/variational_autoencoder_dfc.ipynb). 
  Describes how a perceptual loss can improve the quality of images generated by a variational autoencoder. Example 
  implementation with Keras.  

- [Conditional generation via Bayesian optimization in latent space](https://nbviewer.jupyter.org/github/krasserm/bayesian-machine-learning/blob/master/variational_autoencoder_opt.ipynb). 
  Describes an approach for conditionally generating outputs with desired properties by doing Bayesian optimization in 
  latent space of variational autoencoders. Example application implemented with Keras and GPyOpt.
