<commit_before>---
layout: post
title: "Scaling HMC to larger data sets"
categories: journal
tags: [documentation,sample]
image:
  feature: cumulative_pred_ent_emoji.png
  teaser: cumulative_pred_ent_emoji.png
  credit:
  creditlink:
---
In this blog post I will demonstrate how to use [*hamiltorch*](https://github.com/AdamCobb/hamiltorch) for inference in Bayesian neural networks with larger data sets. Further details can be found in [*"Scaling Hamiltonian Monte Carlo Inference for Bayesian Neural Networks with Symmetric Splitting"*](https://github.com/AdamCobb/hamiltorch/tree/master/notebooks).

## *hamiltorch* for Bayesian neural networks

Towards the end of my previous [post]({{ site.github.url }}{% link _posts/2019-07-15-hamiltorch.md %}) I gave a basic example of how to use `hamiltorch` for inference in
Bayesian neural networks (BNNs). Since then, my aim has been to make it as easy as possible to apply HMC over as many PyTorch models as possible. This includes being able to work with a `torch.utils.data.Dataloader` in the same way as for stochastic gradient descent. However, the main addition since the last post, has been to investigate split HMC and come up with a new way of scaling HMC to larger data sets with a single GPU. Interestingly, we will actually see that `hamiltorch` can compete with stochastic gradient approaches and also provide better uncertainty quantification.<commit_msg>Adding paper link<commit_after>---
layout: post
title: "Scaling HMC to larger data sets"
categories: journal
tags: [documentation,sample]
image:
  feature: cumulative_pred_ent_emoji.png
  teaser: cumulative_pred_ent_emoji.png
  credit:
  creditlink:
---
In this blog post I will demonstrate how to use [*hamiltorch*](https://github.com/AdamCobb/hamiltorch) for inference in Bayesian neural networks with larger data sets. Further details can be found in [*"Scaling Hamiltonian Monte Carlo Inference for Bayesian Neural Networks with Symmetric Splitting"*](https://arxiv.org/pdf/2010.06772.pdf).

## *hamiltorch* for Bayesian neural networks

Towards the end of my previous [post]({{ site.github.url }}{% link _posts/2019-07-15-hamiltorch.md %}) I gave a basic example of how to use `hamiltorch` for inference in
Bayesian neural networks (BNNs). Since then, my aim has been to make it as easy as possible to apply HMC over as many PyTorch models as possible. This includes being able to work with a `torch.utils.data.Dataloader` in the same way as for stochastic gradient descent. However, the main addition since the last post, has been to investigate split HMC and come up with a new way of scaling HMC to larger data sets with a single GPU. Interestingly, we will actually see that `hamiltorch` can compete with stochastic gradient approaches and also provide better uncertainty quantification.
{'fraction_non_alphanumeric': 0.07005689900426743, 'fraction_numerical': 0.008890469416785207, 'mean_word_length': 4.959745762711864, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7186377', 'n_tokens_mistral': 806, 'n_tokens_neox': 749, 'n_words': 369}