<commit_before>---
layout: post
title: Cap'n Proto, FlatBuffers, and SBE
author: kentonv
---

Yesterday, some engineers at Google released [FlatBuffers](http://google-opensource.blogspot.com/2014/06/flatbuffers-memory-efficient.html), a new serialization protocol and library with similar design principles to Cap'n Proto. Also, a few months back, Real Logic released [Simple Binary Encoding](http://mechanical-sympathy.blogspot.com/2014/05/simple-binary-encoding.html), another protocol and library of this nature.

It seems we now have some friendly rivalry. :)

It's great to see that the concept of `mmap()`-able, zero-copy serialization formats are catching on, and it's wonderful that all are open source under liberal licenses. But as a user, you might be left wondering how all these systems compare. You have a vague idea that all these encodings are "fast", particularly compared to Protobufs or other more-traditional formats. But there is more to a serialization protocol than speed, and you may be wondering what else you should be considering.

The goal of this blog post is to highlight some of the main _qualitative_ differences between these libraries as I see them. Obviously, I am biased, and you should consider that as you read. Hopefully, though, this provides a good starting point for your own investigation of the alternatives.

### Feature Matrix

The following are a set of considerations I think are important. See something I missed? Please [let me know](mailto:<EMAIL>) and I'll add it. I'd like in particular to invite the SBE and FlatBuffers authors to suggest advantages of their libraries that I may have missed.
<commit_msg>Add update link.<commit_after>---
layout: post
title: Cap'n Proto, FlatBuffers, and SBE
author: kentonv
---

**Update:** I have made [some corrections](https://github.com/kentonv/capnproto/commit/e4e6c9076ae16804c07968cd3bdf6107155df7ee) since the original version of this post.

Yesterday, some engineers at Google released [FlatBuffers](http://google-opensource.blogspot.com/2014/06/flatbuffers-memory-efficient.html), a new serialization protocol and library with similar design principles to Cap'n Proto. Also, a few months back, Real Logic released [Simple Binary Encoding](http://mechanical-sympathy.blogspot.com/2014/05/simple-binary-encoding.html), another protocol and library of this nature.

It seems we now have some friendly rivalry. :)

It's great to see that the concept of `mmap()`-able, zero-copy serialization formats are catching on, and it's wonderful that all are open source under liberal licenses. But as a user, you might be left wondering how all these systems compare. You have a vague idea that all these encodings are "fast", particularly compared to Protobufs or other more-traditional formats. But there is more to a serialization protocol than speed, and you may be wondering what else you should be considering.

The goal of this blog post is to highlight some of the main _qualitative_ differences between these libraries as I see them. Obviously, I am biased, and you should consider that as you read. Hopefully, though, this provides a good starting point for your own investigation of the alternatives.

### Feature Matrix

The following are a set of considerations I think are important. See something I missed? Please [let me know](mailto:<EMAIL>) and I'll add it. I'd like in particular to invite the SBE and FlatBuffers authors to suggest advantages of their libraries that I may have missed.

{'fraction_non_alphanumeric': 0.06328751431844215, 'fraction_numerical': 0.014032073310423826, 'mean_word_length': 4.84113712374582, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2006995', 'n_tokens_mistral': 926, 'n_tokens_neox': 854, 'n_words': 481}