<story><title>Why you might want a domain-specific database like TigerBeetleDB</title><url>https://twitter.com/phil_eaton/status/1568247444684554241</url></story><parent_chain><item><author>jandrewrogers</author><text>I have always been an unabashed fan of the idea of domain-specific databases. Database implementations make a &lt;i&gt;lot&lt;/i&gt; of compromises for the sake of generality and avoid useful features that overfit a single domain. The idea of a database engine perfectly optimized and feature-fit for a domain has obvious qualitative advantages, and it isn&amp;#x27;t controversial that it is eminently possible to build such a database.&lt;p&gt;So why are they so rare? It is expensive in several dimensions to build a narrowly tailored database engine from scratch. Database engines are not modular, you can&amp;#x27;t assemble them from arbitrary parts while maintaining control of their basic characteristics. Building an optimized domain-specific database from scratch is not trivial because you will have to do a lot of the really hard parts yourself.&lt;p&gt;In my head, there has always been a missing piece of software - a &amp;quot;database compiler&amp;quot;, that can take the very complex and high-dimensionality abstract specification for a domain and codegen a purpose-built database engine. I also recognize that this is exceedingly non-trivial; I codegen storage engines to spec and that is already difficult enough. Doing that for a full database engine in a specific domain would have hundreds of input parameters, many of which would require expertise in the underlying codegen to know how to use them. It would be awesome if such a thing existed though.</text></item></parent_chain><comment><author>jorangreef</author><text>&amp;quot;In my head, there has always been a missing piece of software - a &amp;quot;database compiler&amp;quot;, that can take the very complex and high-dimensionality abstract specification for a domain and codegen a purpose-built database engine.&amp;quot;&lt;p&gt;You nailed it!&lt;p&gt;And this is in fact our design for TigerBeetle, an Iron Man suit that you can put any state machine business logic into. You get global consensus protocol and local storage engine, with all the performance of TigerBeetle, and it&amp;#x27;s a really nice experience writing your own business logic inside of that. You can even test everything using Deterministic Simulation Testing.&lt;p&gt;Long term, we want to extract this into a library. Reading the TB source, you&amp;#x27;ll see all these abstractions are something we&amp;#x27;re thinking about.</text></comment>
<story><title>Why you might want a domain-specific database like TigerBeetleDB</title><url>https://twitter.com/phil_eaton/status/1568247444684554241</url></story><parent_chain><item><author>jandrewrogers</author><text>I have always been an unabashed fan of the idea of domain-specific databases. Database implementations make a &lt;i&gt;lot&lt;/i&gt; of compromises for the sake of generality and avoid useful features that overfit a single domain. The idea of a database engine perfectly optimized and feature-fit for a domain has obvious qualitative advantages, and it isn&amp;#x27;t controversial that it is eminently possible to build such a database.&lt;p&gt;So why are they so rare? It is expensive in several dimensions to build a narrowly tailored database engine from scratch. Database engines are not modular, you can&amp;#x27;t assemble them from arbitrary parts while maintaining control of their basic characteristics. Building an optimized domain-specific database from scratch is not trivial because you will have to do a lot of the really hard parts yourself.&lt;p&gt;In my head, there has always been a missing piece of software - a &amp;quot;database compiler&amp;quot;, that can take the very complex and high-dimensionality abstract specification for a domain and codegen a purpose-built database engine. I also recognize that this is exceedingly non-trivial; I codegen storage engines to spec and that is already difficult enough. Doing that for a full database engine in a specific domain would have hundreds of input parameters, many of which would require expertise in the underlying codegen to know how to use them. It would be awesome if such a thing existed though.</text></item></parent_chain><comment><author>haney</author><text>It’s kind of coming from the opposite direction, but it feels like Postgres extensions solve many of the needs you’re describing. They allow a developer to use the underlying database engine while extending it to include domain specific primitives.</text></comment>