<story><title>Rewriting a high performance vector database in Rust</title><url>https://www.pinecone.io/learn/rust-rewrite/</url></story><parent_chain></parent_chain><comment><author>makmanalp</author><text>&amp;gt; If you’re using a higher level language, you’re not going to have access to how the memory is laid out. A simple change, like removing indirection in our list, was an order of magnitude improvement in our latencies since there’s memory prefetching in the compiler and the CPU can anticipate which vectors are going to be loaded next in order to improve the memory footprint.&lt;p&gt;This is a common experience and I&amp;#x27;m still surprised by the choice I constantly see to use a managed-memory languages to build a database - one of a very small set of special cases where having full control over the memory layout might just be a reasonable thing to want. In this universe (absent doing something completely absurd) it&amp;#x27;s not algorithmic complexity but managing data locality in the cache hierarchy (e.g. reading things from L3 vs main memory vs disk) that makes things orders of magnitude faster, especially if you&amp;#x27;re in the realm of doing things like SIMD operations to speed things up.&lt;p&gt;Perhaps there&amp;#x27;s some level of suck we&amp;#x27;re willing to tolerate for all the other benefits you get, but I&amp;#x27;ve been noticing a pattern of &amp;quot;align things just so at the higher level and hope they mostly turn out the way you want at the lower level&amp;quot; (e.g. also with the Apache java-y databases like hadoop &amp;#x2F; hbase &amp;#x2F; cassandra which I guess were mostly supposed to derive their total throughput from massive scale rather than per-node performance) which is a bit funny.&lt;p&gt;But also it seems like part of Rust&amp;#x27;s promise was &amp;quot;low level but make it high level&amp;quot; which seems to be succeeding (zero-cost abstractions and whatnot), so I imagine this will get better over time - having not attempted a project like this myself, I&amp;#x27;m not sure what the limitations you&amp;#x27;d run up against are in terms of laying things out in memory in a favorable way - I imagine the kind of massive manually managed arena allocations and ad-hoc pointers going everywhere that one normally does doesn&amp;#x27;t really fly.</text></comment>
<story><title>Rewriting a high performance vector database in Rust</title><url>https://www.pinecone.io/learn/rust-rewrite/</url></story><parent_chain></parent_chain><comment><author>einpoklum</author><text>The authors of this post rewrote _their own_ DBMS in Rust. Which is perfectly ok, but I&amp;#x27;m not sure I would trust them to decide that theirs is a &amp;quot;high-performance&amp;quot; DBMS. They don&amp;#x27;t have any benchmark results except images of their own internal performance measures; they don&amp;#x27;t offer any way of comparing their performance with other DBMSes (e.g. Vectorwise&amp;#x2F;Actian Vector, ClickHouse, DuckDB etc. - not to mention Oracle, MS or SAP offerings); and they only have marketing blurb about their numbers: &amp;quot;Up to 10x performance&amp;quot; (with no baseline of course).&lt;p&gt;So, they took some DBMS (which is probably not so hot in terms of performance) and rewrote it in Rust. Surely possible, possibly useful, but not much to write home about if one is interested in DBMS performance.</text></comment>