<story><title>Scaling Databases at Activision [pdf]</title><url>https://static.sched.com/hosted_files/kccnceu2023/3d/KubeCon-2023-Scaling-Databases-Activision.pdf</url></story><parent_chain><item><author>Nathanba</author><text>I really wonder about the justification for using MySQL&amp;#x2F;Postgres sometimes. How is it not simply superior in almost every way to start with ScyllaDB which is faster in every way and the only downside I can see is that you have a very limited way to query things. But so what, at least it&amp;#x27;s fast. I mean isn&amp;#x27;t that the nr.1 thing you want from a database, it&amp;#x27;s not about comfort.. it&amp;#x27;s about storing data. It is genuinely annoying though for example that you can&amp;#x27;t even alter a column type after the column was created in ScyllaDB.</text></item><item><author>aeyes</author><text>I see 500k qps with 30TB data doubling every 2 years on slide 19.&lt;p&gt;In my experience me this isn&amp;#x27;t close to being possible with a single Postgres box. There is no way around sharding this workload.&lt;p&gt;Vitess has different consistency models, it&amp;#x27;s not as eventually consistent as you describe.</text></item><item><author>geenat</author><text>These numbers make more sense on hardware from 5 years ago (to be fair, what they were dealing with), but postgres, on modern consumer hardware (Yes, consumer! Ex: KC3000 + Ryzen 7950x), already does over 50k QPS without being overburdened. You could probably squeeze out 100k QPS with faster NVME&amp;#x27;s.&lt;p&gt;That&amp;#x27;s like 10 tall servers for their peak QPS? (500k QPS on slide 19)&lt;p&gt;Lots of natural sharding points here too: company, game, usage of data, etc.&lt;p&gt;This is assuming you avoid or go light on expensive features like foreign keys (as Vitess already does).&lt;p&gt;The &amp;quot;scale magic&amp;quot; in Spanner (Big Table) inspired DB&amp;#x27;s are just hidden automation of traditional sharding-&lt;p&gt;CockroachDB: Sharded indexes, then runs map reduce for you.&lt;p&gt;Scylla&amp;#x2F;Cassanda: Sharded indexes again, but more limitations for speedups: Eventually consistent. You don&amp;#x27;t have fast delete- only update (discord uses tombstones). JOIN&amp;#x27;s are &amp;quot;in app&amp;quot; only.&lt;p&gt;Vitess: Proxy that dismantles&amp;#x2F;routes your query to the correct server. This scales, but is eventually consistent. JOIN&amp;#x27;s on co-located data, or &amp;quot;in app&amp;quot;.... in simple terms, Vitess is like an externally managed &amp;quot;in app&amp;quot; query parser&amp;#x2F;router.</text></item></parent_chain><comment><author>tomnipotent</author><text>&amp;gt; the only downside I can see is that you have a very limited way to query things&lt;p&gt;Which is a huge downside.&lt;p&gt;&amp;gt; But so what, at least it&amp;#x27;s fast&lt;p&gt;So is an RDBMS for most read&amp;#x2F;write workloads. Few will see an overall benefit from Cassandra and the labor needed to replicate the conveniences offered by an RDBMS.&lt;p&gt;&amp;gt; it&amp;#x27;s about storing data&lt;p&gt;It&amp;#x27;s not about storing data, it&amp;#x27;s about using data to accomplish business objectives with the promises that entails. You opt for something like Cassandra&amp;#x2F;Scylla because it solves a technical challenge otherwise untenable or more expensive, but it&amp;#x27;s a poor choice if your problem is reasonably solved with what Postgres can offer.</text></comment>
<story><title>Scaling Databases at Activision [pdf]</title><url>https://static.sched.com/hosted_files/kccnceu2023/3d/KubeCon-2023-Scaling-Databases-Activision.pdf</url></story><parent_chain><item><author>Nathanba</author><text>I really wonder about the justification for using MySQL&amp;#x2F;Postgres sometimes. How is it not simply superior in almost every way to start with ScyllaDB which is faster in every way and the only downside I can see is that you have a very limited way to query things. But so what, at least it&amp;#x27;s fast. I mean isn&amp;#x27;t that the nr.1 thing you want from a database, it&amp;#x27;s not about comfort.. it&amp;#x27;s about storing data. It is genuinely annoying though for example that you can&amp;#x27;t even alter a column type after the column was created in ScyllaDB.</text></item><item><author>aeyes</author><text>I see 500k qps with 30TB data doubling every 2 years on slide 19.&lt;p&gt;In my experience me this isn&amp;#x27;t close to being possible with a single Postgres box. There is no way around sharding this workload.&lt;p&gt;Vitess has different consistency models, it&amp;#x27;s not as eventually consistent as you describe.</text></item><item><author>geenat</author><text>These numbers make more sense on hardware from 5 years ago (to be fair, what they were dealing with), but postgres, on modern consumer hardware (Yes, consumer! Ex: KC3000 + Ryzen 7950x), already does over 50k QPS without being overburdened. You could probably squeeze out 100k QPS with faster NVME&amp;#x27;s.&lt;p&gt;That&amp;#x27;s like 10 tall servers for their peak QPS? (500k QPS on slide 19)&lt;p&gt;Lots of natural sharding points here too: company, game, usage of data, etc.&lt;p&gt;This is assuming you avoid or go light on expensive features like foreign keys (as Vitess already does).&lt;p&gt;The &amp;quot;scale magic&amp;quot; in Spanner (Big Table) inspired DB&amp;#x27;s are just hidden automation of traditional sharding-&lt;p&gt;CockroachDB: Sharded indexes, then runs map reduce for you.&lt;p&gt;Scylla&amp;#x2F;Cassanda: Sharded indexes again, but more limitations for speedups: Eventually consistent. You don&amp;#x27;t have fast delete- only update (discord uses tombstones). JOIN&amp;#x27;s are &amp;quot;in app&amp;quot; only.&lt;p&gt;Vitess: Proxy that dismantles&amp;#x2F;routes your query to the correct server. This scales, but is eventually consistent. JOIN&amp;#x27;s on co-located data, or &amp;quot;in app&amp;quot;.... in simple terms, Vitess is like an externally managed &amp;quot;in app&amp;quot; query parser&amp;#x2F;router.</text></item></parent_chain><comment><author>geenat</author><text>It&amp;#x27;s a great point- I&amp;#x27;ve been looking into this path myself: ScyllaDB + logic inside app.&lt;p&gt;The main issue I&amp;#x27;ve run into is deployment. It&amp;#x27;s just very rough compared to CockroachDB or Postgres, which are single binary or very close to it.&lt;p&gt;If ScyllaDB became as easy to set up as CockroachDB (or close-ish), they&amp;#x27;d have a lot more users, I think.&lt;p&gt;You can make Postgres setup this easy: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;gist.github.com&amp;#x2F;gnat&amp;#x2F;cfe3754c3dc817c7fb8b2225ef4db628&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;gist.github.com&amp;#x2F;gnat&amp;#x2F;cfe3754c3dc817c7fb8b2225ef4db62...&lt;/a&gt;&lt;p&gt;CockroachDB, also dead easy: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;www.cockroachlabs.com&amp;#x2F;docs&amp;#x2F;v22.2&amp;#x2F;start-a-local-cluster&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;www.cockroachlabs.com&amp;#x2F;docs&amp;#x2F;v22.2&amp;#x2F;start-a-local-clust...&lt;/a&gt;&lt;p&gt;ScyllaDB is an enormous mess of daemons and ports: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;university.scylladb.com&amp;#x2F;courses&amp;#x2F;scylla-operations&amp;#x2F;lessons&amp;#x2F;installation&amp;#x2F;topic&amp;#x2F;installation&amp;#x2F;&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;university.scylladb.com&amp;#x2F;courses&amp;#x2F;scylla-operations&amp;#x2F;le...&lt;/a&gt;</text></comment>