<story><title>Cassandra at Apple: 1000s of Clusters, 300k Nodes, 100 PB</title><url>https://twitter.com/erickramirezau/status/1578063811495477248</url></story><parent_chain><item><author>tpmx</author><text>300k Cassandra nodes seems a bit over the top even for a company with as many active devices as Apple.&lt;p&gt;&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;www.theverge.com&amp;#x2F;2022&amp;#x2F;1&amp;#x2F;28&amp;#x2F;22906071&amp;#x2F;apple-1-8-billion-active-devices-stats&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;www.theverge.com&amp;#x2F;2022&amp;#x2F;1&amp;#x2F;28&amp;#x2F;22906071&amp;#x2F;apple-1-8-billio...&lt;/a&gt;&lt;p&gt;1.8B active devices &amp;#x2F; 300k nodes = (just) 6k devices per Cassandra node</text></item></parent_chain><comment><author>psaux</author><text>Worked there, managed a lot of the teams. An agent is considered a node. So you could have X nodes per server if that clears items up. Even agent per X jbod. The teams are very smart on blast radius and sharding. When I left I had exabytes of data under my mgmt and happy to chat if anyone wants to DM. Folks often forget how many users and services Apple has.</text></comment>
<story><title>Cassandra at Apple: 1000s of Clusters, 300k Nodes, 100 PB</title><url>https://twitter.com/erickramirezau/status/1578063811495477248</url></story><parent_chain><item><author>tpmx</author><text>300k Cassandra nodes seems a bit over the top even for a company with as many active devices as Apple.&lt;p&gt;&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;www.theverge.com&amp;#x2F;2022&amp;#x2F;1&amp;#x2F;28&amp;#x2F;22906071&amp;#x2F;apple-1-8-billion-active-devices-stats&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;www.theverge.com&amp;#x2F;2022&amp;#x2F;1&amp;#x2F;28&amp;#x2F;22906071&amp;#x2F;apple-1-8-billio...&lt;/a&gt;&lt;p&gt;1.8B active devices &amp;#x2F; 300k nodes = (just) 6k devices per Cassandra node</text></item></parent_chain><comment><author>WookieRushing</author><text>I was also surprised by this. 300K nodes for a distributed DB is kind of crazy. I’ve worked with similar systems but they stored much more than 100 PB with 10x less nodes&lt;p&gt;Apple is using less than one TB per server…&lt;p&gt;But when you see the 1000s of clusters it starts to make sense. They probably have a Cassandra cluster as their default storage for any use case and each one probably requires at least 3 nodes. They’re keeping the blast radius small of any issue while being super redundant. It probably grew organically instead of any central capacity management</text></comment>