<story><title>Queues don&apos;t fix overload (2014)</title><url>https://ferd.ca/queues-don-t-fix-overload.html</url></story><parent_chain><item><author>kasey_junk</author><text>This is a weird article because it points out that queues don’t solve overload but neither do load shedding or back pressure.&lt;p&gt;All 3 techniques are just different trade offs on what to do in the face of overload. All 3 have negative ramifications for the users of the system. Load shedding reduces availability, back pressure increases complexity and queues increase latency.&lt;p&gt;In “critical” systems you need all 3. And all 3 can be overloaded. Frankly, your load shedding or back pressure system is probably implemented on a queue one layer down the abstraction.</text></item></parent_chain><comment><author>dijit</author><text>I think it&amp;#x27;s the threshold of all people to truly understand that there&amp;#x27;s no solution to certain problems, only tradeoffs.&lt;p&gt;Queues are great, but can lead to catastrophic failure if you don&amp;#x27;t have a good way of handling the queue, so making an active choice about how you handle overload is part of designing a robust and resilient system.&lt;p&gt;Trading off new requests for current requests is, in my experience, a valid strategy for eCommerce for example. We called it &amp;quot;quenching&amp;quot;.</text></comment>
<story><title>Queues don&apos;t fix overload (2014)</title><url>https://ferd.ca/queues-don-t-fix-overload.html</url></story><parent_chain><item><author>kasey_junk</author><text>This is a weird article because it points out that queues don’t solve overload but neither do load shedding or back pressure.&lt;p&gt;All 3 techniques are just different trade offs on what to do in the face of overload. All 3 have negative ramifications for the users of the system. Load shedding reduces availability, back pressure increases complexity and queues increase latency.&lt;p&gt;In “critical” systems you need all 3. And all 3 can be overloaded. Frankly, your load shedding or back pressure system is probably implemented on a queue one layer down the abstraction.</text></item></parent_chain><comment><author>0xbadcafebee</author><text>The only real solution to overload (that is, the eventuality of the system not having enough capacity), in modern systems, is autoscaling. Nobody seems to talk about this, I guess because it&amp;#x27;s taken for granted? But you can literally just keep adding capacity now. We didn&amp;#x27;t really have that before the cloud; you had the servers you bought, and maybe you&amp;#x27;d rush to repurpose some servers to add capacity. Now an algorithm just adds servers magically when your load gets high.&lt;p&gt;Of course the system still has limits. But if you go from zonal to regional to global autoscaling, and you architect specifically to scale up each point in the system using autoscaling, it kind of doesn&amp;#x27;t have a limit? (that any one product would hit, at a global level)&lt;p&gt;In the past I have spun up a duplicate distributed system in another region and just split the traffic between the two. IaC is crazy.</text></comment>