<story><title>Modules, not microservices</title><url>http://blogs.newardassociates.com/blog/2023/you-want-modules-not-microservices.html</url></story><parent_chain><item><author>cogman10</author><text>You know, I don&amp;#x27;t really think microservices are fundamentally more scalable.  Rather, they expose scaling issues more readily.&lt;p&gt;When you have a giant monolith with the &amp;quot;load the world&amp;quot; endpoint, it can be tricky to pinpoint the the &amp;quot;load the world&amp;quot; endpoint (or, as is often the case, endpoint*s*) is what&amp;#x27;s causing issues.  Instead, everyone just tends to think of it as &amp;quot;the x app having problems.&amp;quot;&lt;p&gt;When you bust the monolith into the x&amp;#x2F;y&amp;#x2F;z, and x and z got the &amp;quot;load the world&amp;quot; endpoints, that starts the fires of &amp;quot;x is constantly killing things and it&amp;#x27;s only doing this one thing.  How do we do that better?&amp;quot;&lt;p&gt;That allows you to better prioritize fixing those scaling problems.</text></item><item><author>dahfizz</author><text>Microservices are less &lt;i&gt;efficient&lt;/i&gt;, but are still more &lt;i&gt;scalable&lt;/i&gt;.&lt;p&gt;Servers can only get so big. If your monolith needs more resources than a single server can provide, then you can chop it up into microservices and each microservice can get its own beefy server. Then you can put a load balancer in front of a microservice and run it on N beefy servers.&lt;p&gt;But this only matters at Facebook scale. I think most devs would be shocked at how much a single beefy server running efficient code can do.</text></item><item><author>eklitzke</author><text>I just want to point out that for the second problem (scalability of CPU&amp;#x2F;memory&amp;#x2F;io), microservices almost always make things worse. Making an RPC necessarily implies serialization and deserialization of data, and nearly always also means sending data over a socket. Plus the fact that most services have some constant overhead of the footprint to run the RPC code and other things (healthchecking, stats collection, etc.) that is typically bundled into each service. And even if the extra CPU&amp;#x2F;memory isn&amp;#x27;t a big deal for you, doing RPCs is going to add latency, and if you get too many microservices the latency numbers can start really adding up and be very difficult to fix later.&lt;p&gt;Running code in a single process is MUCH lower overhead because you don&amp;#x27;t need to transit a network layer and you&amp;#x27;re generally just passing pointers to data around rather than serializing&amp;#x2F;deserializing it.&lt;p&gt;There are definitely some cases where using microservices does make things more CPU&amp;#x2F;memory efficient, but it&amp;#x27;s much rarer than people think. An example where you&amp;#x27;d actually get efficiency would be something like a geofence service (imagine Uber, Doordash, etc.) where the geofence definitions are probably large and have to be stored in memory. Depending on how often geofence queries happen, it might be more efficient to have a small number of geofence service instances with the geofence definitions loaded in memory rather than having this logic as a module that many workers need to have loaded. But again, cases like this are much less common than the cases where services just lead to massive bloat.&lt;p&gt;I was working at Uber when they started transitioning from monolith to microservices, and pretty much universally splitting logic into microservices required provisioning many more servers and were disastrous for end-to-end latency times.</text></item><item><author>barrkel</author><text>Microservices, while often sold as solving a technical problem, usually actually solve for a human problem in scaling up an organization.&lt;p&gt;There&amp;#x27;s two technical problems that microservices purport to solve: modularization (separation of concerns, hiding implementation, document interface and all that good stuff) and scalability (being able to increase the amount of compute, memory and IO to the specific modules that need it).&lt;p&gt;The first problem, modules, can be solved at the language level. Modules can do that job, and that&amp;#x27;s the point of this blog post.&lt;p&gt;The second problem, scalability, is harder to solve at the language level in most languages outside those designed to be run in a distributed environment. But most people need it a lot less than they think. Normally the database is your bottleneck and if you keep your application server stateless, you can just run lots of them; the database can eventually be a bottleneck, but you can scale up databases &lt;i&gt;a lot&lt;/i&gt;.&lt;p&gt;The real reason that microservices may make sense is because they keep people honest around module boundaries. They make it much harder to retain access to persistent in-memory state, harder to navigate object graphs to take dependencies on things they shouldn&amp;#x27;t, harder to create PRs with complex changes on either side of a module boundary without a conversation about designing for change and future proofing. Code ownership by teams is something you need as an organization scales, if only to reduce the amount of context switching that developers need to do if treated as fully fungible; owning a service is more defensible than owning a module, since the team will own release schedules and quality gating.&lt;p&gt;I&amp;#x27;m not so positive on every microservice maintaining its own copy of state, potentially with its own separate data store. I think that usually adds more ongoing complexity in synchronization than it saves by isolating schemas. A better rule is for one service to own writes for a table, and other services can only read that table, and maybe even then not all columns or all non-owned tables. Problems with state synchronization are one of the most common failure modes in distributed applications, where queues get backed up, retries of &amp;quot;bad&amp;quot; events cause blockages and so on.</text></item></parent_chain><comment><author>mirekrusin</author><text>It sounds like creating problem, then spending time=money on fixing it and calling it a win?&lt;p&gt;There is a point when it all starts to make sense. But that point is when you go into billions worth business, hundreds of devs etc. And going there has large cost, especially for small&amp;#x2F;medium systems. And that cost is not one off - it&amp;#x27;s a day-to-day cost of introducing changes. It&amp;#x27;s orders of magnitude chaper and faster (head cound wise) to do changes in ie. single versioned monorepo where everything is deployed at once, as single working, tested, migrated version than doing progressive releases for each piece keeping it all backward compatible at micro level. Again - it does make sense at scale (hundreds of devs kind of scale), but saying your 5 devs team moves faster because they can work on 120 micoservices independently is complete nonsense.&lt;p&gt;In other words micoservices make sense when you don&amp;#x27;t really have other options, you have to do it, it&amp;#x27;s not good start-with default at all; and frankly Sam Newman says it in &amp;quot;Building Microservices&amp;quot; and so do people who know what they&amp;#x27;re talking about. For some reason juniors want to start there and look at anything non-microservice as legacy.</text></comment>
<story><title>Modules, not microservices</title><url>http://blogs.newardassociates.com/blog/2023/you-want-modules-not-microservices.html</url></story><parent_chain><item><author>cogman10</author><text>You know, I don&amp;#x27;t really think microservices are fundamentally more scalable.  Rather, they expose scaling issues more readily.&lt;p&gt;When you have a giant monolith with the &amp;quot;load the world&amp;quot; endpoint, it can be tricky to pinpoint the the &amp;quot;load the world&amp;quot; endpoint (or, as is often the case, endpoint*s*) is what&amp;#x27;s causing issues.  Instead, everyone just tends to think of it as &amp;quot;the x app having problems.&amp;quot;&lt;p&gt;When you bust the monolith into the x&amp;#x2F;y&amp;#x2F;z, and x and z got the &amp;quot;load the world&amp;quot; endpoints, that starts the fires of &amp;quot;x is constantly killing things and it&amp;#x27;s only doing this one thing.  How do we do that better?&amp;quot;&lt;p&gt;That allows you to better prioritize fixing those scaling problems.</text></item><item><author>dahfizz</author><text>Microservices are less &lt;i&gt;efficient&lt;/i&gt;, but are still more &lt;i&gt;scalable&lt;/i&gt;.&lt;p&gt;Servers can only get so big. If your monolith needs more resources than a single server can provide, then you can chop it up into microservices and each microservice can get its own beefy server. Then you can put a load balancer in front of a microservice and run it on N beefy servers.&lt;p&gt;But this only matters at Facebook scale. I think most devs would be shocked at how much a single beefy server running efficient code can do.</text></item><item><author>eklitzke</author><text>I just want to point out that for the second problem (scalability of CPU&amp;#x2F;memory&amp;#x2F;io), microservices almost always make things worse. Making an RPC necessarily implies serialization and deserialization of data, and nearly always also means sending data over a socket. Plus the fact that most services have some constant overhead of the footprint to run the RPC code and other things (healthchecking, stats collection, etc.) that is typically bundled into each service. And even if the extra CPU&amp;#x2F;memory isn&amp;#x27;t a big deal for you, doing RPCs is going to add latency, and if you get too many microservices the latency numbers can start really adding up and be very difficult to fix later.&lt;p&gt;Running code in a single process is MUCH lower overhead because you don&amp;#x27;t need to transit a network layer and you&amp;#x27;re generally just passing pointers to data around rather than serializing&amp;#x2F;deserializing it.&lt;p&gt;There are definitely some cases where using microservices does make things more CPU&amp;#x2F;memory efficient, but it&amp;#x27;s much rarer than people think. An example where you&amp;#x27;d actually get efficiency would be something like a geofence service (imagine Uber, Doordash, etc.) where the geofence definitions are probably large and have to be stored in memory. Depending on how often geofence queries happen, it might be more efficient to have a small number of geofence service instances with the geofence definitions loaded in memory rather than having this logic as a module that many workers need to have loaded. But again, cases like this are much less common than the cases where services just lead to massive bloat.&lt;p&gt;I was working at Uber when they started transitioning from monolith to microservices, and pretty much universally splitting logic into microservices required provisioning many more servers and were disastrous for end-to-end latency times.</text></item><item><author>barrkel</author><text>Microservices, while often sold as solving a technical problem, usually actually solve for a human problem in scaling up an organization.&lt;p&gt;There&amp;#x27;s two technical problems that microservices purport to solve: modularization (separation of concerns, hiding implementation, document interface and all that good stuff) and scalability (being able to increase the amount of compute, memory and IO to the specific modules that need it).&lt;p&gt;The first problem, modules, can be solved at the language level. Modules can do that job, and that&amp;#x27;s the point of this blog post.&lt;p&gt;The second problem, scalability, is harder to solve at the language level in most languages outside those designed to be run in a distributed environment. But most people need it a lot less than they think. Normally the database is your bottleneck and if you keep your application server stateless, you can just run lots of them; the database can eventually be a bottleneck, but you can scale up databases &lt;i&gt;a lot&lt;/i&gt;.&lt;p&gt;The real reason that microservices may make sense is because they keep people honest around module boundaries. They make it much harder to retain access to persistent in-memory state, harder to navigate object graphs to take dependencies on things they shouldn&amp;#x27;t, harder to create PRs with complex changes on either side of a module boundary without a conversation about designing for change and future proofing. Code ownership by teams is something you need as an organization scales, if only to reduce the amount of context switching that developers need to do if treated as fully fungible; owning a service is more defensible than owning a module, since the team will own release schedules and quality gating.&lt;p&gt;I&amp;#x27;m not so positive on every microservice maintaining its own copy of state, potentially with its own separate data store. I think that usually adds more ongoing complexity in synchronization than it saves by isolating schemas. A better rule is for one service to own writes for a table, and other services can only read that table, and maybe even then not all columns or all non-owned tables. Problems with state synchronization are one of the most common failure modes in distributed applications, where queues get backed up, retries of &amp;quot;bad&amp;quot; events cause blockages and so on.</text></item></parent_chain><comment><author>jayd16</author><text>Another way to look at this is microservices reduce the blast radius of problems.</text></comment>