<story><title>Java JIT vs. Java AOT vs. Go for Small, Short-Lived Processes</title><url>http://macias.info/entry/201912201300_graal_aot.md</url></story><parent_chain><item><author>mdasen</author><text>GC is complicated because what works well for one thing might not work well for other things.  GC can be both about clever things and about hard trade-offs.  That said, I&amp;#x27;ll try to talk about GC without having a religious war erupt.  Keep in mind, something here might be wrong.&lt;p&gt;Go&amp;#x27;s GC is tuned for low latency.  People hate pause times.  They hate it even more than they hate slowness.  Go makes a trade-off for short pause times, but in doing so they do sacrifice throughput.  That&amp;#x27;s great for a web server.  With a web server, you care a lot about pauses offering a bad experience.  A 500ms pause is going to give a customer a bad experience.  That&amp;#x27;s less good for batch processing.  Let&amp;#x27;s say you&amp;#x27;re running a backfill job that you expect to take 10-15 hours.  You don&amp;#x27;t really care if it pauses for 1 minute to GC every 30 minutes.  No one will even know.  However, you will know whether it took 10 hours or 15 hours.&lt;p&gt;Go&amp;#x27;s GC is meant to be simple for users.  I think the only option is &amp;quot;how much memory should I be targeting to use?&amp;quot;  That makes it dead simple for users.  I think the Go authors are right that too many tuning knobs can be a bad thing.  People can spend months doing nonsense work.  Worst is that knobs are hard to verify that anything is really different.  If you&amp;#x27;re running a web server, was the traffic really the same?  What about other things happening on the machine?  Wouldn&amp;#x27;t you rather be writing software instead?&lt;p&gt;Go&amp;#x27;s GC is a non-copying GC.  One thing this means is that Go&amp;#x27;s (heap) memory allocation ends up being very different because Go&amp;#x27;s memory is going to become fragmented.  So Go needs to keep a map of free memory and allocations are a bit more expensive.  Java (with GCs like G1) can just bump allocate which is insanely cheap.  This is because Java is allocating everything contiguously so it just needs to move a pointer.  How does that work once something becomes freed?  Java&amp;#x27;s G1 (the default in the latest LTS Java) will copy everything that&amp;#x27;s still alive to a new portion of memory and the old portion is then just empty.  You kind of see this in Go&amp;#x27;s culture.  Web frameworks obsess about not making heap allocations.  Libraries often have you pass in pointers to be filled in rather than returning something.&lt;p&gt;Go misses out on the generational hypothesis.  The generational hypothesis is one of the more durable observations we have about programming - that most things that are allocated die really quickly.  C# and Java both use generational collectors by default and they&amp;#x27;ve done way better than what came before.  C# and Java don&amp;#x27;t have as-low pause times as Go, but part of that is that they&amp;#x27;re targeting other things like throughput or heap overhead more.&lt;p&gt;Go doesn&amp;#x27;t need GC as much.  Go can allocate more on the stack than Java can and, well, Go programmers are sometimes a bit obsessed with stack allocations even when it makes for more complicated code.  Having structs means creating something where you can just have contiguous memory rather than allocating separate things for the fields in your object.  Go&amp;#x27;s authors have observed that a lot of their objects that die-young are stack allocated and so while the generational hypothesis holds, it&amp;#x27;s a bit different.  Go has put a good amount of effort into escape analysis to get more stuff stack allocated.&lt;p&gt;Java has two new algorithms ZGC and Shenandoah which are available in the latest Java.  They&amp;#x27;re pretty impressive and usually get down to sub-millisecond pause times and even 99th percentile pauses of 1-2ms.&lt;p&gt;Go&amp;#x27;s new GC was constrained by the fact that &amp;quot;Go also desperately needed short term success in 2015&amp;quot; (Rick Hudson from Google&amp;#x27;s Go team) and the fact that they wanted their foreign function interface to be simple - if you don&amp;#x27;t move objects in memory, you don&amp;#x27;t have to worry about dealing with the indirection you&amp;#x27;d need between C&amp;#x2F;C++ expecting them to be in one place and Go moving them around.  Google&amp;#x27;s going to have a lot of code they want to use without replacing it with Go code and so C&amp;#x2F;C++ interop is going to be huge in terms of their internal goals (and in terms of what the team targeted regardless of whether it&amp;#x27;s useful to you).  And I think once they had shown off sub-millisecond pause times, they were really hesitant to do something that might introduce things like 5ms pause times.  I think they might have also said that Google had an obsession with the long-tail at the time.  Especially at Google, there&amp;#x27;s going to be a very long tail and if that&amp;#x27;s what people are all talking about and caring about, you end up wanting to target that.&lt;p&gt;Go has tried other algorithms.  They had a request-oriented-collector and that worked well, but it slowed down certain applications that Go programmers care about - namely, the compiler.  They tried a non-copying generational GC, but didn&amp;#x27;t have a lot of success there.&lt;p&gt;Ultimately, Go wanted fast success and to solve the #1 complaint people had: extreme pause times (pause times that would make JVM developers feel sorry for Go programmers).  Going with a copying GC might have offered better performance, but would have meant a lot more work.  And Go gets away with some things because more stuff gets stack allocated and Go programmers try to avoid heap allocations which would be more expensive given Go&amp;#x27;s choices (programming for the GC algorithm).&lt;p&gt;I don&amp;#x27;t think that JVM languages have a programming style that lends themselves to an over-reliance on churning through short-term garbage.  Well, Clojure probably since I think it does go for the functional&amp;#x2F;immutable allocate-a-lot style.  Maybe Kotlin and Scala if you&amp;#x27;re creating lots of immutable stuff just to re-allocate&amp;#x2F;copy when you want to change one field.  That doesn&amp;#x27;t really apply to most Java programs.  And I have covered the way that Go potentially leads to more stack allocations.  However, I don&amp;#x27;t think most people know how their Go programs work any more than their JVM programs and this really just seems to be a &amp;quot;I want to dislike Java&amp;quot; kind of thing rather than something about memory.&lt;p&gt;Java programs tend to start slow because of the JVM and JIT compilation.  Java has been focused on throughput more than Go has (at the expense of responsiveness).  That is changing with the two latest Java GC algorithms (and even G1 which is really good).  Java is also working on modularizing itself so that you won&amp;#x27;t bring along as much of what you don&amp;#x27;t need (Jigsaw) and AOT compilation.  Saying that Java is slow just isn&amp;#x27;t really true, but it might feel true - things like startup times and pause times can inform our opinions a lot.  There&amp;#x27;s absolutely no question that Java is a lot faster than Python, but Python can feel faster for simple programs that aren&amp;#x27;t doing a lot (or are just small amounts of Python doing most of the heavy work in C).&lt;p&gt;I mean, are you including Android in &amp;quot;all Java apps&amp;quot;?&lt;p&gt;Java, C#, and Go are all really wonderful languages&amp;#x2F;platforms (including things hosted on them like Kotlin).  They&amp;#x27;re all around the same performance, but they do have some differences.  I think Go should re-visit their GC decisions in the future, especially as ZGC and Shenandoah take shape, but their GC works pretty well.  But there are certainly trade-offs being made (and it isn&amp;#x27;t around language features that make the platform productive for programmers).  I think GC is very interesting, but ultimately Java, C#, and Go all have very good GC that offers a good experience.</text></item><item><author>smabie</author><text>The style of programming for the dominant JVM languages (Java, Scala, Kotlin) involves an over reliance on churning through a lot of short-term garbage. I think this partially due to how annoying the platform is to actually use. Incredibly complex and chalk full of programmer pitfalls (like type-erasure). In fact, the majority of devs have no clue at how the JVM works, treating it as just “magic.”&lt;p&gt;I would imagine that Go’s GC is worse than the tunable JVM ones, but go isn’t powerful enough that one would ever be tempted to program in an abstraction heavy style. While I would argue it takes a lot more work to program in Go ca, day Scala, I think the constraints imposed lead to better software. I for one have never seen a JVM desktop or business app that worked super well. The JVM manages to be a highly optimized efficient platform in which almost exclusively slow, laggy, and memory hungry apps are produced. With that being said, it’s a highly productive platform (if you’re using something besides Java), especially for backend business apps.</text></item><item><author>shantly</author><text>It’s been the case ever since I can remember that “Java’s basically as efficient as native, it’s super fast” but all the actual java software I encounter is a slowish, bloated memory hog. I don’t know why, but that’s how it is.</text></item><item><author>robfig</author><text>It&amp;#x27;s hard to square these articles with the reality I see on the ground: our baseline memory usage for common types of Java service is 1 GB, vs 50 MB for Go. We do have a few mammoth servers at the top end in both languages though (e.g. 75 GB heaps)&lt;p&gt;The deploy JARs have 100+ MB of class files, so perhaps it&amp;#x27;s a function of all the dependencies that you &amp;quot;need&amp;quot; for an enterprise Java program and not something more fundamental.&lt;p&gt;These blog posts also present AOT as if its just another option you can toggle, while my impression is that it&amp;#x27;s incompatible with common Java libraries and requires swapping &amp;#x2F; maintaining a parallel compiler in the build toolchain, configuring separate rules to build it AOT, etc. I don&amp;#x27;t have actual experience with it though so I could be missing something.</text></item></parent_chain><comment><author>loopz</author><text>Nice writeup!&lt;p&gt;If a programmer really cares for performant batch-processing (not sure it&amp;#x27;s a direction we&amp;#x27;re going for anymore but), you&amp;#x27;ll just reuse your objects and dataspace. Understanding Go in order to minimize GC is pretty trivial and the standard library includes packages and tooling exactly for such purposes. Most likely, there are only a few hotspots needing to be tended to like this.&lt;p&gt;So this sounds like Golang is optimized for what people love about computing: fast response time. Also providing programmers with basic performance &amp;quot;out of the box&amp;quot;, which is another good tradeoff for me.&lt;p&gt;The tradeoff works best when you make simple designs, not huge behemoths. Spending more time reiterating clever designs, rather than jamming the keyboard until done.</text></comment>
<story><title>Java JIT vs. Java AOT vs. Go for Small, Short-Lived Processes</title><url>http://macias.info/entry/201912201300_graal_aot.md</url></story><parent_chain><item><author>mdasen</author><text>GC is complicated because what works well for one thing might not work well for other things.  GC can be both about clever things and about hard trade-offs.  That said, I&amp;#x27;ll try to talk about GC without having a religious war erupt.  Keep in mind, something here might be wrong.&lt;p&gt;Go&amp;#x27;s GC is tuned for low latency.  People hate pause times.  They hate it even more than they hate slowness.  Go makes a trade-off for short pause times, but in doing so they do sacrifice throughput.  That&amp;#x27;s great for a web server.  With a web server, you care a lot about pauses offering a bad experience.  A 500ms pause is going to give a customer a bad experience.  That&amp;#x27;s less good for batch processing.  Let&amp;#x27;s say you&amp;#x27;re running a backfill job that you expect to take 10-15 hours.  You don&amp;#x27;t really care if it pauses for 1 minute to GC every 30 minutes.  No one will even know.  However, you will know whether it took 10 hours or 15 hours.&lt;p&gt;Go&amp;#x27;s GC is meant to be simple for users.  I think the only option is &amp;quot;how much memory should I be targeting to use?&amp;quot;  That makes it dead simple for users.  I think the Go authors are right that too many tuning knobs can be a bad thing.  People can spend months doing nonsense work.  Worst is that knobs are hard to verify that anything is really different.  If you&amp;#x27;re running a web server, was the traffic really the same?  What about other things happening on the machine?  Wouldn&amp;#x27;t you rather be writing software instead?&lt;p&gt;Go&amp;#x27;s GC is a non-copying GC.  One thing this means is that Go&amp;#x27;s (heap) memory allocation ends up being very different because Go&amp;#x27;s memory is going to become fragmented.  So Go needs to keep a map of free memory and allocations are a bit more expensive.  Java (with GCs like G1) can just bump allocate which is insanely cheap.  This is because Java is allocating everything contiguously so it just needs to move a pointer.  How does that work once something becomes freed?  Java&amp;#x27;s G1 (the default in the latest LTS Java) will copy everything that&amp;#x27;s still alive to a new portion of memory and the old portion is then just empty.  You kind of see this in Go&amp;#x27;s culture.  Web frameworks obsess about not making heap allocations.  Libraries often have you pass in pointers to be filled in rather than returning something.&lt;p&gt;Go misses out on the generational hypothesis.  The generational hypothesis is one of the more durable observations we have about programming - that most things that are allocated die really quickly.  C# and Java both use generational collectors by default and they&amp;#x27;ve done way better than what came before.  C# and Java don&amp;#x27;t have as-low pause times as Go, but part of that is that they&amp;#x27;re targeting other things like throughput or heap overhead more.&lt;p&gt;Go doesn&amp;#x27;t need GC as much.  Go can allocate more on the stack than Java can and, well, Go programmers are sometimes a bit obsessed with stack allocations even when it makes for more complicated code.  Having structs means creating something where you can just have contiguous memory rather than allocating separate things for the fields in your object.  Go&amp;#x27;s authors have observed that a lot of their objects that die-young are stack allocated and so while the generational hypothesis holds, it&amp;#x27;s a bit different.  Go has put a good amount of effort into escape analysis to get more stuff stack allocated.&lt;p&gt;Java has two new algorithms ZGC and Shenandoah which are available in the latest Java.  They&amp;#x27;re pretty impressive and usually get down to sub-millisecond pause times and even 99th percentile pauses of 1-2ms.&lt;p&gt;Go&amp;#x27;s new GC was constrained by the fact that &amp;quot;Go also desperately needed short term success in 2015&amp;quot; (Rick Hudson from Google&amp;#x27;s Go team) and the fact that they wanted their foreign function interface to be simple - if you don&amp;#x27;t move objects in memory, you don&amp;#x27;t have to worry about dealing with the indirection you&amp;#x27;d need between C&amp;#x2F;C++ expecting them to be in one place and Go moving them around.  Google&amp;#x27;s going to have a lot of code they want to use without replacing it with Go code and so C&amp;#x2F;C++ interop is going to be huge in terms of their internal goals (and in terms of what the team targeted regardless of whether it&amp;#x27;s useful to you).  And I think once they had shown off sub-millisecond pause times, they were really hesitant to do something that might introduce things like 5ms pause times.  I think they might have also said that Google had an obsession with the long-tail at the time.  Especially at Google, there&amp;#x27;s going to be a very long tail and if that&amp;#x27;s what people are all talking about and caring about, you end up wanting to target that.&lt;p&gt;Go has tried other algorithms.  They had a request-oriented-collector and that worked well, but it slowed down certain applications that Go programmers care about - namely, the compiler.  They tried a non-copying generational GC, but didn&amp;#x27;t have a lot of success there.&lt;p&gt;Ultimately, Go wanted fast success and to solve the #1 complaint people had: extreme pause times (pause times that would make JVM developers feel sorry for Go programmers).  Going with a copying GC might have offered better performance, but would have meant a lot more work.  And Go gets away with some things because more stuff gets stack allocated and Go programmers try to avoid heap allocations which would be more expensive given Go&amp;#x27;s choices (programming for the GC algorithm).&lt;p&gt;I don&amp;#x27;t think that JVM languages have a programming style that lends themselves to an over-reliance on churning through short-term garbage.  Well, Clojure probably since I think it does go for the functional&amp;#x2F;immutable allocate-a-lot style.  Maybe Kotlin and Scala if you&amp;#x27;re creating lots of immutable stuff just to re-allocate&amp;#x2F;copy when you want to change one field.  That doesn&amp;#x27;t really apply to most Java programs.  And I have covered the way that Go potentially leads to more stack allocations.  However, I don&amp;#x27;t think most people know how their Go programs work any more than their JVM programs and this really just seems to be a &amp;quot;I want to dislike Java&amp;quot; kind of thing rather than something about memory.&lt;p&gt;Java programs tend to start slow because of the JVM and JIT compilation.  Java has been focused on throughput more than Go has (at the expense of responsiveness).  That is changing with the two latest Java GC algorithms (and even G1 which is really good).  Java is also working on modularizing itself so that you won&amp;#x27;t bring along as much of what you don&amp;#x27;t need (Jigsaw) and AOT compilation.  Saying that Java is slow just isn&amp;#x27;t really true, but it might feel true - things like startup times and pause times can inform our opinions a lot.  There&amp;#x27;s absolutely no question that Java is a lot faster than Python, but Python can feel faster for simple programs that aren&amp;#x27;t doing a lot (or are just small amounts of Python doing most of the heavy work in C).&lt;p&gt;I mean, are you including Android in &amp;quot;all Java apps&amp;quot;?&lt;p&gt;Java, C#, and Go are all really wonderful languages&amp;#x2F;platforms (including things hosted on them like Kotlin).  They&amp;#x27;re all around the same performance, but they do have some differences.  I think Go should re-visit their GC decisions in the future, especially as ZGC and Shenandoah take shape, but their GC works pretty well.  But there are certainly trade-offs being made (and it isn&amp;#x27;t around language features that make the platform productive for programmers).  I think GC is very interesting, but ultimately Java, C#, and Go all have very good GC that offers a good experience.</text></item><item><author>smabie</author><text>The style of programming for the dominant JVM languages (Java, Scala, Kotlin) involves an over reliance on churning through a lot of short-term garbage. I think this partially due to how annoying the platform is to actually use. Incredibly complex and chalk full of programmer pitfalls (like type-erasure). In fact, the majority of devs have no clue at how the JVM works, treating it as just “magic.”&lt;p&gt;I would imagine that Go’s GC is worse than the tunable JVM ones, but go isn’t powerful enough that one would ever be tempted to program in an abstraction heavy style. While I would argue it takes a lot more work to program in Go ca, day Scala, I think the constraints imposed lead to better software. I for one have never seen a JVM desktop or business app that worked super well. The JVM manages to be a highly optimized efficient platform in which almost exclusively slow, laggy, and memory hungry apps are produced. With that being said, it’s a highly productive platform (if you’re using something besides Java), especially for backend business apps.</text></item><item><author>shantly</author><text>It’s been the case ever since I can remember that “Java’s basically as efficient as native, it’s super fast” but all the actual java software I encounter is a slowish, bloated memory hog. I don’t know why, but that’s how it is.</text></item><item><author>robfig</author><text>It&amp;#x27;s hard to square these articles with the reality I see on the ground: our baseline memory usage for common types of Java service is 1 GB, vs 50 MB for Go. We do have a few mammoth servers at the top end in both languages though (e.g. 75 GB heaps)&lt;p&gt;The deploy JARs have 100+ MB of class files, so perhaps it&amp;#x27;s a function of all the dependencies that you &amp;quot;need&amp;quot; for an enterprise Java program and not something more fundamental.&lt;p&gt;These blog posts also present AOT as if its just another option you can toggle, while my impression is that it&amp;#x27;s incompatible with common Java libraries and requires swapping &amp;#x2F; maintaining a parallel compiler in the build toolchain, configuring separate rules to build it AOT, etc. I don&amp;#x27;t have actual experience with it though so I could be missing something.</text></item></parent_chain><comment><author>ptx</author><text>Java is slower than Python for simple short-running programs. When Python is finished Java is still struggling through start-up and class-loading.</text></comment>