Chrome's V8 compiler, the Java HotSpot compiler, and many more have multiple tiers of interpretation and compilation. A function starts off as interpreted in HotSpot and then, if it is run often enough, it is compiled to native code (and later optimized). How does the interface between such layers work?For example, if you have 2 functions: Function A and Function B. If Function A is compiled and calls Function B, which is interpreted at the time Function A is compiled, Function A simply has a few instructions to trigger the interpreter for Function B, right?But, what happens if, at some point after Function A is compiled, Function B is compiled too? Then, Function A will end up calling an interpreted version of Function B despite Function B having a faster, compiled, version.So, what happens in modern compilers? Do they hot patch Function A to point to the new code for Function B? Do they add a layer of indirection and hot patch that instead of patching every single function that references Function B? Or something else entirely?
How do hybrid interpreter-JIT compilers work?
performance;compiler;interpreters;x86;jit
You can split any code into pieces consisting of a linear sequence of steps that are separated by control flow changes (branches, function calls, returns, etc).To do JIT compiling, you'd start immediately after a change in control flow, scan ahead until you find the next control flow change, convert the linear sequence of steps into native code, optimise the native code, place a terminating control flow change at the end of it, then store the result (lets call it a trace) in some sort of trace cache and execute the trace. If there's no trace in the cache for the next piece of code then that terminating control flow change can pass control flow back to the virtual machine. If there is a trace for the next piece of code then that terminating control flow change can pass control directly (or indirectly) to the next trace. Note: I am over-simplifying here - that terminating control flow change may have 2 or more destinations (e.g. it may be a branch, where one destination is the true path and the other is the false path), and may need assistance from the virtual machine even if the next trace is in the cache.The problem here is that it's all expensive (and the harder you try to optimise the native code being generated the more expensive it gets). If the sequence of code is run often then the overhead of JIT compiling becomes insignificant and there's a performance gain. If the sequence of code is not run often then the cost of JIT compiling can easily exceed the benefits.Now; if there's no trace for the next piece of code then that terminating control flow change can pass control flow back to the virtual machine. Instead of JIT compiling, nothing prevents the virtual machine from interpreting at this point. If you decide to do JIT compiling you can patch the previous trace's terminating control flow change so that it points to the new trace that you generate. If you decide to interpret instead; then that's fine - immediately after the interpreter interprets a change in control flow it checks the trace cache to see if there's an already native trace for the next piece. If there is it can execute the already compiled trace, and if there isn't it can just keep interpreting or decide to switch to JIT compiling.The only other thing you'd need to do is decide when to use JIT and when to interpret. Typically there are a few hints you can use to infer that JIT is likely to be worthwhile (specifically, taken conditional branches where the destination is at a lower address typically indicate loops, and loops are good candidates for JIT compiling). Beyond that you'd need to maintain some statistics (like the number of times something has been interpreted) and use them as the basis for the JIT or interpret decision.