JIT optimizations

the picture

you know how people say "the JIT will optimize that away"? sometimes it does, sometimes it really doesn't, and the difference is almost always one thing: whether the code got inlined.

think of a chef who writes the recipe for a dish in terms of other recipes: "make the sauce (see page 40)". following it literally means flipping back and forth. an experienced chef first copies page 40 into the recipe, and only then notices that half the sauce steps are wasted for this dish. inlining is the copying. every other optimization is the noticing, and it can only notice what's on the page.

inlining, then escape analysis

here's a method that looks allocation-heavy: three Point records per call. step through what C2 does to it.

the inlining decisions are real, from -XX:+PrintInlining (a diagnostic flag, so it needs -XX:+UnlockDiagnosticVMOptions) when C2 compiled distanceSquared on jdk 25:

@ 6   Inline$Point::<init> (15 bytes)   inline (hot)
  @ 1   java.lang.Record::<init> (5 bytes)   inline (hot)
    @ 1   java.lang.Object::<init> (1 bytes)   inline (hot)
@ 17  Inline$Point::<init> (15 bytes)   inline (hot)
  ...
@ 20  Inline$Point::plus (26 bytes)   inline (hot)
  @ 22  Inline$Point::<init> (15 bytes)   inline (hot)
  ...
@ 25  Inline$Point::lengthSquared (20 bytes)   inline (hot)

and here's what escape analysis is worth, measured with ThreadMXBean.getThreadAllocatedBytes over 50 million calls after warmup, with the default and with -XX:-DoEscapeAnalysis:

15 times faster, and the GC never hears about these objects. which is why "don't allocate small objects, they're expensive" is mostly outdated advice: short-lived objects that stay local are often free. the catch is the word local.

what decides inlining

C2 inlines a call when it's both possible and worth it. possible: the target must be known. for a static, private, final or constructor call it always is. for a virtual or interface call, C2 uses the type profile from tier 3:

@ 27   Mega$Circle::area (14 bytes)   inline (hot)
       \-> TypeProfile (40960/40960 counts) = Mega$Circle
 
@ 27   Mega$Circle::area (14 bytes)   inline (hot)
@ 27   Mega$Square::area (10 bytes)   inline (hot)
       \-> TypeProfile (20480/40960 counts) = Mega$Square
       \-> TypeProfile (20480/40960 counts) = Mega$Circle
 
@ 27   Mega$Shape::area (0 bytes)   failed to inline: virtual call

one class (monomorphic): inline it behind a cheap class check. two (bimorphic): inline both, check which one you have. three or more (megamorphic): give up and do a real vtable or itable call. here's the loop for (Shape s : shapes) t += s.area() over 4096 shapes, each run in a fresh JVM:

worth it: methods under MaxInlineSize = 35 bytes of bytecode are inlined even when they aren't hot, hot ones up to FreqInlineSize = 325 bytes, nested at most MaxInlineLevel = 15 deep, and not if the callee is already compiled into a big chunk of machine code (InlineSmallCode). that's why small methods are good for performance, not bad: they're exactly what inlining loves.

loops

hot code is mostly loops, and C2 has a bag of tricks for them. range check elimination: a[i] normally checks 0 <= i < a.length every time; when i runs from 0 to a.length, C2 proves the checks can't fail and hoists them out of the loop. unrolling: do several iterations per trip around the loop, so the loop's own overhead (compare, jump, increment) is paid less often. vectorization (C2 calls it SuperWord): turn several iterations into single SIMD instructions that work on 4 or 8 ints at once.

here's b[i] = a[i] + c[i] over 4096 ints, turning each trick off in turn:

12 times between C1 and C2 for a loop you'd never think twice about. (flags used: -XX:LoopUnrollLimit=1 to stop unrolling, -XX:-UseSuperWord to stop vectorizing.)

intrinsics

some JDK methods are never compiled from their bytecode at all. the JVM recognizes them by name and drops in hand-written machine code instead. PrintInlining marks them:

@ 25   java.lang.Integer::bitCount (53 bytes)   (intrinsic)
@ 1    java.lang.Math::sqrt (5 bytes)   (intrinsic)
@ 11   java.lang.System::arraycopy (0 bytes)   (intrinsic)
@ 44   java.lang.StringLatin1::equals (36 bytes)   (intrinsic)

Integer.bitCount is 53 bytes of clever bit-twiddling in Integer.java, but on this machine C2 uses the CPU's population-count instruction instead. counting bits over a 4096-int array took 0.12 ns per element normally and 0.25 ns with -XX:DisableIntrinsic=_bitCount_i. there are hundreds of intrinsics: System.arraycopy, string comparisons and searches, Math functions, CRC32, AES and SHA, Object.hashCode, Thread.currentThread. the practical upshot: use the JDK method instead of hand-rolling it. yours won't be intrinsified.

speculation and deoptimization

everything above is a bet. "this call site only ever sees Circle" is true so far. C2 compiles as if it's always true, with a cheap check that falls off a cliff if it isn't: an uncommon trap. when a trap fires, the JVM throws away the compiled code (marks it not entrant), rebuilds an interpreter frame from the compiled frame's state, and continues in the interpreter. later, with a better profile, it compiles again.

here's a real one. the call site in area sees 200,000 Circles, then one Square:

static double area(Shape s) {
    return s.area();
}
// phase 1: area(new Circle(i)) 200,000 times
// phase 2: area(new Square(2)), then a mix of both

14   21       4       Deopt::area (7 bytes)
phase 2: a square shows up
[0.017s][debug][deoptimization] cid=21 level=4 Deopt.area(LDeopt$Shape;)D trap_bci=1 class_check maybe_recompile
17   21       4       Deopt::area (7 bytes)   made not entrant: uncommon trap
17   25       4       Deopt::area (7 bytes)

deoptimization is what makes aggressive optimization safe. C2 can assume a branch is never taken, a class has no subclasses (class hierarchy analysis), or a value is never null, because being wrong costs a recompile instead of a wrong answer. the price shows up as occasional hiccups when a program changes phase.