OpenJDK: Check JIT Disassembly Code

We’re using Ubuntu 26.04 and OpenJDK-26 version on AArch64 machine.

hsdis

Note from Building hsdis in 2022. Jorn Vernee:

hsdis is a disassembler plugin for the OpenJDK/HotSpot JVM. It can be used in conjunction with the PrintAssembly option (as well as other options) to disassemble and print out code generated by HotSpot’s JIT compilers. It is a separate shared library that can be installed on the PATH or in the JDK directory next to (lib)jvm.(dll/so/dylib). The VM will dynamically load this library and call the function it exposes to disassemble dynamically generated code.

• Manual build: See the steps in Building hsdis in 2022. Jorn Vernee
• Prebuilt package: See OpenJDK hsdis (HotSpot disassembly plugin) downloads

Note: Need to put it under /usr/lib directory or export the path to LD_LIBRARY_PATH.

PrintAssembly Option

Here list the commonly used PrintAssembly options.

Option	Usage
-XX:+PrintAssembly	print assembly code for bytecode and native methods
-XX:+PrintNMethods	print nmethods as they are generated
-XX:+PrintNativeNMethods	print native method wrappers as they are generated
-XX:+PrintSignatureHandlers	print native method signature handlers
-XX:+PrintAdapterHandlers	print adapters (i2c, c2i) as they are generated
-XX:+PrintStubCode	print stubs: deopt, uncommon trap, exception, safepoint, runtime support
-XX:+PrintInterpreter	print interpreter code

Filtering Output

Note from PrintAssembly:

The -XX:+PrintAssembly option prints everything. If that’s too much, drop it and use one of the following options. Individual methods may be printed:

• -XX:CompileCommand=print,*MyClass.myMethod prints assembly for just one method

Commonly Used Options

Reference: Building hsdis in 2022. Jorn Vernee

Option	Usage
-Xbatch	block execution until the JIT finishes, so we can get our assembly before the program exits
-Xcomp	disable Interpreter
-XX:-TieredCompilation	disable the C1 JIT compiler, so we get a somewhat reduced output. The C2 output is usually what’s interesting, as that is the most optimized.
-XX:CompileCommand=dontinline,Main::add*	disable inlining of the add method, so that we get a clean compilation of that method without it being inlined into the loop, and also so that the JIT doesn’t know that the return value is not actually being used.
-XX:CompileCommand=PrintAssembly,Main::add*	print out the assembly for the add method.
-XX:CompileCommand=compileonly,Main::add*	only compile add method. interpret other methods. further reduce the output.
-XX:CompileCommand=print,Main::add*	print out the assembly for the add method.

Simple Stand-alone Java Case

Note: this test case is copied from Building hsdis in 2022. Jorn Vernee and minor change is done.

add1() is invoked 100 times while add2() is invoked more times.

public class Main {
  public static void main(String[] args) {
    for (int i = 0; i < 100; i++) {
      add1(42, 42);
    }
    for (int i = 0; i < 20_000; i++) {
      add2(42, 42);
    }
  }

  private static int add1(int a, int b) {
    return a + b;
  }
  private static int add2(int a, int b) {
    return a + b;
  }
}

Hot Code

Only dontinline and print are passed. As add1() is not hot, we can only get the disassembly code for add2().

$ javac Main.java
$ java -Xbatch -XX:CompileCommand=dontinline,Main::add* \
  -XX:CompileCommand=print,Main::add* Main >res.txt
$ grep "Compiled method .* Main::add.*" res.txt
Compiled method (c1) 774  469       3       Main::add2 (4 bytes)
Compiled method (c2) 779  470       4       Main::add2 (4 bytes)

Disable C1

Further pass -XX:-TieredCompilation and we can only get the disassembly code generated by C2 for add2() now.

$ java -Xbatch -XX:-TieredCompilation -XX:CompileCommand=dontinline,Main::add* \
  -XX:CompileCommand=print,Main::add* Main >res.txt
$ grep "Compiled method .* Main::add.*" res.txt
Compiled method (c2) 455  178             Main::add2 (4 bytes)

Always C2

Further pass -Xcomp and we can get the disassembly code generated for add1() now.

$ java  -Xbatch -Xcomp -XX:-TieredCompilation \
  -XX:CompileCommand=dontinline,Main::add* -XX:CompileCommand=print,Main::add* \
  Main >res.txt
$ grep "Compiled method .* Main::add.*" res.txt
Compiled method (c2) 6808 3157             Main::add1 (4 bytes)
Compiled method (c2) 6812 3158             Main::add2 (4 bytes)

More Internal Method besides add2

Following the section Disable C1, replace -XX:CompileCommand=print,Main::add* with the raw -XX:+PrintAssembly. As a result, the disassembly code for more internal methods are printed.

$ java -Xbatch -XX:-TieredCompilation -XX:CompileCommand=dontinline,Main::add* \
  -XX:+PrintAssembly Main >res.txt
$ grep "Compiled method" res.txt | wc -l
178
$ grep "Compiled method" res.txt | head -10
Compiled method (n/a) 92    1     n       jdk.internal.misc.Unsafe::getReferenceVolatile (native)
Compiled method (n/a) 124   2     n       java.lang.Object::hashCode (native)
Compiled method (n/a) 126   3     n       java.lang.invoke.MethodHandle::linkToStatic(LLLLLLL)L (native)
Compiled method (c2) 133    4             java.lang.Enum::ordinal (5 bytes)
Compiled method (n/a) 139   5     n       java.lang.System::arraycopy (native)
Compiled method (c2) 153    6             jdk.internal.classfile.impl.AbstractPoolEntry::index (5 bytes)
Compiled method (n/a) 156   7     n       java.lang.invoke.MethodHandle::linkToSpecial(LLL)L (native)
Compiled method (c2) 158    8             jdk.internal.classfile.impl.AbstractPoolEntry::width (2 bytes)
Compiled method (c2) 161    9             java.lang.Class::isPrimitive (5 bytes)
Compiled method (n/a) 162  10     n       java.lang.invoke.MethodHandle::invokeBasic(LLLLLL)L (native)

JTreg Test

Note: Debug build is used.

In order to print out the disassembly code generated by JIT compiler for one test or one specific function in one test, just pass the PrintAssembly options discussed above to the JTreg test framework.

• Use make test TEST=XYZ.java
  • Set JTREG='VM_OPTIONS=AAA BBB'. It will pass JDK options AAA BBB to the test JVM.
• Run jtreg binary directly
  • JDK options can be specified via -vmoption flag

Initially make test TEST=XYZ.java would be essentially expanded to jtreg test command. Hence, we only talk about make test TEST=XYZ.java in the following section.

Simple Case

Take the JTreg case TestAbs.java as an example. Here shows the test config.

• othervm mode is used, i.e. forking one child JVM and running the test there
  • By setting JTREG='VM_OPTIONS=AAA BBB', we can pass extra JDK options to this child JVM.
• -XX:-TieredCompilation -Xbatch is specified. We talked about them early.
• -XX:CompileOnly=java.lang.Math::abs: Only compile Math::abs

 * @requires vm.debug == true
 *
 * @run main/othervm -XX:-TieredCompilation -Xbatch -XX:CompileOnly=java.lang.Math::abs compiler.c2.TestAbs

As shown in the following test commands, we can check the disassembly code for function Math::abs now.

$ log_file=/tmp/abc.log
$ test_case=test/hotspot/jtreg/compiler/c2/TestAbs.java
# Print out all methods
$ make test TEST=$test_case JTREG="VM_OPTIONS=-XX:+PrintAssembly \
  -XX:LogFile=$log_file"
# Or, print out only Math::abs
$ make test TEST=$test_case \
  JTREG="VM_OPTIONS=-XX:CompileCommand=print,*Math::abs -XX:LogFile=$log_file"
$ grep "Compiled\ method" $log_file
Compiled method (c2) 135   39             java.lang.Math::abs (13 bytes)

Case with IR Test

Take the JTreg case VectorAbsDiffTest.java as an example. This test case has a number of sub-tests inside and each sub-test has one IR test respectively. It differs from TestAbs.java in that driver test mode rather than othervm is used.

 * @library /test/lib /
 * @run driver compiler.vectorapi.VectorAbsDiffTest

driver VM would create one test VM to run each @Test method. Hence, we specify to run one method via -DTest=XZY and pass the extra JDK options to this test VM. Note that -DReportStdout=true is set to print out the output of this test VM.

$ log_file=/tmp/abc.log
$ test_case=test/hotspot/jtreg/compiler/vectorapi/VectorAbsDiffTest.java
$ make test TEST=$test_case JTREG="VM_OPTIONS=-DTest=testFloatAbsDiff \
  -DReportStdout=true \
  -XX:CompileCommand=print,compiler.vectorapi.VectorAbsDiffTest::testFloatAbsDiff \
  -XX:LogFile=$log_file"
$ grep "Compiled\ method" $log_file
Compiled method (c1) 1098  837       3       compiler.vectorapi.VectorAbsDiffTest::testFloatAbsDiff (64 bytes)
Compiled method (c2) 1175  848 %     4       compiler.vectorapi.VectorAbsDiffTest::testFloatAbsDiff @ 2 (64 bytes)
Compiled method (c2) 1195  849       4       compiler.vectorapi.VectorAbsDiffTest::testFloatAbsDiff (64 bytes)

JMH Test

Note: Release build is used.

Take test/micro/org/openjdk/bench/jdk/incubator/vector/VectorMultiplyOptBenchmark.java as an example. We can use the following command to run one specify benchmark with one specific parameter.

$ make test TEST="micro:VectorMultiplyOptBenchmark.test_bm_pattern1" \
  MICRO="OPTIONS=-p SIZE=1024"

Same as JTreg, we can obtain the disassembly code by passing some PrintAssembly related options.

$ log_file=/tmp/abc.log
# All methods are printed out
$ make test TEST="micro:VectorMultiplyOptBenchmark.test_bm_pattern1" \
  MICRO="OPTIONS=-p SIZE=1024;VM_OPTIONS=-XX:+UnlockDiagnosticVMOptions \
  -XX:+PrintAssembly" >$log_file
# Note that test_bm_pattern1_thrpt_jmhStub is the wrapper to run test_bm_pattern1
$ grep "Compiled\ method" $log_file | grep "test_bm_pattern1"
Compiled method (c1) 5352 1035       3       org.openjdk.bench.jdk.incubator.vector.VectorMultiplyOptBenchmark::test_bm_pattern1 (92 bytes)
Compiled method (c2) 5618 1042 %     4       org.openjdk.bench.jdk.incubator.vector.VectorMultiplyOptBenchmark::test_bm_pattern1 @ 2 (92 bytes)
Compiled method (c2) 5721 1064       4       org.openjdk.bench.jdk.incubator.vector.VectorMultiplyOptBenchmark::test_bm_pattern1 (92 bytes)
Compiled method (c1) 5738 1065 %     3       org.openjdk.bench.jdk.incubator.vector.jmh_generated.VectorMultiplyOptBenchmark_test_bm_pattern1_jmhTest::test_bm_pattern1_thrpt_jmhStub @ 13 (52 bytes)
Compiled method (c1) 5749 1066       3       org.openjdk.bench.jdk.incubator.vector.jmh_generated.VectorMultiplyOptBenchmark_test_bm_pattern1_jmhTest::test_bm_pattern1_thrpt_jmhStub (52 bytes)
Compiled method (c2) 5800 1067 %     4       org.openjdk.bench.jdk.incubator.vector.jmh_generated.VectorMultiplyOptBenchmark_test_bm_pattern1_jmhTest::test_bm_pattern1_thrpt_jmhStub @ 13 (52 bytes)
Compiled method (c2) 6698 1073 %     4       org.openjdk.bench.jdk.incubator.vector.jmh_generated.VectorMultiplyOptBenchmark_test_bm_pattern1_jmhTest::test_bm_pattern1_thrpt_jmhStub @ 13 (52 bytes)
Compiled method (c2) 7721 1076       4       org.openjdk.bench.jdk.incubator.vector.jmh_generated.VectorMultiplyOptBenchmark_test_bm_pattern1_jmhTest::test_bm_pattern1_thrpt_jmhStub (52 bytes)

# Only test_bm_pattern1 is printed out
make test TEST="micro:VectorMultiplyOptBenchmark.test_bm_pattern1" \
  MICRO="OPTIONS=-p SIZE=1024;VM_OPTIONS=-XX:+UnlockDiagnosticVMOptions \
  -XX:CompileCommand=print,*VectorMultiplyOptBenchmark::test_bm_pattern1" \
  >$log_file
$ grep "Compiled\ method" $log_file
Compiled method (c1) 998 1056       3       org.openjdk.bench.jdk.incubator.vector.VectorMultiplyOptBenchmark::test_bm_pattern1 (92 bytes)
Compiled method (c2) 1087 1057 %     4       org.openjdk.bench.jdk.incubator.vector.VectorMultiplyOptBenchmark::test_bm_pattern1 @ 2 (92 bytes)
Compiled method (c2) 1139 1058       4       org.openjdk.bench.jdk.incubator.vector.VectorMultiplyOptBenchmark::test_bm_pattern1 (92 bytes)

Besides make test TEST=XX, we can also run Java binary with the jar package to execute the benchmark. Note: we should pass the VM options via JMH option --jvmArgsAppend.

# Run this benchmark
$ ./jdk/bin/java -jar  ./images/test/micro/benchmarks.jar -f1 -i10 \
  VectorMultiplyOptBenchmark.test_bm_pattern1 -p SIZE=1024
# Print out the disassembly
$ ./jdk/bin/java -jar ./images/test/micro/benchmarks.jar -f1 -i10 \
  VectorMultiplyOptBenchmark.test_bm_pattern1 -p SIZE=1024 \
  -jvmArgsAppend "-XX:+UnlockDiagnosticVMOptions \
  -XX:CompileCommand=print,*VectorMultiplyOptBenchmark::test_bm_pattern1" \
  >$log_file