JEP 178: Statically-Linked JNI Libraries

Summary

Enhance the JNI specification to support statically linked native libraries.

Goals

1. Modify the Java SE specification, and the JDK, to enable developers to package a Java runtime, native application code, and Java application code together into a single binary executable that does not require the use of shared native libraries.

2. Require no changes to existing Java code in order to use a static native library as opposed to a dynamic native library. A method invocation of the form System.loadLibrary("foo"), in particular, should be able to load the "foo" library regardless of whether that library is provided in static or dynamic form.

3. Allow a Java application to use a combination of static and dynamic native libraries, although static libraries must be in memory prior to any attempt to use them.

4. Allow JVMTI Java Agents to be optionally statically linked with Java runtimes.

Non-Goals

It is not a goal to preserve complete native C/C++ source compatibility for existing dynamic native libraries that are converted to static form. Existing uses of the JNI_OnLoad and JNI_OnUnLoad functions will need to be modified in order to allow multiple static libraries to co-exist.

Motivation

There are two major scenarios in which static JNI libraries can be useful:

1. Native applications that embed the JRE may wish to use statically linked JNI code rather than dynamically-linked libraries.

2. Java applications running in environments that limit or do not support shared libraries require a JRE and all of its native API-library code to be linked into a single executable.

As an additional benefit, with statically linked JNI libraries an object-file linker can optimize the entire executable, potentially reducing its size.

Description

Two main problems need to be addressed to add support for static JNI libraries:

1. The current Java API that initiates the dynamic-library loading process needs to be enhanced to support built-in static libraries. A Java application that uses a static JNI library needs a way to notify the VM that the library code is already included in the application image. In this situation, a System.loadLibrary request for a static library should skip the usual platform-specific dynamic loading process.

   The current JNI specification alludes to this type of support, though the Hotspot VM does not implement that behavior.

2. The JNI_OnLoad and JNI_OnUnload function interface need to be enhanced to support library specific names since only a single function name can exist within an application. This could be implemented by appending the library name to these well-known-names. For example libnet.so could use JNI_OnLoad_net, JNI_OnUnload_net.

This feature requires changes to both the Java SE library-loading APIs and the JNI specification. What follows is an initial draft of the specification updates in both areas.

Java API Changes

The specifications of the java.lang.System.load and java.lang.Runtime.load methods will be revised to read:

Loads the native library specified by the filename argument. The filename argument must be an absolute path name.

If the filename argument, when stripped of any platform-specific library prefix, path, and file extension, indicates a library whose name is L, and a native library called L is statically linked with the VM, then the JNI_OnLoad_L function exported by the library is invoked rather than attempting to load a dynamic library. A filename matching the argument does not have to exist in the file system. See the JNI Specification for more details.

Otherwise, the filename argument is mapped to a native library image in an implementation-dependent manner.

The specifications of when these methods throw an UnsatisfiedLinkError will be revised to read:

UnsatisfiedLinkError - if either the filename is not an absolute path name, the native library is not statically linked with the VM, or the library cannot be mapped to a native library image by the host system.

The specifications of the java.lang.System.loadLibrary and java.lang.Runtime.loadLibrary methods will be revised to read:

Loads the native library specified by the libname argument. The libname must not contain any platform-specific prefix, file extension, or path.

If a native library called libname is statically linked with the VM, then the JNI_OnLoad_libname function exported by the library is invoked. See the JNI Specification for more details.

Otherwise, the libname is loaded from a system library location and mapped to a native-library image in an implementation-dependent manner.

The specifications of when these methods throw an UnsatisfiedLinkError will be revised to read:

UnsatisfiedLinkError - if either the libname argument contains a file path, the native library is not statically linked with the VM, or the library cannot be mapped to a native-library image by the host system.

JNI Specification Changes

• A native library may be statically linked with the VM. The manner in which the library and VM image are combined is implementation-dependent.

• A System.loadLibrary or equivalent API call must succeed for this library to be considered loaded.

• A library L whose image has been combined with the VM is defined as statically linked if and only if the library exports a function called JNI_OnLoad_L.

• If a statically linked library L exports a function called JNI_OnLoad_L and a function called JNI_OnLoad, the JNI_OnLoad function will be ignored.

• If a library L is statically linked then upon the first invocation of System.loadLibrary("L") or equivalent, the JNI_OnLoad_L function will be invoked with the same arguments and expected return value as specified for the JNI_OnLoad function.

• If a library L is statically linked then it will be prohibited to link a library of the same name dynamically.

• When the class loader containing a statically linked native library L is garbage collected, the VM will invoke the JNI_OnUnload_L function of the library if such a function is exported.

• If a statically linked library L exports a function called JNI_OnUnLoad_L and a function called JNI_OnUnLoad, the JNI_OnUnLoad function will be ignored.

The JNI version specification will be incremented to JNI_VERSION_1_8. Statically-linked libraries will only be supported this version or greater.

JVMTI -agentlib Command Line Option Specification Changes

The -agentlib command line specification descriptions will be revised in JDK 8 to read:

If the library argument, indicates a library whose name is L, and a native library called L is statically linked with the VM, then an Agent_OnLoad_L function must be exported by the agent. A library matching the argument does not have to exist in the file system. This Agent_OnLoad_L function will be invoked by the VM as described in the JVMTI specification. The options will be passed to the Agent_OnLoad_L function when called.

Otherwise, The name following -agentlib: is the name of the library to load. Lookup of the library, both its full name and location, proceeds in a platform-specific manner. Typically, the agent-lib-name is expanded to an operating system specific file name. The options will be passed to the agent on start-up. For example, if the option -agentlib:foo=opt1,opt2 is specified, the VM will attempt to load the shared library foo.dll from the system PATH under WindowsTM or libfoo.so from the LD_LIBRARY_PATH under the SolarisTM operating environment.

JVMTI -agentpath Command Line Option Specification Changes

The -agentpath command line specification descriptions will be revised in JDK 8 to read:

If the filename argument, when stripped of any platform-specific library prefix, path, and file extension, indicates a library whose name is L, and a native library called L is statically linked with the VM, then the Agent_OnLoad_L function must be exported by the agent. A filename matching the argument does not have to exist in the file system. This Agent_OnLoad_L function is invoked by the VM, as described in the JVMTI specification. The options will be passed to the Agent_OnLoad_L function when called.

Otherwise, The path following -agentpath: is the absolute path from which to load the library. No library name expansion will occur. The options will be passed to the agent on start-up. For example, if the option -agentpath:/myLibs/foo.so=opt1,opt2 is specified, the VM will attempt to load the shared library /myLibs/foo.so.

JVMTI Native Interface Specification Changes

• A native JVMTI Agent may be statically linked with the VM. The manner in which the library and VM image are combined is implementation-dependent.

• An agent L whose image has been combined with the VM is defined as statically linked if and only if the agent exports a function called Agent_OnLoad_L.

• If a statically linked agent L exports a function called Agent_OnLoad_L and a function called Agent_OnLoad, the Agent_OnLoad function will be ignored.

• If an agent L is statically linked, an Agent_OnLoad_L function will be invoked with the same arguments and expected return value as specified for the Agent_OnLoad function.

• An agent L that is statically linked will prohibit an agent of the same name from being loaded dynamically.

• The VM will invoke the Agent_OnUnload_L function of the agent, if such a function is exported, at the same point during startup as it would have called the dynamic entry point Agent_OnUnLoad.

• If a statically linked agent L exports a function called Agent_OnUnLoad_L and a function called Agent_OnUnLoad, the Agent_OnUnLoad function will be ignored.

• If an agent L is statically linked, an Agent_OnAttach_L function will be invoked with the same arguments and expected return value as specified for the Agent_OnAttach function.

• If a statically linked agent L exports a function called Agent_OnAttach_L and a function called Agent_OnAttach, the Agent_OnAttach function will be ignored.

com.sun.tools.attach.VirtualMachine.loadAgentLibrary

This language will be added to the javadocs for this method:

If the agent is statically linked with the VM that would otherwise load it, the specific Agent_OnAttach function name that is called will be library specific as defined in the -agentlib JVMTI specification section.

com.sun.tools.attach.VirtualMachine.loadAgentPath

This language will be added to the javadocs for this method:

If the agent is statically linked with the VM that would otherwise load it, the specific Agent_OnAttach function name that is called will be library specific as defined in the -agentpath JVMTI specification.

The JVMTI version specification will be incremented to JDK18_JVMTI_VERSION.
JDK18_JVMTI_VERSION will be set to 0x30010203, which translates to 1.2.3.

This new functionality will be supported in Virtual Machines that support JDK18_JVMTI_VERSION or greater.

Impact

• Compatibility: This new functionality should not affect existing dynamic libraries.
• Portability: JNI native source code requires function-name changes when built statically.
• TCK: JNI native-library tests will need to be adapted to validate support for statically linked native libraries.
• TCK: JVMTI agent tests will need to be adapted to validate support for statically linked agent libraries.