Semantic Versions in Java 9 modules as a safety-net

When building a modular system not all modules on the module path will be in your own control. Some of them will be third party libraries and others could be provided by other teams within your organisation. Nevertheless a modular software will be composed from a carefully selected mix of all kinds of modules.

There is no guarantee that all these modules will be compatible with each other. For example, consider a library module compiling against a version of a dependency module that has a required java method. When running your application you put that same library on the module path with an earlier version of that dependency module which is missing that method. Then certain execution scenarios will only fail at runtime indicating incompatibilities. We need to make sure that we use compatible versions of our modules when composing applications. We need to fail fast with a safety-net!

Semantic versioning as safety-net

Semantic versioning of modules allows to add meaning to the versions we give to modules. For more details I refer to semver.org, in summary a semantic version is formatted as MAJOR.MINOR.PATCH, we increment the:

1. MAJOR version when you make incompatible or breaking API changes,
2. MINOR version when you add functionality in a backwards-compatible manner, and
3. PATCH version when you make backwards-compatible bug fixes.

Having this extra semantics for all module versions in our modular application would allow us to determine if a certain composition of modules is compatible with each other. This safety-net avoids the typical ‘dependency hell’.

For example, consider a module called “Foo”. It requires a module named “Bar”. At the time that Foo is created, Bar is at version 1.0.0. Since Foo uses some functionality that was first introduced in 1.0.0, you can use all versions of Bar greater than or equal to 1.0.0 but less than 2.0.0. Now, when Bar version 1.0.1 and 1.1.0 become available, you can release them, use them together with the original Foo module and know that they will be compatible.

So we need to use the module versions to automatically check and validate for incompatible modules.

Wait! Java modules do not support versioning?

There is 1 big problem. The first java module system in Java 9 does not support putting versions inside our module-info.java source code to indicate which version of that module we are looking at and/or indicate which version of other modules we require.

The following rationale can be found in ‘The State of the module system‘:

A module’s declaration does not include a version string, nor constraints upon the version strings of the modules upon which it depends. This is intentional: It is not a goal of the module system to solve the version-selection problem, which is best left to build tools and container applications.

and in the jigsaw spec:

Version selection — The process of configuring a set of modules need not consider more than one version of any particular module.

In other words, this specification need not define yet another dependency-management mechanism. Maven, Ivy, and Gradle have all tackled this difficult problem. We should leave it to these and other build tools, and container applications, to discover and select a set of candidate modules for a given library or application. The module system need only validate that the set of selected modules satisfies each module’s dependencies.

I agree with the fact that the module system should not solve selecting the right combination of dependencies, that is up to the build systems. However, the last sentence also states that the module system should validate that the set of selected modules satisfies each module’s dependencies!

Earlier in this post it was shown that satisfying a dependency also involves code compatibility. Problems arise when another version than expected is selected. Only checking that required modules are present based on the module name is not good enough as safety-net.

To support this validation logic it should be possible to add semantic version information to the module and its required dependencies. This is not possible in the module-info.java source code.

Strange, Java modules can have version info at runtime !?

Although version information is not supported in de module source code, there are methods on the Module API to retrieve versions:

• module.getDescriptor().version() which returns e.g. ‘9’ in the case of java.base
• module.getDescriptor().requires().compiledVersion() which returns the version of a ‘requires’ clause

Then how do these properties get a value?

Adding a module version @ jar time

As it is not possible to put a version in the source code of a module, it cannot be compiled into it. It can be added to the jar when packaging the module. The –module-version option should be used with the jar command:

jar --update --module-version 1.0.0 --file jar-to-update-with-version.jar -C classes/ .

Using maven, at the moment of writing, the maven-jar-plugin does not yet support this. We can use the exec-maven-plugin to update the jar packaged by the maven-jar-plugin with the maven ${project.version} as follows:

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>exec-maven-plugin</artifactId>
    <version>1.6.0</version>
    <executions>
       <execution>
          <id>add-version-to-jar</id>
          <phase>package</phase>
          <goals>
             <goal>exec</goal>
          </goals>
          <configuration>
             <executable>jar</executable>
             <workingDirectory>${project.build.directory}</workingDirectory>
             <arguments>
                <argument>--update</argument>
                <argument>--verbose</argument>
                <argument>--module-version</argument>
                <argument>${project.version}</argument>
                <argument>--file</argument>
                <argument>${project.build.finalName}.jar</argument>
                <argument>-C</argument>
                <argument>${project.build.outputDirectory}</argument>
                <argument>.</argument>
             </arguments>
          </configuration>
       </execution>
    </executions>
 </plugin>

On github a version-info module is available allowing to print all versioned modules in a runtime using VersionInfo.print() which results in output like this:

-------------------------------------------------------------- 
Version information for modules in layer 
'jdk.javadoc, jdk.scripting.nashorn, be.aca.poc.events.kafka.membership, java.rmi, jdk.charsets, java.security.sasl, jdk.crypto.ec, java.management.rmi, jdk.crypto.cryptoki, jdk.localedata, java.management, jdk.jartool, spring.jcl, java.xml, jdk.jdeps, jdk.security.auth, be.aca.poc.events.kafka.application, java.smartcardio, java.xml.crypto, spring.expression, jdk.zipfs, java.security.jgss, jdk.naming.rmi, jdk.security.jgss, jdk.naming.dns, jdk.deploy, jdk.unsupported, io.github.tomdw.jpms.version.info, be.aca.poc.events.kafka.assessment, java.compiler, spring.context, jdk.internal.opt, spring.beans, java.logging, java.desktop, jdk.jlink, be.aca.poc.events.kafka.supplychain, java.scripting, java.naming, jdk.management, jdk.dynalink, java.datatransfer, javax.inject, io.github.tomdw.jpms.context.boot, jdk.compiler, java.base, spring.core, java.prefs, spring.aop, java.sql' 
--------------------------------------------------------------
... 
ModuleVersion[name=io.github.tomdw.jpms.versions.samples.basicapplication.application, version=1.0.0-SNAPSHOT] 
ModuleVersion[name=io.github.tomdw.jpms.versions.samples.basicapplication.speaker, version=1.0.0-SNAPSHOT] 
ModuleVersion[name=io.github.tomdw.jpms.versions.samples.basicapplication.microphone, version=1.1.0-SNAPSHOT] 
ModuleVersion[name=io.github.tomdw.jpms.version.info, version=1.0.0-SNAPSHOT] 
... 
ModuleVersion[name=spring.context, version=5.0.0.RELEASE] 
... 
ModuleVersion[name=spring.beans, version=5.0.0.RELEASE] 
... 
ModuleVersion[name=java.logging, version=9] ModuleVersion[name=java.desktop, version=9] 
... 
ModuleVersion[name=javax.inject, version=1] 
... 
ModuleVersion[name=java.base, version=9] 
ModuleVersion[name=spring.core, version=5.0.0.RELEASE] 
...

Next to some modules we created ourselves, also the jdk modules like java.base and even spring includes the version.

‘requires’ gets a version @ compile time

It is less clear how and when also the version for a requires is filled in. The name of the method compiledVersion() provides the hint that it is done at compile time. But if it is not possible to add it in source code, how can it be filled in?

As it turns out, when a module with a requires is compiled against another module which has had its version added to the jar’s module-info.class, then the requires.compiledVersion() also gets that version in the byte-code.

So as long as modules include their version, then requires clauses on it will also get it. Again using the tool on github a we can show the version info in the requires clauses:

--------------------------------------------------------------- 
Version information for requires in layer 
'java.datatransfer, jdk.zipfs, jdk.unsupported, jdk.localedata, jdk.dynalink, io.github.tomdw.jpms.versions.samples.basicapplication.application, java.security.sasl, java.base, io.github.tomdw.jpms.versions.samples.basicapplication.speaker, jdk.internal.opt, java.xml, jdk.naming.dns, jdk.jlink, jdk.management, jdk.charsets, java.rmi, jdk.javadoc, jdk.jartool, java.desktop, jdk.crypto.cryptoki, java.xml.crypto, jdk.naming.rmi, java.naming, io.github.tomdw.jpms.versions.samples.basicapplication.microphone, java.smartcardio, java.scripting, java.logging, java.security.jgss, java.prefs, jdk.crypto.ec, java.management, jdk.jdeps, jdk.compiler, java.compiler, jdk.security.auth, jdk.security.jgss, jdk.scripting.nashorn, jdk.deploy, java.management.rmi, io.github.tomdw.jpms.version.info' 
--------------------------------------------------------------- 
ModuleRequiresVersion[requiringModule=io.github.tomdw.jpms.versions.samples.basicapplication.application, requiredModule=io.github.tomdw.jpms.versions.samples.basicapplication.speaker, version=1.0.0-SNAPSHOT] 
ModuleRequiresVersion[requiringModule=io.github.tomdw.jpms.versions.samples.basicapplication.application, requiredModule=java.base, version=9] 
ModuleRequiresVersion[requiringModule=io.github.tomdw.jpms.versions.samples.basicapplication.application, requiredModule=io.github.tomdw.jpms.version.info, version=1.0.0-SNAPSHOT] 
ModuleRequiresVersion[requiringModule=io.github.tomdw.jpms.versions.samples.basicapplication.speaker, requiredModule=java.base, version=9] 
ModuleRequiresVersion[requiringModule=io.github.tomdw.jpms.versions.samples.basicapplication.speaker, requiredModule=io.github.tomdw.jpms.versions.samples.basicapplication.microphone, version=1.0.0-SNAPSHOT] 
ModuleRequiresVersion[requiringModule=io.github.tomdw.jpms.versions.samples.basicapplication.microphone, requiredModule=java.base, version=9] 
ModuleRequiresVersion[requiringModule=io.github.tomdw.jpms.version.info, requiredModule=java.base, version=9] 
---------------------------------------------------------------

Note: although spring.core shows its version in an output shown earlier, a module requiring it does not get that version in its requires. We make the assumption that it only works with explicit modules and not with automatic modules like spring.core. (edit: this was confirmed by the Jigsaw team on the jigsaw-dev mailing list, see http://mail.openjdk.java.net/pipermail/jigsaw-dev/2018-January/013457.html)

Validating compatibility

At this point we have an approach to add the version inside a module jar and know when the version information is put into the ‘requires’ clauses of a module jar.

The next step is to validate if all modules present have versions that are compatible with the versions that were required at compile time.

Using the ModuleLayer API it is quite easy to traverse and find all modules in for example the boot layer:

ModuleLayer.boot().modules();

ModuleLayer.boot().findModule(moduleName);

Using the Module API we can inspect the available versions of all modules:

String moduleName = module.getName();

if (module.getDescriptor().version().isPresent()) {
    String moduleVersion = module.getDescriptor().version().get().toString();
}

and inspect the required versions of all requires clauses:

module.getDescriptor().requires().forEach(requires -> {
    String requiredModuleName = requires.name();
    if (requires.compiledVersion().isPresent()) {
        String requiredModuleVersion = requires.compiledVersion().get().toString();
    }
});

Only if both versions are filled in it makes sense to compare the version of the requires and the version of the available module with that name. For the proof of concept we take the assumption that all versions are semantic versions (which is not always valid).

If there is no module on the module path with the requiredModuleName then not everything is available to consider it a compatible module path.

If there is a module found on the module path with the requiredModuleName then we can check if that module’s version is semantically compatible with the requiredModuleVersion. A SemanticVersion utility class is used in the code that parses the given version string for a MAJOR.MINOR.PATCH pattern, and checks backwards compatibility with this condition:

major == expectedVersion.getMajor() && minor >= expectedVersion.getMinor()

Putting it all together

On github there is a sample application which has a module graph as follows:

The module io.github.tomdw.jpms.version.info is a re-usable module to print and validate version compatibility in any java module path application.

The module application uses the module speaker to deliver a given message to the user. The module speaker will use a Microphone service from the microphone module for this. Here a SimpleMicrophone will print the message to the console.

A compatiblemodulepath pom.xml is setup that allows to run the application with a speaker requiring version 1.0.0-SNAPSHOT of microphone and using a compatible feature version 1.1.0-SNAPSHOT of microphone on the module path. To run it execute:

mvn toolchains:toolchain exec:exec

The validation output is:

------------------------------------------------------------------- 
Validating module compatibility for modules in layer 
java.datatransfer, jdk.zipfs, jdk.unsupported, jdk.localedata, jdk.dynalink, io.github.tomdw.jpms.versions.samples.basicapplication.application, java.security.sasl, java.base, io.github.tomdw.jpms.versions.samples.basicapplication.speaker, jdk.internal.opt, java.xml, jdk.naming.dns, jdk.jlink, jdk.management, jdk.charsets, java.rmi, jdk.javadoc, jdk.jartool, java.desktop, jdk.crypto.cryptoki, java.xml.crypto, jdk.naming.rmi, java.naming, io.github.tomdw.jpms.versions.samples.basicapplication.microphone, java.smartcardio, java.scripting, java.logging, java.security.jgss, java.prefs, jdk.crypto.ec, java.management, jdk.jdeps, jdk.compiler, java.compiler, jdk.security.auth, jdk.security.jgss, jdk.scripting.nashorn, jdk.deploy, java.management.rmi, io.github.tomdw.jpms.version.info 
------------------------------------------------------------------- 
Validating ModuleRequiresVersion[
  requiringModule=io.github.tomdw.jpms.versions.samples.basicapplication.application, 
  requiredModule=io.github.tomdw.jpms.versions.samples.basicapplication.speaker, version=1.0.0-SNAPSHOT] 
Module ModuleVersion[name=io.github.tomdw.jpms.versions.samples.basicapplication.speaker, version=1.0.0-SNAPSHOT] is compatible with required version 1.0.0-SNAPSHOT 
Validating ModuleRequiresVersion[
  requiringModule=io.github.tomdw.jpms.versions.samples.basicapplication.application, 
  requiredModule=java.base, version=9] 
Module ModuleVersion[name=java.base, version=9] is compatible with required version 9 
Validating ModuleRequiresVersion[
  requiringModule=io.github.tomdw.jpms.versions.samples.basicapplication.application, 
  requiredModule=io.github.tomdw.jpms.version.info, version=1.0.0-SNAPSHOT] 
Module ModuleVersion[name=io.github.tomdw.jpms.version.info, version=1.0.0-SNAPSHOT] is compatible with required version 1.0.0-SNAPSHOT 
Validating ModuleRequiresVersion[
  requiringModule=io.github.tomdw.jpms.versions.samples.basicapplication.speaker, 
  requiredModule=java.base, version=9] 
Module ModuleVersion[name=java.base, version=9] is compatible with required version 9 
Validating ModuleRequiresVersion[
  requiringModule=io.github.tomdw.jpms.versions.samples.basicapplication.speaker, 
  requiredModule=io.github.tomdw.jpms.versions.samples.basicapplication.microphone, version=1.0.0-SNAPSHOT] 
Module ModuleVersion[name=io.github.tomdw.jpms.versions.samples.basicapplication.microphone, version=1.1.0-SNAPSHOT] is compatible with required version 1.0.0-SNAPSHOT 
Validating ModuleRequiresVersion[
  requiringModule=io.github.tomdw.jpms.versions.samples.basicapplication.microphone, 
  requiredModule=java.base, version=9] 
Module ModuleVersion[name=java.base, version=9] is compatible with required version 9 
Validating ModuleRequiresVersion[
  requiringModule=io.github.tomdw.jpms.version.info, 
  requiredModule=java.base, version=9] 
Module ModuleVersion[name=java.base, version=9] is compatible with required version 9 Validated that modules are compatible 
------------------------------------------------------------------- 
MODULEPATH IS COMPATIBLE, STARTING APPLICATION LOGIC ... 
--------------------------------------------------------

The result is a compatible module path and the application can start.

Note: when you execute the example there are some additional WARNING lines in between WARNING adapted version 9 to 9.0.0 for semantic compatibility checking These indicate that the version of some jdk modules are rewritten from 9 to 9.0.0 to make it adhere to the semantic version path. However, given the recent decision in JEP 322: Time-Based Release Versioning it is clear that the jdk version will not be semantically versioned. How to differentiate between modules that follow semantic versioning and those that do not is not solved in this proof of concept.

An incompatiblemodulepath pom.xml is setup that allows to run the application with a speaker requiring version 1.0.0-SNAPSHOT of microphone and using a breaking version 2.0.0-SNAPSHOT of microphone on the module path. The validation output is:

------------------------------------------------------------------- 
Validating module compatibility for modules in layer 
java.naming, jdk.zipfs, java.xml, java.smartcardio, java.xml.crypto, io.github.tomdw.jpms.versions.samples.basicapplication.application, java.base, java.compiler, java.logging, jdk.naming.dns, java.scripting, jdk.localedata, jdk.charsets, jdk.crypto.ec, io.github.tomdw.jpms.versions.samples.basicapplication.microphone, jdk.scripting.nashorn, jdk.management, java.rmi, java.desktop, jdk.crypto.cryptoki, jdk.javadoc, jdk.security.auth, jdk.naming.rmi, jdk.security.jgss, jdk.unsupported, jdk.internal.opt, jdk.compiler, jdk.jlink, java.prefs, io.github.tomdw.jpms.version.info, jdk.jartool, java.security.sasl, io.github.tomdw.jpms.versions.samples.basicapplication.speaker, jdk.jdeps, java.management, java.security.jgss, java.datatransfer, jdk.deploy, java.management.rmi, jdk.dynalink 
------------------------------------------------------------------- 
Validating ModuleRequiresVersion[
  requiringModule=io.github.tomdw.jpms.versions.samples.basicapplication.application, 
  requiredModule=io.github.tomdw.jpms.versions.samples.basicapplication.speaker, version=1.0.0-SNAPSHOT] 
Module ModuleVersion[name=io.github.tomdw.jpms.versions.samples.basicapplication.speaker, version=1.0.0-SNAPSHOT] is compatible with required version 1.0.0-SNAPSHOT 
Validating ModuleRequiresVersion[
  requiringModule=io.github.tomdw.jpms.versions.samples.basicapplication.application, 
  requiredModule=java.base, version=9] 
Module ModuleVersion[name=java.base, version=9] is compatible with required version 9 
Validating ModuleRequiresVersion[
  requiringModule=io.github.tomdw.jpms.versions.samples.basicapplication.application, 
  requiredModule=io.github.tomdw.jpms.version.info, version=1.0.0-SNAPSHOT] 
Module ModuleVersion[name=io.github.tomdw.jpms.version.info, version=1.0.0-SNAPSHOT] is compatible with required version 1.0.0-SNAPSHOT 
Validating ModuleRequiresVersion[
  requiringModule=io.github.tomdw.jpms.versions.samples.basicapplication.microphone, 
  requiredModule=java.base, version=9] 
Module ModuleVersion[name=java.base, version=9] is compatible with required version 9 
Validating ModuleRequiresVersion[
  requiringModule=io.github.tomdw.jpms.version.info, 
  requiredModule=java.base, version=9] 
Module ModuleVersion[name=java.base, version=9] is compatible with required version 9 
Validating ModuleRequiresVersion[
  requiringModule=io.github.tomdw.jpms.versions.samples.basicapplication.speaker, 
  requiredModule=java.base, version=9] 
Module ModuleVersion[name=java.base, version=9] is compatible with required version 9 
Validating ModuleRequiresVersion[
  requiringModule=io.github.tomdw.jpms.versions.samples.basicapplication.speaker, 
  requiredModule=io.github.tomdw.jpms.versions.samples.basicapplication.microphone, version=1.0.0-SNAPSHOT] 
Module ModuleVersion[name=io.github.tomdw.jpms.versions.samples.basicapplication.microphone, version=2.0.0-SNAPSHOT] is NOT compatible with required version 1.0.0-SNAPSHOT 
Exception in thread "main" io.github.tomdw.jpms.version.info.internal.RequiredModuleHasIncompatibleVersionException:
 ModuleVersion[name=io.github.tomdw.jpms.versions.samples.basicapplication.microphone, version=2.0.0-SNAPSHOT] is not compatible with required version 1.0.0-SNAPSHOT
    at io.github.tomdw.jpms.version.info@1.0.0-SNAPSHOT/io.github.tomdw.jpms.version.info.internal.ModuleCompatibilityValidator.lambda$validate$0(ModuleCompatibilityValidator.java:17)
    at java.base/java.util.Optional.ifPresentOrElse(Optional.java:193)
    at io.github.tomdw.jpms.version.info@1.0.0-SNAPSHOT/io.github.tomdw.jpms.version.info.internal.ModuleCompatibilityValidator.lambda$validate$2(ModuleCompatibilityValidator.java:12)
    at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1492)
    at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:591)
    at io.github.tomdw.jpms.version.info@1.0.0-SNAPSHOT/io.github.tomdw.jpms.version.info.internal.ModuleCompatibilityValidator.lambda$validate$3(ModuleCompatibilityValidator.java:10)
    at java.base/java.lang.Iterable.forEach(Iterable.java:75) at java.base/java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1081)
    at io.github.tomdw.jpms.version.info@1.0.0-SNAPSHOT/io.github.tomdw.jpms.version.info.internal.ModuleCompatibilityValidator.validate(ModuleCompatibilityValidator.java:9)
    at io.github.tomdw.jpms.version.info@1.0.0-SNAPSHOT/io.github.tomdw.jpms.version.info.api.VersionInfo.validateCompatibility(VersionInfo.java:37)
    at io.github.tomdw.jpms.version.info@1.0.0-SNAPSHOT/io.github.tomdw.jpms.version.info.api.VersionInfo.validateCompatibility(VersionInfo.java:44)
    at io.github.tomdw.jpms.versions.samples.basicapplication.application@1.0.0-SNAPSHOT/io.github.tomdw.jpms.versions.samples.basicapplication.application.internal.Application.main(Application.java:10) 

The result is a compatibility failure which fails the application. This additional safety-net validates that the set of selected modules satisfies each module’s dependencies with a compatible version.

Wish list for the Java Module System

Although this blog post showed that we can start using versioning of modules in a useful manner, it would be even better if this could plug into the java module system itself. A small wish list for the next java module system:

• still do not solve the version selection problem
• do allow to add version information in the source code
  • to the module
  • to the requires on other modules
• make it possible to implement a VersionStrategy feature as a plugin to the java module system which
  • allows to choose which versioning strategy is most applicable for the application
  • allows to already apply these checks in tools at build time for the composed application instead of only failing at runtime
  • allows to block startup of a java application if compatibility is not guaranteed
• make it possible to annotate which versioning strategy a module uses (e.g. semver) so that validation algorithms can intelligently include and exclude modules that do not adhere to the same strategy

Conclusion

An application is always composed of a mix of different kind of modules which were compiled and packaged at different moments in time and with different versions of dependencies.

In order to validate that a set of selected modules satisfies each module’s dependencies, also the compatibility of code needs to be taken into account. Semantic versioning can be an additional safety-net here.

The version-info module available on github adds the ability to check version compatibility at startup using the bytecode support in java 9 to add versions to modules and requires clauses on other modules.

Doing this ourselves is possible as shown by this blog post. It would be better if the java module system would have first-class support for adding this version information in source code and validating compatibility using different versioning strategies.