Fixing metadata with component metadata rules

Component metadata rules are applied in the components (ComponentMetadataHandler) section of the dependencies block (DependencyHandler) of a build script or in the settings script. The rules can be defined in two different ways:

While defining rules inline as action can be convenient for experimentation, it is generally recommended to define rules as separate classes. Rules that are written as isolated classes can be annotated with @CacheableRule to cache the results of their application such that they do not need to be re-executed each time dependencies are resolved.

As can be seen in the examples above, component metadata rules are defined by implementing ComponentMetadataRule which has a single execute method receiving an instance of ComponentMetadataContext as parameter. In this example, the rule is also further configured through an ActionConfiguration. This is supported by having a constructor in your implementation of ComponentMetadataRule accepting the parameters that were configured and the services that need injecting.

Gradle enforces isolation of instances of ComponentMetadataRule. This means that all parameters must be Serializable or known Gradle types that can be isolated.

In addition, Gradle services can be injected into your ComponentMetadataRule. Because of this, the moment you have a constructor, it must be annotated with @javax.inject.Inject. A commonly required service is ObjectFactory to create instances of strongly typed value objects like a value for setting an Attribute. A service which is helpful for advanced usage of component metadata rules with custom metadata is the RepositoryResourceAccessor.

A component metadata rule can be applied to all modules — all(rule) — or to a selected module — withModule(groupAndName, rule). Usually, a rule is specifically written to enrich metadata of one specific module and hence the withModule API should be preferred.

The component metadata rules API is oriented at the features supported by Gradle Module Metadata and the dependencies API in build scripts. The main difference between writing rules and defining dependencies and artifacts in the build script is that component metadata rules, following the structure of Gradle Module Metadata, operate on variants directly. On the contrary, in build scripts you often influence the shape of multiple variants at once (e.g. an api dependency is added to the api and runtime variant of a Java library, the artifact produced by the jar task is also added to these two variants).

Variants can be addressed for modification through the following methods:

The following details of each variant can be adjusted:

There are also a few properties of the whole component that can be changed:

Depending on the format of the metadata of a module, it is mapped differently to the variant-centric representation of the metadata:

In general, if you consider using component metadata rules to adjust the metadata of a certain module, you should check first if that module was published with Gradle Module Metadata (.module file) or traditional metadata only (.pom or ivy.xml).

If a module was published with Gradle Module Metadata, the metadata is likely complete although there can still be cases where something is just plainly wrong. For these modules you should only use component metadata rules if you have clearly identified a problem with the metadata itself. If you have an issue with the dependency resolution result, you should first check if you can solve the issue by declaring dependency constraints with rich versions. In particular, if you are developing a library that you publish, you should remember that dependency constraints, in contrast to component metadata rules, are published as part of the metadata of your own library. So with dependency constraints, you automatically share the solution of dependency resolution issues with your consumers, while component metadata rules are only applied to your own build.

If a module was published with traditional metadata (.pom or ivy.xml only, no .module file) it is more likely that the metadata is incomplete as features such as variants or dependency constraints are not supported in these formats. Still, conceptually such modules can contain different variants or might have dependency constraints they just omitted (or wrongly defined as dependencies). In the next sections, we explore a number existing oss modules with such incomplete metadata and the rules for adding the missing metadata information.

As a rule of thumb, you should contemplate if the rule you are writing also works out of context of your build. That is, does the rule still produce a correct and useful result if applied in any other build that uses the module(s) it affects?

Let’s consider as an example the publication of the Jaxen XPath Engine on Maven central. The pom of version 1.1.3 declares a number of dependencies in the compile scope which are not actually needed for compilation. These have been removed in the 1.1.4 pom. Assuming that we need to work with 1.1.3 for some reason, we can fix the metadata with the following rule:

Within the withDependencies block you have access to the full list of dependencies and can use all methods available on the Java collection interface to inspect and modify that list. In addition, there are add(notation, configureAction) methods accepting the usual notations similar to declaring dependencies in the build script. Dependency constraints can be inspected and modified the same way in the withDependencyConstraints block.

If we take a closer look at the Jaxen 1.1.4 pom, we observe that the dom4j, jdom and xerces dependencies are still there but marked as optional. Optional dependencies in poms are not automatically processed by Gradle nor Maven. The reason is that they indicate that there are optional feature variants provided by the Jaxen library which require one or more of these dependencies, but the information what these features are and which dependency belongs to which is missing. Such information cannot be represented in pom files, but in Gradle Module Metadata through variants and capabilities. Hence, we can add this information in a rule as well.

Here, we first use the addVariant(name, baseVariant) method to create an additional variant, which we identify as feature variant by defining a new capability jaxen-dom4j to represent the optional dom4j integration feature of Jaxen. This works similar to defining optional feature variants in build scripts. We then use one of the add methods for adding dependencies to define which dependencies this optional feature needs.

In the build script, we can then add a dependency to the optional feature and Gradle will use the enriched metadata to discover the correct transitive dependencies.

While in the previous example, all variants, "main variants" and optional features, were packaged in one jar file, it is common to publish certain variants as separate files. In particular, when the variants are mutual exclusive — i.e. they are not feature variants, but different variants offering alternative choices. One example all pom-based libraries already have are the runtime and compile variants, where Gradle can choose only one depending on the task at hand. Another of such alternatives discovered often in the Java ecosystems are jars targeting different Java versions.

As example, we look at version 0.7.9 of the asynchronous programming library Quasar published on Maven central. If we inspect the directory listing, we discover that a quasar-core-0.7.9-jdk8.jar was published, in addition to quasar-core-0.7.9.jar. Publishing additional jars with a classifier (here jdk8) is common practice in maven repositories. And while both Maven and Gradle allow you to reference such jars by classifier, they are not mentioned at all in the metadata. Thus, there is no information that these jars exist and if there are any other differences, like different dependencies, between the variants represented by such jars.

In Gradle Module Metadata, this variant information would be present and for the already published Quasar library, we can add it using the following rule:

In this case, it is pretty clear that the classifier stands for a target Java version, which is a known Java ecosystem attribute. Because we also need both a compile and runtime for Java 8, we create two new variants but use the existing compile and runtime variants as base. This way, all other Java ecosystem attributes are already set correctly and all dependencies are carried over. Then we set the TARGET_JVM_VERSION_ATTRIBUTE to 8 for both variants, remove any existing file from the new variants with removeAllFiles(), and add the jdk8 jar file with addFile(). The removeAllFiles() is needed, because the reference to the main jar quasar-core-0.7.5.jar is copied from the corresponding base variant.

We also enrich the existing compile and runtime variants with the information that they target Java 7 — attribute(TARGET_JVM_VERSION_ATTRIBUTE, 7).

Now, we can request a Java 8 versions for all of our dependencies on the compile classpath in the build script and Gradle will automatically select the best fitting variant for each library. In the case of Quasar this will now be the jdk8Compile variant exposing the quasar-core-0.7.9-jdk8.jar.

Another solution to publish multiple alternatives for the same library is the usage of a versioning pattern as done by the popular Guava library. Here, each new version is published twice by appending the classifier to the version instead of the jar artifact. In the case of Guava 28 for example, we can find a 28.0-jre (Java 8) and 28.0-android (Java 6) version on Maven central. The advantage of using this pattern when working only with pom metadata is that both variants are discoverable through the version. The disadvantage is that there is no information what the different version suffixes mean semantically. So in the case of conflict, Gradle would just pick the highest version when comparing the version strings.

Turning this into proper variants is a bit more tricky, as Gradle first selects a version of a module and then selects the best fitting variant. So the concept that variants are encoded as versions is not supported directly. However, since both variants are always published together we can assume that the files are physically located in the same repository. And since they are published with Maven repository conventions, we know the location of each file if we know module name and version. We can write the following rule:

Similar to the previous example, we add runtime and compile variants for both Java versions. In the withFiles block however, we now also specify a relative path for the corresponding jar file which allows Gradle to find the file no matter if it has selected a -jre or -android version. The path is always relative to the location of the metadata (in this case pom) file of the selection module version. So with this rules, both Guava 28 "versions" carry both the jdk6 and jdk8 variants. So it does not matter to which one Gradle resolves. The variant, and with it the correct jar file, is determined based on the requested TARGET_JVM_VERSION_ATTRIBUTE value.

Jars with classifiers are also used to separate parts of a library for which multiple alternatives exists, for example native code, from the main artifact. This is for example done by the Lightweight Java Game Library (LWGJ), which publishes several platform specific jars to Maven central from which always one is needed, in addition to the main jar, at runtime. It is not possible to convey this information in pom metadata as there is no concept of putting multiple artifacts in relation through the metadata. In Gradle Module Metadata, each variant can have arbitrary many files and we can leverage that by writing the following rule:

This rule is quite similar to the Quasar library example above. Only this time we have five different runtime variants we add and nothing we need to change for the compile variant. The runtime variants are all based on the existing runtime variant and we do not change any existing information. All Java ecosystem attributes, the dependencies and the main jar file stay part of each of the runtime variants. We only set the additional attributes OPERATING_SYSTEM_ATTRIBUTE and ARCHITECTURE_ATTRIBUTE which are defined as part of Gradle’s native support. And we add the corresponding native jar file so that each runtime variant now carries two files: the main jar and the native jar.

In the build script, we can now request a specific variant and Gradle will fail with a selection error if more information is needed to make a decision.

Gradle is able to understand the common case where a single attribute is missing that would have removed the ambiguity. In this case, rather than listing information about all attributes on all available variants, Gradle helpfully lists only possible values for that attribute along with the variants each value would select.

Because it is difficult to model optional feature variants as separate jars with pom metadata, libraries sometimes compose different jars with a different feature set. That is, instead of composing your flavor of the library from different feature variants, you select one of the pre-composed variants (offering everything in one jar). One such library is the well-known dependency injection framework Guice, published on Maven central, which offers a complete flavor (the main jar) and a reduced variant without aspect-oriented programming support (guice-4.2.2-no_aop.jar). That second variant with a classifier is not mentioned in the pom metadata. With the following rule, we create compile and runtime variants based on that file and make it selectable through a capability named com.google.inject:guice-no_aop.

The new variants also have the dependency on the standardized aop interfaces library aopalliance:aopalliance removed, as this is clearly not needed by these variants. Again, this is information that cannot be expressed in pom metadata. We can now select a guice-no_aop variant and will get the correct jar file and the correct dependencies.

Another usage of capabilities is to express that two different modules, for example log4j and log4j-over-slf4j, provide alternative implementations of the same thing. By declaring that both provide the same capability, Gradle only accepts one of them in a dependency graph. This example, and how it can be tackled with a component metadata rule, is described in detail in the feature modelling section.

Modules with Ivy metadata, do not have variants by default. However, Ivy configurations can be mapped to variants as the addVariant(name, baseVariantOrConfiguration) accepts any Ivy configuration that was published as base. This can be used, for example, to define runtime and compile variants. An example of a corresponding rule can be found here. Ivy details of Ivy configurations (e.g. dependencies and files) can also be modified using the withVariant(configurationName) API. However, modifying attributes or capabilities on Ivy configurations has no effect.

For very Ivy specific use cases, the component metadata rules API also offers access to other details only found in Ivy metadata. These are available through the IvyModuleDescriptor interface and can be accessed using getDescriptor(IvyModuleDescriptor) on the ComponentMetadataContext.

For Maven specific use cases, the component metadata rules API also offers access to other details only found in POM metadata. These are available through the PomModuleDescriptor interface and can be accessed using getDescriptor(PomModuleDescriptor) on the ComponentMetadataContext.

While all the examples above made modifications to variants of a component, there is also a limited set of modifications that can be done to the metadata of the component itself. This information can influence the version selection process for a module during dependency resolution, which is performed before one or multiple variants of a component are selected.

The first API available on the component is belongsTo() to create virtual platforms for aligning versions of multiple modules without Gradle Module Metadata. It is explained in detail in the section on aligning versions of modules not published with Gradle.

Gradle and Gradle Module Metadata also allow attributes to be set on the whole component instead of a single variant. Each of these attributes carries special semantics as they influence version selection which is done before variant selection. While variant selection can handle any custom attribute, version selection only considers attributes for which specific semantics are implemented. At the moment, the only attribute with meaning here is org.gradle.status. It is therefore recommended to only modify this attribute, if any, on the component level. A dedicated API setStatus(value) is available for this. To modify another attribute for all variants of a component withAllVariants { attributes {} } should be utilised instead.

A module’s status is taken into consideration when a latest version selector is resolved. Specifically, latest.someStatus will resolve to the highest module version that has status someStatus or a more mature status. For example, latest.integration will select the highest module version regardless of its status (because integration is the least mature status as explained below), whereas latest.release will select the highest module version with status release.

The interpretation of the status can be influenced by changing a module’s status scheme through the setStatusScheme(valueList) API. This concept models the different levels of maturity that a module transitions through over time with different publications. The default status scheme, ordered from least to most mature status, is integration, milestone, release. The org.gradle.status attribute must be set, to one of the values in the components status scheme. Thus each component always has a status which is determined from the metadata as follows:

The following example demonstrates latest selectors based on a custom status scheme declared in a component metadata rule that applies to all modules:

Compared to the default scheme, the rule inserts a new status rc and replaces integration with nightly. Existing modules with the state integration are mapped to nightly.