Add support for target_compatible_with to maven_export

[AustinSchuh]

It is helpful, expecially when exporting things with shared libraries, to only export on specific architectures. The canonical way to do this is target_compatible_with. Plumb that through maven_export.

[shs96c]

I'm not sure what the use case is here? This implies that the java_export will be different per platform, which breaks the promise of maven coordinates. The more normal thing I've seen in Maven projects (Netty is a good example) is for the exported artifact to have dependencies on every possible platform, and then at runtime the appropriate dependency is used.

[AustinSchuh]

Hi Simon,

I'm trying to move the build for https://github.com/wpilibsuite/allwpilib over from Gradle to Bazel. The hard part is the JNI libraries, and the C++ libraries which get exported as classifier artifacts, one per (os, architecture, dbg/opt, shared/static library) combination. (24 ish different artifacts.) The recent changes you helped review enable exporting the zip files we are using for those shared libraries. (thanks!!!) The real challenge is that I need to match the current set of artifacts in Maven for a dual build leading to a graceful transition.

In Gradle, each of the artifacts is done with a separate build on the relevant os/architecture, and then a script combines them all together before uploading to the maven repository.

With Bazel, I've got configuration transitions working such that I'm able to build all the combinations from 1 Windows platform and bazel execution, 1 Linux platform, and 1 OSX platform. Due to licencing challenges, from what I can tell it is not feasible to cross compile for OSX, and cross compilation from Linux to Windows is rough. This gets me down to 3 separate bazel builds, 1 per OS, to build all the artifacts. I then use the same combination script to combine everything and upload the whole set.

The other alternative I see for managing this would be to make the classifier artifacts configurable, and then to pick the artifacts that are built and upload those. That looks challenging from my reading of the starlark.

If you are interested in seeing the change I'm working through, https://github.com/wpilibsuite/allwpilib/pull/8049/files#diff-b549bfb1d321f0fe05834f6d90de44f13779edaff6c29a952ce2371020661191 has the relevant bits. There is a good chunk of the change fixing toolchain bugs and missing for cross compilation that isn't very relevant.

There's another use case I've hit in the past (not in this repo right now) when microcontroller code gets added to a monorepo. The bigger languages (java, python, typescript, etc) don't work for the microcontroller, and all targets relevant to those languages get marked as requiring an OS through target_compatible_with. That keeps bazel test //... working for all the various targets.

If you see another route through all this, I'm all ears. This felt the most in line with Bazel's tools and the way the rest of the features mix and match.

[shs96c]

So if I understand you correctly, you can't build everything on the same machine as there's no way to use a transition to do what you want, and you've not got access to an remote build environment that can do that for you. So you'd like to build some artifacts locally and upload them from (say Windows) and then others from (for example) macOS.

I think there are two kinds of users you want to care about here:

1. Someone in the same repo, who needs to have their dependencies be correct
2. Third party users who are downloading the coordinates needed into their own apps, who need everything combined

The most correct thing to do would be to have a single java_export target that looks something like:

java_export(
    name = "example",
    maven_coordinates = "...",
    classifier_artifacts = {
        "so": "//linux:artifact",
        "dll": "//windows:artifact",
    },
)

However, given your constraints, that's not possible.

For users in the same repo, you can use a java_library with two runtime_deps, one being the unified java_export, and then other being a select which pulls in the per-platform classifier artifacts. As long as you consistently use that within the repo everything will Just Work.

For third party users who are downloading coordinates, why not use the pom_template attribute of java_export? The default template is here, but you can supply your own. Provided everything has the same version (which I assume is true), you can add anything you like to the dependencies block, so could spell out the classifier deps clearly there.

[AustinSchuh]

--- snip ---

However, given your constraints, that's not possible.

Yup, you nailed it. Trust me, I wish I could just build everything in 1 spot.

For users in the same repo, you can use a java_library with two runtime_deps, one being the unified java_export, and then other being a select which pulls in the per-platform classifier artifacts. As long as you consistently use that within the repo everything will Just Work.

What I've got right now is (in a macro):

maven_export(
    name = name + "-linux",
    maven_coordinates = maven_coordinates,
    classifier_artifacts = all_linux_artifacts,
    target_compatible_with = ["@platforms//os:linux"],
    **common_args
)

maven_export(
    name = name + "-osx",
    maven_coordinates = maven_coordinates,
    classifier_artifacts = all_osx_artifacts,
    target_compatible_with = ["@platforms//os:osx"],
    **common_args
)

maven_export(
    name = name + "-windows",
    maven_coordinates = maven_coordinates,
    classifier_artifacts = all_windows_artifacts,
    target_compatible_with = ["@platforms//os:windows"],
    **common_args
 )

alias(
    name = name + ".publish",
    actual = select({
        "@platforms//os:linux": name + "-linux.publish",
        "@platforms//os:osx": name + "-osx.publish",
        "@platforms//os:windows": name + "-windows.publish",
    }),
    visibility = visibility,
)

This is for an artifact which has no java library dependencies to export (See #1368), just zip files.

Unfortunately, I think that means runtime_deps isn't available as a solution :(

Thinking out loud, maybe it is worth making classifier_artifacts configurable? Shouldn't be too hard. I have gotten a request from some users to be able to build only one of the configurations when iterating locally to safe development time. --define=only_native=true and a select on the way into the classifier_artifacts could work.

Thanks for taking the time to think through this!