MVP POC of a working MavenResolver in Enola

[vorburger]

As part of, and the basics in order to later be able to do further work around what I envision for #726,

would be to have a sort of MVP POC of a working MavenResolver kind of utility class that works within Enola.

So this would be something which (functionally) given an (input) "GAV" like say ch.vorburger.mariaDB4j:mariaDB4j-core:3.1.0 (or whatever), and (a list of) repository URL/s (say https://repo1.maven.org/maven2, but could also be e.g. https://jitpack.io, or whatever private repos, etc.), "fetch" to POM from say https://repo1.maven.org/maven2/ch/vorburger/mariaDB4j/mariaDB4j-core/3.1.0/, and turn that into an (output) Java object Artifact (or whatever it's called) that represents a "resolved POM", where one can then grab the <dependencies>, and what not.

I have never actually done this myself (only used tools which did this for me). I couldn't easily figure out how to this, so far. My current understanding is that there are at least 2 libraries which we could use to do this kind of thing - the https://maven.apache.org/resolver/ (which is https://github.com/apache/maven-resolver) and the https://get-coursier.io/docs/api ? Are there even more OTHER libraries for doing this kind of thing? Which of those 2 is "better" - why would I choose one or the other?

https://github.com/apache/maven-resolver/tree/master/maven-resolver-demos/maven-resolver-demo-snippets/src/main/java/org/apache/maven/resolver/examples smells like it may have what I'm looking for, somewhere? But it felt a bit "overwhelming", from a first look...

@cstamas just because we're "talking" over in #1040, and technically totally unrelated to that, may I "pick your brain" about this here? It's completely unrelated to that other issue - and I'm only tagging you here just because I gather that you are into this sort of thing... perhaps you could provide me a pointer how to get started with doing this? (And comment on the Apache Maven Resolver, vs Coursier, vs others if any; although I have a hunch I know which one you will recommend - but I'm interesting in learning a bit more background, if you would like to comment on that.)

[cstamas]

Yes, you talk about "transitively resolve" stuff. That is done by Maven Resolver.

Sure, but using Resolver "alone" is very laborious and error prone, hence, I'd recommend:

• use MIMA to bootstrap Resolver https://github.com/maveniverse/mima (btw, JBang uses MIMA as well) as it gives you Resolver with all the whistle and bells out of the box
• look into Toolbox https://github.com/maveniverse/toolbox as it's "shared" module is in fact reusable, or just "grab" some examples from it, reason of Toolbox is really to showcase MIMA + Resolver + "common (and less common) use cases".

Am here to help, if anything needed.

[vorburger]

@cstamas Thank You! I started to have a bit of a look, so:

The https://github.com/maveniverse/toolbox/tree/main/shared/src/main/java/eu/maveniverse/maven/toolbox/shared is overwhelming me a little bit, so far; I'm not sure that all that is, and if I need it.

But https://github.com/maveniverse/mima/blob/main/demo/library/src/main/java/eu/maveniverse/maven/mima/impl/library/Classpath.java with https://github.com/maveniverse/mima/blob/main/demo/library-standalone-static/src/test/java/eu/maveniverse/maven/mima/impl/library/ClasspathTest.java might be close enough to what I need, here.

What I don't quite understand just yet is there the resolved POM Object Model (?) is... I (at least initially) don't so much want a Classpath, and I don't really care much for the String model in XML, but, given e.g. ch.vorburger.mariaDB4j:mariaDB4j-core:3.1.0, "just" the (correctly resolved, e.g. <properties> replaced, <parent> inherited, etc.) <dependencies> (just as "GAVs", so that I can transitively fetch them as well) as well as some "other Model metadata" - like I would perhaps want the (e.g. say) <license> of that artifact (which in its case isn't in that POM, but <parent> inherited), etc. How would I do that?

I guess I need to play with this a bit to learn by exploration.

[cstamas]

For that you need:

• MIMA extension MMR https://github.com/maveniverse/mima?tab=readme-ov-file#extensions
• example of use is here: https://github.com/maveniverse/toolbox/blob/main/shared/src/main/java/eu/maveniverse/maven/toolbox/shared/internal/ToolboxResolverImpl.java#L115-L121

The resulting object:
https://github.com/maveniverse/mima/blob/main/extensions/mmr/src/main/java/eu/maveniverse/maven/mima/extensions/mmr/ModelResponse.java

Hold "effective model", that is what you need.

One remark: this "effective model" != "maven effective model", as one step is missing (and makes no sense in "Mini Maven" or "Maven without Maven"), the build plugin injections. But yes, properties are interpolated, inheritance applied and so on...

[vorburger]

One remark: this "effective model" != "maven effective model", as one step is missing (and makes no sense in "Mini Maven" or "Maven without Maven"), the build plugin injections. But yes, properties are interpolated, inheritance applied and so on...

That's totally fine, as (in this application, what I'm after here) I'm interested in the "metadata" and not "how it was built it". Said differently, based on what I've found from your Social Media (thanks for posting that), here I'm only after what Slide 4 on https://gnodet.github.io/maven4-presentation/ seems to now call the Consumer POM and not the Build POM.

I'll play with it, but am wondering if I you would recommend that one use the 2.4.21 or the 3.0.0-alpha-3 already? (BTW where are the sources for the 3.0.0? https://github.com/maveniverse/mima's main is 2.4.22-SNAPSHOT, and I don't see a 3.x.y on https://github.com/maveniverse/mima/branches/all, so just wondering.)

[cstamas]

Stick with MIMA 2.4.21, is on Central and is "stable". MIMA 3.x was an attempt for "parallel version stream" to introduce Maven4 support, but I am not maintaining it currently.

In short: MIMA 2.x => Maven 3.9.x, MIMA 3.x => Maven 4.x... (but in umaintained, is not updated maveniverse/mima#28)

[cstamas]

And don't worry about Resolver 1.x vs Resolver 2.x, as if you do all "cleanly" by the book (see [1] and [2]), you will be in "safe place" and switching from 1.x to 2.x will be breeeze, we ensure compatibility and MIMA will handle the rest.

[1] https://maven.apache.org/resolver/api-compatibility.html
[2] https://maven.apache.org/resolver/upgrading-resolver.html

[vorburger]

And don't worry about Resolver 1.x vs Resolver 2.x, as if you do all "cleanly" by the book (see [1] and [2]), you will be in "safe place" and switching from 1.x to 2.x will be breeeze, we ensure compatibility and MIMA will handle the rest.

Sorry I'm probably lacking some required context here to fully comprehend what this mean, but TL;DR should I still be using org.apache.maven.resolver:maven-resolver-api 1.9.22 like here, or already go for 2.0.6? Newer sounds better... 😺 if it already just works?

[cstamas]

Use MIMA 2.x (2.4.21) and Resolver 1.x (1.9.22) for now.

[cstamas]

Just to explain: Maven3 (3.9.9) uses Resolver 1.x (1.9.22) and MIMA 2.x (2.4.21) is "Maven 3 aligned".
Maven 4 (only RC yet) uses Resolver 2.x (2.0.6) and MIMA 3.x will be "Maven 4 aligned".

But for your use case, inspecting effective models, there is no functional difference between the two.

[vorburger]

@cstamas just FYI, it (well, very basics) actually seems to work!

See https://github.com/enola-dev/enola/pull/1065/files.

Not that I doubted it / you, of course. Thank You for getting me this far!

Now I'll try to actually do more with it.

[cstamas]

Coolio!

[cstamas]

Use toolbox shared module for "patterns" but also take a peek at simpler Resolver examples:
https://github.com/apache/maven-resolver/tree/master/maven-resolver-demos/maven-resolver-demo-snippets/src/main/java/org/apache/maven/resolver/examples

[vorburger]

@cstamas something I'd like to "record" (in the context of the #726 that this is for, ultimately), is the "origin" where a POM Model "came from". Let's say I add 2 repositories. Then the artifact, and its dependencies, could have come from either of the two. Which API gives me (the URL of) the repo an artifact was actually found on?

[cstamas]

Use this method:
https://github.com/maveniverse/mima/blob/main/extensions/mmr/src/main/java/eu/maveniverse/maven/mima/extensions/mmr/ModelResponse.java#L69
And it gives you ArtifactDescriptorReader and:
https://github.com/apache/maven-resolver/blob/maven-resolver-1.9.x/maven-resolver-api/src/main/java/org/eclipse/aether/resolution/ArtifactDescriptorResult.java#L232-L239

[vorburger]

Use this method: https://github.com/maveniverse/mima/blob/main/extensions/mmr/src/main/java/eu/maveniverse/maven/mima/extensions/mmr/ModelResponse.java#L69 And it gives you ArtifactDescriptorReader and: https://github.com/apache/maven-resolver/blob/maven-resolver-1.9.x/maven-resolver-api/src/main/java/org/eclipse/aether/resolution/ArtifactDescriptorResult.java#L232-L239

The ArtifactRepository appears to be a LocalRepository if the artifact is already locally present. I would want to find "the RemoteRepository that it originally came from" to be able to getUrl().

Also, and more importantly, if I access anything that is not already locally present, then it does not actually quite work just yet, and fails due:

org.eclipse.aether.resolution.ArtifactResolutionException: The following artifacts could not be resolved: 
ch.vorburger.mariaDB4j:mariaDB4j-core:pom:3.1.0 (absent): Could not transfer artifact
ch.vorburger.mariaDB4j:mariaDB4j-core:pom:3.1.0 from/to central (https://repo.maven.apache.org/maven2/): 
/home/vorburger/.m2/repository/ch/vorburger/mariaDB4j: Read-only file system

I'm guessing that I'm missing something to permit it to write into ~/.m2 ?

[vorburger]

Ah, no... ignore the Read-only file system thing... that has something to do with Bazel's "Sandbox"; I'll fix.

But re. the (real, original; not local) "origin" I would still be interested if you have suggestions.

[cstamas]

Hm, that is strange, when I debug the MMR UT, I do get "central" (and POM is in my local repo):
https://github.com/maveniverse/mima/blob/main/extensions/mmr/src/test/java/eu/maveniverse/maven/mima/extensions/mmr/MavenModelReaderTest.java#L56

[vorburger]

Hm, that is strange, when I debug the MMR UT, I do get "central" (and POM is in my local repo):

Sorry, it seem to have misinterpreted it; and with this convinced myself that it does appear to work perfectly!

[vorburger]

@cstamas thanks to your guidance, this is coming nicely together... I'll soon merge https://github.com/enola-dev/enola/pull/1065/files.

[vorburger]

With #1065 and #1067 now merged I actually consider this Done, for now; and I'm closing this already.

#1068 is to follow-up re. maveniverse/mima#167.

#726 overall is the main next step.