Build a repository with direct-from-maven dependencies for which OSGi metadata needs to be generated

[merks]

[HannesWell]

That would be great.

Is currently Oomph's MavenBND.target meant to be that?

In general I think it would be more suitable to place such repo in the eclipse-orbit organization.

Btw. what do you think about always generating a special Header into wrapped artifacts like?

Eclipse-Wrapped-Bundle: ${mvnGroupId}.${mvnArtifactId}:${version}

[merks]

Do you see Eclipse-Wrapped-Bundle: ${mvnGroupId}.${mvnArtifactId}:${version} as just like a comment or might it be used in some semantic way, e.g., to specify a required capability for example?

[merks]

@HannesWell

There's a new version of Lucene for wrapping so I'm wondering about this header's purpose?

Bundle-Name: Bundle org.apache.lucene : lucene-analysis-common
Bundle-SymbolicName: org.apache.lucene.lucene-analysis-common
Bundle-Version: 9.6.0
Eclipse-Wrapped-Bundle: org.apache.lucene.lucene-analysis-common:9.6.0

It contains the same information as the Bundle-SymbolicName and Bundle-Version. Is that idea that multiple different-mapped jars could be tracked back to their original Maven coordinates? I guess an artifact ID cannot contains a '.' such that one could always determine the original Maven coordinates. If it can contain a '.', wouldn't it be better to use a ':' to separate the group and the artifact?

[merks]

@HannesWell

No one has complained about what's generated but this surely looks very wrong:

That's not a valid package name.

I wonder what the services are for?

This one has services and versions:

But I don't know how those things end up on the classpath properly.

I don't get a warm fuzzy feeling that these are actually correct, yet the Mylyn folks seems happy with them...

[HannesWell]

Do you see Eclipse-Wrapped-Bundle: ${mvnGroupId}.${mvnArtifactId}:${version} as just like a comment or might it be used in some semantic way, e.g., to specify a required capability for example?

It contains the same information as the Bundle-SymbolicName and Bundle-Version. Is that idea that multiple different-mapped jars could be tracked back to their original Maven coordinates? I guess an artifact ID cannot contains a '.' such that one could always determine the original Maven coordinates. If it can contain a '.', wouldn't it be better to use a ':' to separate the group and the artifact?

Sorry I somehow forgot half of my comment.
The idea was that M2E respectively Tycho always generate such header regardless of the specified instructions, not that users would have to do it. And yes it would be better to always use a colon as separator.

Yes the idea is that you can determine reliably if a bundle was wrapped and to find the original artifact of a wrapped bundle. This could be used to avoid multiple wrappings of the same artifacts and if we want that we could implement some checks for SimRel contributions based on that.
Maybe if projects would be not permitted to contribute own wrapped bundles and must only use the managed ones, the header could be used to check that or if they can use them we could check that the wrapped bundles are not contained in features and only referenced via Import-Package so that the resolver can reduce it to one wrapping or a after-the-fact analysis can check if multiple wrappings of the same artifact made it to the SimRel.

That's not a valid package name.

I wonder what the services are for?

I have not yet looked into that in depth (I can do that tomorrow), but my first guess is that those are ServiceLoader files.

No one has complained about what's generated but this surely looks very wrong:

Yes that looks wrong, I can have a look at it tomorrow.

[merks]

The mockito-inline was actually correct, just very odd and only needed for old versions of mockito. Also I don't think lucene-queries and lucene-sandbox are actually used or needed by Mylyn...

That leaves me to compare the Orbit versions of these remaining three things with these wrapped versions. They actually have the same odd structure (services and versions). The complex recipes really makes me concerned though that the simple recipe being used might be wrong, might need to change, and at that point it will be hard to change without a version increment:

I definitely don't get a warm fuzzy feeling...

[HannesWell]

The mockito-inline was actually correct, just very odd and only needed for old versions of mockito. Also I don't think lucene-queries and lucene-sandbox are actually used or needed by Mylyn...

Yes, since Mockito 5 the inline-mock is the default and delivered as part of the main artifact. They now have even discontinued that sub-project. So yes I agree that this should just be removed. The main mockito artifact has proper OSGi metadata.

That leaves me to compare the Orbit versions of these remaining three things with these wrapped versions. They actually have the same odd structure (services and versions). The complex recipes really makes me concerned though that the simple recipe being used might be wrong, might need to change, and at that point it will be hard to change without a version increment:

I'll look into this tomorrow/today in detail (this time really, we have public holiday 😅) and check the results.

The services entries seem to contain ServiceLoader files, so we either have to add corresponding ServiceLoader Medaitor entires or squash all the jars into one module by making all jars for example fragments of lucene-core.

What I just have noticed is that org.apache.lucene.lucene-analyzers-common is now org.apache.lucene:lucene-analysis-common according to https://github.com/apache/lucene/blob/0c6e8aec67c375f9a2f934d28ac3f4ceb5f0ebb9/lucene/MIGRATE.md#rename-of-binary-artifacts-from--analyzers--to--analysis--lucene-9562

But when already investigating the lucene project, I think it would be worth trying to make it proper bundles, as requested in

• Make the Lucene jar an OSGi bundle [LUCENE-7522] apache/lucene#8573
• Make lucene/solr a OSGI bundle through Ant [LUCENE-3167] apache/lucene#4240

But from a quick look the maintainers seem to be reluctant and this project is build with Gradle (😒) .

[merks]

@ruspl-afed

FYI, I am looking to rework the recipes for all these Lucene bundles. In particular, I have modified the recipe so that they will have the same bundle symbolic names as they had in Orbit making it easier to update to them. I was trying to switch to use package imports yesterday for Mylyn and for Data Tools, but with all these different Lucene versions floating around, that is simply exercise in frustration and futility; it's great in principle but in this case, it's unworkable. So instead I want to have bundles that are an easier drop-in replacement for the Orbit bundles. I will have to switch Mylyn to use them. I am also be jar-signing these things.

I have also worked on a generator that will allow me to maintain version qualifiers for these things such that if the recipe needs to change a new qualifier version can be built that will allow updates to the new version of the artifact; more details on that when it's polished...

[merks]

Note that the build opened a whole whack of reviews now that the dash tool is properly kicking in again:

https://gitlab.eclipse.org/eclipsefdn/emo-team/iplab/-/issues

Some may be issues with the name not mapping directly to a Maven coordinate or with the version qualifier. It will need to get ironed out...

[merks]

@HannesWell

If you want to look at any of the details and try things out, all the builds and generators can be run locally from this project setup:

The Analyzer maintains these two files and there is a launcher to run it:

In particular, it maintains the SHA1 value by digesting the BND instructions (normalized and without the underlined version qualifier) along with the dependency coordinates. If the digest changes, e.g, if the instructions are changed, it updates the digest and the version qualifier. This way I can be more reassured that it's possible to build new artifacts if some important BND instruction is overlooked....

[HannesWell]

If you want to look at any of the details and try things out, all the builds and generators can be run locally from this project setup:

Yes I already wanted to ask, what you want me to help with in detail because I have the impression that you are already making good progress. :)
I'm about to try it out.

Btw. have you already found out how to handle the ServiceLoader config well, i.e. if we need ServiceLoader-Mediator entires or use fragments?

[merks]

My sense is that the current equivalent Orbit artifacts have exactly the same kind of "strange" structure and no one has complained, so I'm inclined to ignore it for the time being. I've also done all these experiments with the 9.5.0 versions even though there are 9.6.0 versions. And I added a bunch of things to deal with older Orbit artifacts that are stuck at the 7.x version...

I produced this repository with jar-signed artifacts:

https://ci.eclipse.org/oomph/job/simrel/job/simrel-maven-bnd/lastBuild/artifact/maven-bnd/site/target/repository/

I wonder if we can move the platform to use these new versions? And more importantly too, I wonder if we can get the Platform to stop including a specific version because that's a nightmare for downstream clients who want newer versions of Lucene than what the Platform is strictly requiring and redistributing...