Skip to content

Enable resolving multiple architectures of the same package#175

Open
aszady wants to merge 2 commits into
rmohr:mainfrom
aszady:multipkgarch
Open

Enable resolving multiple architectures of the same package#175
aszady wants to merge 2 commits into
rmohr:mainfrom
aszady:multipkgarch

Conversation

@aszady
Copy link
Copy Markdown
Contributor

@aszady aszady commented Jan 14, 2026

A series of small changes to finally implement / enable #166.
This stems quite naturally from all the preparatory work.

Let Id contain package arch

After setting up this function for identifying unique files (#172, #173) we can finally incorporate package arch here.

This change affects the repository naming, but users should consider it implementation detail and use alias repository to refer to RPMs, so this change should be transparent to proper users.

Enable resolving multiple architectures of the same package

After extensive preparations (see e.g. changes referencing multi-arch proposal #166),
we can simply remove the conflict.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jan 14, 2026

⚠️ Optional job e2e-bzlmod-toolchain-circular-dependencies failed ⚠️

  • exit status: 1

@aszady aszady force-pushed the multipkgarch branch 5 times, most recently from bd5b2fa to d21d63e Compare January 15, 2026 00:06
@aszady aszady changed the title Multipkgarch Enable resolving multiple architectures of the same package Jan 15, 2026
@aszady aszady force-pushed the multipkgarch branch 2 times, most recently from 059d437 to fdd563d Compare January 28, 2026 16:55
@aszady aszady marked this pull request as ready for review January 28, 2026 17:04
Comment thread e2e/bazel-bzlmod-multiarch/MODULE.bazel Outdated
@aszady aszady force-pushed the multipkgarch branch 4 times, most recently from ab528aa to 798d43b Compare January 29, 2026 21:03
@aszady
Copy link
Copy Markdown
Contributor Author

aszady commented Jan 29, 2026

  • fixed a typo in actions, so that the new e2e test is actually run
  • fixed the new e2e test to run on the fresh copy of bazeldnf binary, not prebuilt one
  • fixed the determinism when constructing the lockfile (added Sort packages in config also by arch commit)

@aszady aszady requested a review from manuelnaranjo January 29, 2026 21:15
Comment thread bazeldnf/alias_macros.bzl Outdated
if len(rpms) > 1:
fail("Package resolved multiple times, not implemented.")
for rpm in rpms:
if "arch" not in rpm or "package" not in rpm:
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why shouldn't this be a failure case instead of a skip case?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is indeed an issue, and I have a valid use case when the len(rpms) will be > 1. With this commit I'm fixing it cf97632 in the circular dependencies branch.
The failure becomes relevant with the e2e/bazel-bzlmod-lock-file which is a totally valid case of 2 lock files mentioning the same file: libvirt-libs-11.0.0-1.fc42.x86_64.rpm. If we would like to do any kind of deduplication detection we should detect 2 entries in the lock files with same name and different checksum, which wouldn't be "legal" in a rpm repository. But the use case of multiple lock files mentioning the same file is totally valid. All of those should resolve to the same entry though.
Don't forget that given how bzlmod works it's not multiple times that our module gets initialized but 1 time with all the calls together.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kellyma2, bothrpm["arch"] and rpm["package"] are marked as optional, therefore it would be improper to interrupt in this case. An example of simpler lockfile (with only urls and sometimes name – that get translated here to just rpm["id"]) was provided by @manuelnaranjo.

@manuelnaranjo, I don't understand the concern. Lockfiles are processed separately and the only interference is when there is an rpm_repository to be created with the same repo name – in which case arbitrary one is used for this conflict (ignoring lockfile content).
Otherwise, lock files are handled separately, each one has its own packages_metadata collected and passed to the _alias_repository, that is the source of rpms argument to the alias macro.

Nevertheless, inspired by your comments, I'm sending for review an updated version that should handle a bit more gracefully some corner cases that may appear in manually crafted lockfiles.

Comment thread bazeldnf/alias_macros.bzl
@aszady aszady force-pushed the multipkgarch branch 2 times, most recently from 9acad12 to 41203f8 Compare February 5, 2026 17:02
@manuelnaranjo
Copy link
Copy Markdown
Collaborator

I've been thinking and researching about this topic, and turns out we can extract the architecture from the file name.

I think we need to change entirely how we deal with this, the blob it self should be a single entity based on the file name no matter how many lock files or strict rpms we get passed, the name of the rpm_repository instance should be as much as possible the name of the file, we will have to do some mangling because some characters are not allowed, but not a major thing. This approach will not only allow for multiple archs of the same package but also for multiple versions across all of the bazeldnf instances of a given bazel module. I would say per lock file we should have only 1 version for a given package, I would even challenge if multiple archs make sense. What do you folks think? I have the code for this I just didn´t finish integrating it.

Then we should have entities that collect the dependencies and binds the dependency tree together, and finally aliases for those things we've been requested to make public. This is basically what my circular dependencies fix branch does btw

aszady added 2 commits March 12, 2026 21:02
After setting up this function for identifying unique files (rmohr#172, rmohr#173) we can finally incorporate package arch here.

This change affects the repository naming, but users should consider it implementation detail and use alias repository to refer to RPMs, so this change should be transparent to proper users.
After extensive preparations (see e.g. changes referencing multi-arch proposal rmohr#166),
we can simply remove the conflict.
@aszady
Copy link
Copy Markdown
Contributor Author

aszady commented Mar 12, 2026

I removed some commits from this PR so that we can discuss them separately:

  • whether to add separate arch field to the lockfile, or parse the filename
  • how the aliases should be constructed – I really don't have any strong opinion here

The part that remained is the dependency resolution that will allow creating multi-arch lock file – which is a valuable artifact by itself.
Now, while I agree that some typical multi-arch usages will be fine with combination of multiple lock file generations, I need to challenge the constraint of max one architecture of a package per lock file. Sufficiently complex cross-arch dependency structure (e.g. #166 (comment)) might render the resolution impossible, what this change tries to address.

@aszady aszady requested a review from manuelnaranjo March 12, 2026 20:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants