Add API to automatically generate matching EROFS given a digest by cgwalters · Pull Request #367 · composefs/composefs-rs

cgwalters · 2026-07-28T16:13:50Z

This is intending to address bootc-dev/bootc#2334

Copilot

Pull request overview

This PR adds support for recovering (and caching) a bootable EROFS image that matches a known expected digest by searching across EROFS format versions and xattr-filtering modes, addressing bootc-dev/bootc#2334’s “find the original digest despite changed defaults / repo-fixed format config” scenario.

Changes:

Introduce XattrFiltering + OciTransformOptions and thread these options through OCI filesystem construction/transform paths.
Cache boot images per (format version, xattr filtering mode) and add find_matching_boot_image() to search/generate/persist the matching artifact for an expected digest.
Add an explicit commit_image_version() API and define FormatVersion::BOOT_VERSIONS to support the search space.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
crates/composefs/src/generic_tree.rs	Adds xattr filtering mode enum/options and applies it in `transform_for_oci()`, plus unit tests.
crates/composefs/src/fs.rs	Threads `OciTransformOptions` into `read_container_root()`.
crates/composefs/src/filesystem_ops.rs	Adds `commit_image_version()` to commit non-default EROFS format versions.
crates/composefs/src/erofs/format.rs	Adds `FormatVersion::BOOT_VERSIONS` constant for boot-relevant versions.
crates/composefs/Cargo.toml	Adds `strum` dependency for enum parsing/display/variant enumeration.
crates/composefs-oci/src/test_util.rs	Updates helpers to pass OCI transform options through.
crates/composefs-oci/src/oci_image.rs	Stores boot image refs as a map keyed by named-ref key; adds mode-aware accessors.
crates/composefs-oci/src/lib.rs	Introduces per-mode boot ref keys, splits boot refs from layer refs, and updates config read/write APIs accordingly.
crates/composefs-oci/src/image.rs	Threads `OciTransformOptions` into `create_filesystem()` and expands tests for mode behavior.
crates/composefs-oci/src/delta.rs	Updates config write call site for new boot-images-map parameter.
crates/composefs-oci/src/boot.rs	Adds `find_matching_boot_image()` and mode-aware boot image generation/caching semantics + tests.
crates/composefs-oci/Cargo.toml	Adds optional `strum` dep under the `boot` feature.
crates/composefs-integration-tests/src/tests/varlink.rs	Updates varlink pull call signature to include new parameters.
crates/composefs-integration-tests/src/tests/cli.rs	Adds integration coverage for `--xattrs keep-user-xattrs` and expected-digest recovery behavior.
crates/composefs-ctl/src/varlink.rs	Extends varlink API: adds xattrs/expected_digest handling, validation, and richer boot-image reporting.
crates/composefs-ctl/src/lib.rs	Adds CLI flags for xattr filtering and expected-digest boot-image recovery flow.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+/// Error parsing a [`XattrFiltering`] value from a string.
+#[derive(Debug, Clone, PartialEq, Eq, Error)]
+#[error("invalid xattr filtering mode {0:?} (expected \"allowlist-only\" or \"keep-user-xattrs\")")]
+pub struct XattrFilteringParseError(String);
+
+impl XattrFilteringParseError {
+    fn not_found(s: &str) -> Self {
+        Self(s.to_owned())
+    }
+}


+    pub fn boot_image_ref_for_mode(
+        &self,
+        version: FormatVersion,
+        mode: XattrFiltering,
+    ) -> Option<&ObjectID> {
+        self.boot_image_refs
+            .get(crate::boot_image_ref_key(version, mode).as_str())
+    }


    let oc = open_config(repo, config_digest, verity)?;
-    Ok(match version.epoch() {
-        FormatEpoch::Epoch1 => oc.boot_image_ref_v1,
-        FormatEpoch::Epoch2 => oc.boot_image_ref,
-    })
+    Ok(oc
+        .boot_image_refs
+        .get(&*boot_image_ref_key(version, mode))
+        .cloned())


Johan-Liebert1

Overall look okay, but a few comments

Johan-Liebert1 · 2026-07-29T04:49:53Z

            b"\x02\x00\x00\x02\x00\x20\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00";
-        assert_eq!(cap_val.as_ref(), expected_cap);
+
+        for (mode, keep_user_xattr) in [


Missing two combinations here, is that intentional?

Historically, transform_for_oci() unconditionally stripped every xattr except a fixed container allowlist, discarding legitimate `user.*` xattrs along with host-leaked ones (like build-time security.selinux). This can cause composefs digest mismatches when the allowlist or filtering logic changes across composefs-rs versions, since images built with different versions can end up with different content even though the source layers didn't change. Add composefs::generic_tree::XattrFiltering, an enum selecting between the historical AllowlistOnly behavior and a new KeepUserXattrs mode that additionally preserves `user.*` xattrs. It derives strum's Display/EnumString (kebab-case: allowlist-only, keep-user-xattrs) so it works directly as a clap value, and VariantArray so callers can iterate every mode without a hand-maintained list; behind the varlink feature it also derives Serialize/Deserialize for use as a wire parameter. transform_for_oci() and the OCI filesystem/boot-image builders now take an explicit &OciTransformOptions carrying the mode, and cfsctl gains a --xattrs flag (on the OCI-to-filesystem and compute-id paths) so callers can pick a mode instead of being stuck with the default. Boot images are cached per mode instead of one shared slot: AllowlistOnly keeps using the existing composefs.image.boot[.v1] named-ref key (no back-compat concerns for the common case), while other modes get composefs.image.boot[.v1].xattrs=<mode>, built from XattrFiltering's Display impl. OciImage and OpenConfig grow a single boot_image_refs map (extracted by key prefix) in place of the old fixed pair of V1/V2 Option fields, so AllowlistOnly and KeepUserXattrs boot images can be generated, cached, and garbage-collected side by side without evicting or masking each other; remove_boot_image now clears every cached mode rather than just the default one. Since a UKI embeds the composefs digest of the boot image it was built against, and that digest depends on which mode built the image, add find_matching_boot_image(): it tries every mode in turn (generating or cache-hitting the boot image for each) until one produces a digest matching the caller's expectation, so callers like bootc can recover from a default-mode change across composefs-rs versions without hard-coding retry logic or enumerating modes themselves. Assisted-by: AI Signed-off-by: Colin Walters <walters@verbum.org>

`find_matching_boot_image()` in composefs-oci needs to probe every EROFS format version a repository could plausibly have produced for a boot image, not just the one it's currently configured for -- a repository's `FormatConfig` is fixed once `meta.json` is written, so it can never spontaneously start generating a version it wasn't originally set up for. `FormatVersion::BOOT_VERSIONS` gives the two versions worth trying (V1, V2; the legacy V0 is never produced for boot images). `commit_image_version()` mirrors `commit_image()` but takes the version explicitly instead of deriving it from `repository.format_config()`, so a recovered non-default-version image can actually be persisted into the repo. Prep for the boot-image format-version search in composefs-oci. Assisted-by: AI Signed-off-by: Colin Walters <walters@verbum.org>

…_image `find_matching_boot_image()` already tried every `XattrFiltering` mode to recover a boot image digest embedded in an old UKI, because the default xattr-filtering mode has changed across releases. But it only ever built/compared against a single, fixed `FormatVersion` (whatever `repo.erofs_version()` currently resolves to), leaving the same class of bug unfixed for the EROFS on-disk format version axis. This matters concretely for bootc-dev/bootc#2334: a repository initialized by an older composefs-rs/bootc build may be permanently locked to `FormatConfig::single(V2)` -- format config is validated immutable once a repo's meta.json is written. If a newer build's default format version changes to V1 and seals a UKI with a V1 digest, that digest can never be found by looking at this repo's cached boot-image refs, no matter how many times the boot image is regenerated through the normal commit path, since the repo will only ever commit V2. Recovering requires building the filesystem tree once and computing the V1 image ID directly via the low-level, config-independent `FileSystem::compute_image_id()`. Restructure the search as mode-outer / version-inner, since rebuilding the tree is the expensive part: for each xattr mode, first check the cheap cached named-ref lookups for every `FormatVersion::BOOT_VERSIONS` entry, and only if none hit, build the tree once and compute the image ID directly for whichever versions weren't already covered by the cache. A match found this way is persisted back into the repo (writing the already-computed image bytes, plus a config/manifest rewrite) so the next lookup is a cache hit. `BootImageMatch::Found` now also carries the matched `FormatVersion` alongside the `XattrFiltering` mode; `NotFound` carries the total number of (mode, version) combinations that were attempted, since the per-attempt digests aren't useful for a caller building an error message (they're already known not to match, by construction). Assisted-by: AI Signed-off-by: Colin Walters <walters@verbum.org>

find_matching_boot_image (added in the prior two commits) can recompute a boot image digest for any (xattr filtering mode, EROFS format version) combination, which is exactly what's needed to recover from the scenario in bootc-dev/bootc#2334: a UKI embeds a composefs digest computed with whatever defaults were current at build time, and after an upgrade changes those defaults (or locks the repository to a newer format version), the digest can no longer be reproduced by a plain `--bootable` pull. Wire this up as `cfsctl oci pull --bootable --expected-digest <hex>` and as a new `expected_digest` parameter on the varlink `pull` method. Instead of generating the boot image with the default mode and the repository's format version, every combination is searched until one matches. `--expected-digest` requires `--bootable` and is rejected together with `xattrs` on the varlink side, since searching for an unknown mode and pinning one are contradictory requests. The varlink `Completed` reply gains `boot_image_mode` and `boot_image_format_version` fields (populated on both the recovery and the normal bootable-generate paths) so callers can tell which combination actually produced the digest without having to guess. Assisted-by: AI Signed-off-by: Colin Walters <walters@verbum.org>

Exercise the CLI end to end against the bootc-dev/bootc#2334 scenario: pull a bootable image into a V2-locked repository, separately compute the boot image digest an older bootc build would have embedded (a V1 image, via --erofs-version), then confirm --expected-digest recovers by finding the (mode, version) combination that reproduces it. Also cover the "no combination matches" and "requires --bootable" failure paths. create_bootable_oci_layout builds the minimal fixture transform_for_boot needs (empty boot/ and sysroot/ directories, some content under usr/) without any real kernel/UKI content, since find_matching_boot_image only cares about the resulting composefs digest. Assisted-by: AI Signed-off-by: Colin Walters <walters@verbum.org>

cgwalters mentioned this pull request Jul 28, 2026

Error during upgrade w/ composefs-backend and UKI since 1.16.4: composefs= parameter mismatch bootc-dev/bootc#2334

Open

cgwalters enabled auto-merge July 28, 2026 19:55

cgwalters requested a review from Copilot July 28, 2026 21:02

Copilot started reviewing on behalf of cgwalters July 28, 2026 21:03 View session

Copilot AI reviewed Jul 28, 2026

View reviewed changes

Johan-Liebert1 reviewed Jul 29, 2026

View reviewed changes

cgwalters added 5 commits July 29, 2026 16:14

cgwalters force-pushed the xattr-filter-mode branch from 410d1cb to e9724f7 Compare July 29, 2026 20:33

Johan-Liebert1 approved these changes Jul 30, 2026

View reviewed changes

cgwalters added this pull request to the merge queue Jul 30, 2026

Merged via the queue into composefs:main with commit 0fbc853 Jul 30, 2026
33 of 35 checks passed

Navigation Menu

Uh oh!

Add API to automatically generate matching EROFS given a digest - #367

Add API to automatically generate matching EROFS given a digest#367
cgwalters merged 5 commits into
composefs:mainfrom
cgwalters:xattr-filter-mode

cgwalters commented Jul 28, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Johan-Liebert1 left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Johan-Liebert1 Jul 29, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

cgwalters commented Jul 28, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Johan-Liebert1 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Johan-Liebert1 Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants