Skip to content

Close libarchive archive/filter parity gaps - #374

Merged
Hawkynt merged 19 commits into
mainfrom
feature/libarchive-parity
Sep 19, 2026
Merged

Hawkynt merged 19 commits into
mainfrom
feature/libarchive-parity

Conversation

@Hawkynt

@Hawkynt Hawkynt commented Sep 15, 2026

Copy link
Copy Markdown
Owner

What changed

Two libarchive formats CWB did not have:

  • BSD mtree(5) manifest reader, writer and registry descriptor. mtree is treated as what it is — a metadata manifest with no embedded file bodies — so the descriptor advertises list/create/test and not extract, and CWB never dereferences a contents= host path out of an untrusted manifest.
  • begin-base64 / ==== wrapper as B64Encoding, libarchive's b64encode filter: streaming 57-byte input blocks, canonical 76-character lines, and name/mode options.

The CPIO half of the original branch has been dropped. Read and write every historical cpio header variant (#376) landed all four header variants on main first, with a shared CpioLayout, verified crc checksums, variant-preserving edits and a larger test set. This branch no longer touches any CPIO file.

Fixes made while rebasing

  • The wrapper announced a mode C# had already turned into a different number. 0644 and 0777 are decimal literals in C#, not octal, so the mask cleared permission bits and the default was never 0644: the header carried 1000 by default and 400 for a requested 0600. The mask and default are now named constants written in hex alongside the octal value they stand for.
  • The same bug was live on main's classic uuencode path, which emitted begin 1204. libarchive rejects that header outright (Unrecognized archive format) and accepts the begin 644 now emitted, so this was an interoperability defect rather than a cosmetic one.
  • The conversion matrix verified every conversion by reading the payload back, which assumes each creatable target stores bytes. mtree is the first target that does not, and the matrix read the absent payload as a broken conversion. Byte comparison is now skipped for a target that does not advertise CanExtract; the conversion still runs and the manifest is still re-listed and checked for the expected entry names. mtree is the only descriptor in the repository this applies to.

Not included

PWB/UNIX binary CPIO is deliberately left out. It is byte-identical to the little-endian 7th Edition variant, so the original branch detected it with a mode-bit heuristic that latched for the remainder of the archive and rewrote mode bits once it fired. Placing that in front of binary CPIO reads which #376 validated against bsdcpio and GNU cpio risks changing archives that read correctly today. If PWB is wanted, an explicit write option plus an explicit read override — the shape libarchive itself uses — is the safe design, and belongs in its own change.

Validation

  • Gating suite: 0 failed, 30337 passed, 1446 skipped. Measured against main (0 failed, 30295 passed, 1446 skipped) test-for-test: no test present on main is missing or changed outcome; 42 are new.
  • bsdtar / bsdcpio (libarchive 3.8.9) run as real oracles in both directions for the wrapper and for mtree.
  • Build: 0 errors, 348 warnings, matching main.
  • Package READMEs regenerated; the support-matrix rows for both new formats are checked against the live registry by ArchivesReadmeStateTests.

References

  • FreeBSD/libarchive mtree(5)
  • libarchive b64encode filter and its published test vector

@Hawkynt
Hawkynt force-pushed the feature/libarchive-parity branch 3 times, most recently from 3a5981b to 75ba219 Compare September 19, 2026 01:40
…her number

The header mode was written as `0644` and masked with `0777`. C# has no octal
literal, so those are the decimal numbers 644 and 777: the mask cleared bits
that belong to the permission field and the default was never 0644 at all.
`begin-base64` came out carrying "1000" for the default and "400" for a
requested 0600, which is not a mode bsdtar will accept.

The permission mask and the default are now named constants written in hex,
with the octal value they stand for stated next to them, so the two notations
cannot be confused again. The round-trip test asked for `0600` and expected
`0600` back, which held only while both ends were wrong in the same way; it now
pins the header text as well as the value.
…ores

Every conversion target so far could also be extracted, so the matrix verified
a conversion by reading the payload back. mtree is the first target that
records metadata and no file bodies, and the matrix read the absent payload as
a broken conversion rather than as the format's own shape.

Payload comparison is now skipped for a target that does not advertise
CanExtract. The conversion still runs and the manifest is still re-listed and
checked for the expected entry names, so the pair keeps its coverage instead of
being excluded from the grid.
The generated reference prints a signature and its summary, so the mode default
now reads as the number it is. Both encoder summaries say which octal value that
number stands for, rather than leaving a bare 420 in the table.
@Hawkynt
Hawkynt force-pushed the feature/libarchive-parity branch from 75ba219 to fb8a443 Compare September 19, 2026 02:18
@Hawkynt
Hawkynt merged commit df0585b into main Sep 19, 2026
5 checks passed
@Hawkynt
Hawkynt deleted the feature/libarchive-parity branch September 19, 2026 02:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant