test(compat): HEAD sample sealer plus published pg-core reader crate

[dobby-coder]

Rust half of the bidirectional wire-compat gate. Closes #260; part of #251 (epic #247, workstream C).

Today only one direction is tested: committed old fixtures opened by a new reader (pg-core/tests/wire_format.rs). The reverse is what breaks external consumers, and nothing checks it. This adds the two halves that do.

Sealer

cargo run -p pg-core --features stream --example seal-samples -- <dir>

Seals a sample set with the code in this tree. Five cases: in-memory, in-memory with a private signing key, streaming, streaming with a private signing key, and streaming across two AEAD segments (which also carries a populated size_hint). Every container is sealed for two recipients, so the multi-recipient KEM entries in the header get read too.

The two private-signing cases are not redundant. In memory mode the signature goes in the header; in stream mode it goes in the encrypted payload trailer. They are different wire surfaces and a break confined to one of them passes the other.

Everything derives from one 32-byte seed (master keys, USKs, signatures, IVs), so two runs of the same tree produce byte-identical files. That is what makes a diff between two sets mean "the wire format moved" rather than "the RNG moved". The manifest carries no timestamp or commit hash for the same reason.

pg-core/tests/sample_sealer.rs keeps the sealer honest inside the existing Test workspace job: determinism, manifest-names-every-file (a case a reader cannot find is a case it skips), and a HEAD read-back of every case for every recipient.

Reader

pg-compat/ depends on published pg-core =0.6.1 from crates.io and opens the sealed bytes. It sits outside the workspace on purpose: the root Cargo.toml exclude is what stops cargo resolving pg-core to ../pg-core and turning the gate into a tautology. It keeps its own Cargo.lock and runs with --locked.

The dependency list is the support window from #252. Adding a version is a renamed manifest entry plus one reader! invocation; the check body is a macro because each published version is a separate crate with separate types.

A missing or empty artifact set fails. A gate that reads nothing and passes is worse than no gate.

One process per case

Each case is opened by a separate pg-compat-case child process. That is the part of the diff to look at hardest, because it is a response to how the gate failed rather than to how it was specified.

A published reader handed a shifted header does not return an error. It reads a garbage length prefix, attempts a 20 GiB allocation, and the process aborts. In one process that killed the whole run on the first case, so the gate printed no per-case list for exactly the breakage class it exists to catch. catch_unwind is no help, because an abort is not a panic.

With one child per case, a case that dies is reported by name, with the signal and the child's last stderr line, and the other cases still run:

HEAD-sealed containers do not open with published pg-core:
  pg-core 0.6.1: mem: reader died on signal 6 before it could report: memory allocation of 21474836480 bytes failed
  pg-core 0.6.1: mem-privsig: reader died on signal 6 before it could report: memory allocation of 21474836480 bytes failed
  pg-core 0.6.1: stream: reader died on signal 6 before it could report: memory allocation of 21474836480 bytes failed
  pg-core 0.6.1: stream-privsig: reader died on signal 6 before it could report: memory allocation of 21474836480 bytes failed
  pg-core 0.6.1: stream-multi-segment: reader died on signal 6 before it could report: memory allocation of 21474836480 bytes failed

The children run under target/, so an abort's core dump stays out of the working tree; core is gitignored too, for anyone who runs a reader by hand.

The five unit tests in pg-compat/src/lib.rs cover the harness itself, including one whose child aborts on SIGABRT for real rather than through a mock. They fail against the previous single-process shape, which had nowhere for them to hook in.

Artifact layout

Documented in pg-compat/README.md, including the manifest schema, what each field means and why the plaintext is a sibling file rather than base64. #261 (the Node old-reader half) consumes the same directory as a job artifact, so treat it as a contract: schemaVersion is there to be bumped when it changes. vk.json and usk-*.json are shaped like real PKG responses, so the same files feed a Rust reader and a JS one with no translation step.

Two things there aimed squarely at #261: sender.public and sender.private are written up alongside the rest of the contract, because they are what a JS reader checks the header signature against. And the proposed job uploads nothing on a PR that misses its path filter, so a wire-compat-js job that downloads the artifact unconditionally will hard-fail rather than skip. That is in the README under "Consuming the set from another job".

Does it actually go red

Checked by hand on this branch, reverted after:

change to HEAD	result
VERSION_V3: 2 → 3	red, before reading a byte: one line, "sample set claims wire version 3, this reader speaks 2"
new field at the start of Header	red on all five cases, with the transcript above
new field at the end of Header	green, and correctly so

Read that last row before taking a green run as permission. The header is a length-prefixed region and bincode ignores trailing bytes, so a field appended at the end of Header does still open with 0.6.1. Anywhere else shifts every following byte. It is written down in CLAUDE.md and the README.

The CI job

The wire-compat-rust job is not in this diff, because the App cannot push .github/workflows/. The YAML is in a PR comment below for a maintainer to apply; it path-filters on pg-core/**, pg-wasm/** and pg-compat/**, and uploads the sealed set so #261's wire-compat-js can download it. Until someone lands it, nothing here gates a PR, which is how CLAUDE.md describes it.

pg-compat is in none of the per-crate CI matrices either, so its fmt and clippy invocations are in the proposed YAML and written down in CLAUDE.md.

Notes

• pg-core gains one dev-dependency, rand_chacha. StdRng is explicitly not guaranteed reproducible across rand releases and ChaCha20Rng is; it is already in the tree via rand's std_rng, so the dependency graph does not grow.
• The example is gated behind required-features = ["rust", "stream"], so cargo build and a default-feature cargo check skip it.
• Ran locally: cargo test --manifest-path pg-core/Cargo.toml --features test,rust,stream (67 pass), the pg-compat suite (6 unit plus the gate), cargo build --workspace, cargo fmt --all -- --check from the repo root, and fmt and clippy for pg-compat.
• Not this PR's problem, but review turned it up and it should get its own issue: Unsealer::new aborting the process on a malformed header is pre-existing on main and reachable from pg-pkg, which parses attacker-supplied ciphertext.

[dobby-coder]

wire-compat-rust job for .github/workflows/build.yml

The App cannot push .github/workflows/, so this needs a maintainer. Append the block below to the jobs: map in .github/workflows/build.yml (it splices in as-is at two-space indent; I parsed the combined file to check). Nothing on this PR gates anything until it lands.

Two things worth a look before you apply it:

1. The path filter is a step, not an on: paths: key. ci: bidirectional wire-compat gate — HEAD-sealed output must open with all published readers in the support window #251 wants this as a required check via branch protection. A job skipped by an on: paths: filter never reports a status, so a required check would sit pending forever on every PR that does not touch pg-core/**. With the filter as a step, the job always reports and only does the expensive work when the wire surface changed.
2. It needs dorny/paths-filter@v3, a new third-party action for this repo. If you would rather not add one, the alternative is dropping the filter and letting the job run on every PR. A full run is a cargo run plus a cargo test against one published crate, so it is not expensive; the filter mostly saves the toolchain install. On workflow_dispatch the action diffs against the default branch, which at worst means the job runs when it did not have to.

  # ---------------------------------------------------------------------------
  # Wire compatibility: HEAD-sealed containers must open with published readers
  # (#251 / #260). #261 adds a wire-compat-js job that downloads the artifact
  # this job uploads.
  #
  # The path filter is a step, not an `on: paths:` key, on purpose: a job that
  # is skipped by a path filter never reports, so a required check would sit
  # pending forever on PRs that do not touch the wire surface. This shape always
  # reports and only does the expensive work when it needs to.
  # ---------------------------------------------------------------------------
  wire-compat-rust:
    name: Wire compat (published pg-core)
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: dorny/paths-filter@v3
        id: changes
        with:
          filters: |
            wire:
              - 'pg-core/**'
              - 'pg-wasm/**'
              - 'pg-compat/**'
              - '.github/workflows/build.yml'

      - if: steps.changes.outputs.wire == 'true'
        uses: dtolnay/rust-toolchain@stable
        with:
          components: clippy, rustfmt

      # pg-compat is outside the workspace, so the per-crate matrices above
      # do not lint it.
      - name: Format pg-compat
        if: steps.changes.outputs.wire == 'true'
        run: cargo fmt --manifest-path pg-compat/Cargo.toml --all -- --check

      - name: Clippy pg-compat
        if: steps.changes.outputs.wire == 'true'
        run: cargo clippy --manifest-path pg-compat/Cargo.toml --all-targets --locked -- -D warnings

      - name: Seal the sample set with HEAD
        id: seal
        if: steps.changes.outputs.wire == 'true'
        shell: bash
        run: |
          cargo run --locked -p pg-core --features stream --example seal-samples \
            -- "$RUNNER_TEMP/wire-compat-artifacts"

      # Hand-off to wire-compat-js (#261). Uploaded before the reader runs so a
      # red gate still leaves the bytes that broke it.
      - name: Upload the sample set
        if: steps.seal.outcome == 'success'
        uses: actions/upload-artifact@v4
        with:
          name: wire-compat-artifacts-${{ github.sha }}
          path: ${{ runner.temp }}/wire-compat-artifacts
          retention-days: 7

      - name: Open it with published pg-core
        if: steps.changes.outputs.wire == 'true'
        shell: bash
        env:
          PG_COMPAT_ARTIFACTS: ${{ runner.temp }}/wire-compat-artifacts
        run: cargo test --manifest-path pg-compat/Cargo.toml --locked

The artifact it uploads is the hand-off to #261: same directory, layout documented in pg-compat/README.md, name wire-compat-artifacts-<sha>.

[dobby-coder]

VERDICT: request-changes

Rules sweep plus the consolidated review, cycle 1. Fourteen rules applied (the Rust, CI-workflow, patch-in-comment, PR-process and prose sets); all pass. The proposed YAML matches path-filtered-job-cannot-be-a-required-check: step filter rather than an on: paths: key, the workflow file itself included in the filter, and dorny/paths-filter flagged as new third-party with the drop-the-filter alternative offered. The CLAUDE.md wording matches patch-in-comment-gate-is-not-live-yet by naming the job as not yet a gate and saying where the patch lives. Closes #260 is a firing keyword. CI is green on all 22 checks with nothing in flight.

One blocking issue, in two places. It is a documentation-accuracy problem, not a hole in the gate.

The gate does not produce the diagnostic three files promise

Reproduced on this HEAD. Added pub injected: u32 as the first field of Header (pg-core/src/client/header.rs:104), set it in the single constructor at header.rs:167, resealed, ran the gate:

running 1 test
memory allocation of 21474836480 bytes failed
error: test failed, to rerun pass `--test wire_compat`
Caused by:
  process didn't exit successfully: ... (signal: 6, SIGABRT: process abort signal)

The shifted bytes make bincode read a garbage length prefix, Unsealer::new attempts a 20 GiB allocation, and the process aborts on the first case. The failures: Vec<String> accumulator in pg-compat/src/lib.rs never gets to run, so there is no per-case list and no invalid length 0, expected struct Header with 3 elements. The run also left a 3.1 MB core in pg-compat/, which the root .gitignore does not cover: it ignores target, not core, and git check-ignore confirms the file is untracked and unignored.

The gate still goes red, which is the property that matters. What is wrong is that pg-compat/README.md:131, CLAUDE.md:9 and the PR body's verification table each describe an outcome the gate does not deliver for exactly the breakage class it exists to catch. CLAUDE.md is auto-loaded agent context, so a later run reads "fails at header decode", meets an allocation abort, and has no reason to connect the two.

Either correct the three claims, or isolate each case in a subprocess so one abort does not take the whole run with it. catch_unwind will not help, because an abort is not a panic. Correcting the wording is cheaper and it is the honest option. Add core to .gitignore either way.

Not blocking

Four nits inline. Plus one thing that is not this PR's:

Unsealer::<Vec<u8>, UnsealerMemoryConfig>::new aborting the process on a malformed 1231-byte header that clears the preamble_checked length bounds is pre-existing on main, not introduced here, and it deserves its own issue. CLAUDE.md lists malformed-ciphertext panics as a pitfall class with fixes already shipped; this is a surviving instance, and an abort is worse than a panic because it cannot be contained. It matters for pg-pkg, which parses attacker-supplied ciphertext. GitHub cannot anchor an inline comment to header.rs because the file is not in this diff, so the note lives here.

Staying in draft until the fix lands.

[dobby-coder]

VERDICT: approve

Rules sweep plus the consolidated review, cycle 2. The cycle-1 blocker is fixed at the root rather than papered over in the docs: each case now runs in its own child process, so a first-position Header field produces a five-line list naming every case instead of one SIGABRT that ends the run. I re-ran that repro on this branch and got the list.

Rules checked against the delta since 68086ae: the Rust set, the CI-workflow set, patch-in-comment, the PR-process set and the prose set. All pass. Three specific ones worth recording, since they are the ones that usually bite:

• patch-in-comment-gate-is-not-live-yet: the CLAUDE.md bullet still says the wire-compat-rust job is not yet a gate and names the comment it lives in. The README's new section calls it "the proposed job". Neither claims a gate that does not exist.
• actions-default-shell-has-no-pipefail: no run: block in the proposed YAML contains a shell pipe, and the two multi-line ones declare shell: bash anyway. GitHub cannot anchor an inline comment to a file outside the diff, so the YAML review lives here.
• The YAML still drives the code correctly now that pg-compat has a second binary target. cargo test --manifest-path pg-compat/Cargo.toml --locked builds pg-compat-case before the integration test looks it up through CARGO_BIN_EXE_pg-compat-case, so the job needs no edit.

No bugs. Three nits from the review, all fixed on the branch rather than sent back for another cycle.

Fixed here

fix(compat): check the wire version in the per-case child too (51ff1f6). The wire_version guard moved into the parent when cases gained their own processes, and the child had no equivalent, so the hand-run path in README:36 lost it. Verified both directions: with wireVersion forced to 3 in a copied set, pg-compat-case ... 0.6.1 mem now prints pg-core 0.6.1: sample set claims wire version 3, this reader speaks 2 and exits 1, where before it handed the bytes to 0.6.1. The parent still reports the bump once per reader, not once per case.

docs(compat): the manifest carries no wall-clock timestamp (0c09dbb). The determinism section said the manifest carries no timestamp, but the schema block 70 lines above it shows sender.public.ts and sender.private.ts, both 1704067200. Those values are part of the policy the header signature is computed over, so #261 has to read them. The determinism claim itself is intact, because the value is a constant in sample_set.rs, not a clock.

chore: ignore core.<pid> dumps as well as core (3779735). kernel.core_pattern is core.%p on many hosts, so the bare entry missed the file on exactly the hand-run it was added for.

Verified locally on 3779735

cargo fmt and cargo clippy -- -D warnings on pg-compat (outside the workspace, so no CI matrix covers it), the sealer, and the full gate: 6 unit tests plus the integration test, green. All 21 CI checks green, nothing in flight.

Not this PR

Carried forward from cycle 1 and still worth its own issue: Unsealer::<Vec<u8>, UnsealerMemoryConfig>::new aborts the process rather than returning Err on a malformed header that clears the preamble length bounds. That is pre-existing on main, and it matters for pg-pkg, which parses ciphertext it did not produce.

Flipping to ready.