Skip to content

conformance: a worker-mode leg over the same corpus, with per-case regular/worker equivalence #209

Description

@melihucar

Gate: default
Agent: codex

Depends on: #200, #208

vendor/frankenphp/testdata/_executor.php exists so one fixture runs unchanged
in both regular and worker mode — that is upstream's own _module/_worker
test pattern (vendor/frankenphp/frankenphp_test.go:154-163). Our corpus
(tests/conformance/corpus.toml) replays every case in regular mode only, so
nothing checks that our worker mode produces the same answers as our regular
mode.

Split out of #14, which could not host this: at the time it was written the
frankenrust replay leg was disarmed (tests/conformance/lib/replay.py:56,
FRANKENRUST_LEG_ENABLED = False), so worker cases added to the corpus would
have compared upstream to itself and passed unconditionally.

This issue must not be started until #200 has armed the frankenrust leg.
State in your final message whether it was armed when you ran, and if it was
not, stop — an acceptance criterion that cannot fail is worse than no criterion.

The equivalence is per-case, not global

Do not assert "worker goldens == regular goldens" across the corpus. It is
already false for a case in the tree: tests/conformance/golden/exception.http
pins

#0 {documentRoot}/_executor.php(5): {closure:{documentRoot}/exception.php:5}()

Line 5 of _executor.php is $fn(); — the regular-mode branch. In worker
mode the closure is invoked by zend_call_function from
PHP_FUNCTION(frankenphp_handle_request) (vendor/frankenphp/frankenphp.c:882),
entered from _executor.php:9, so the frame cannot be byte-identical. Worse,
an uncaught exception there goes through zend_exception_error(..., E_ERROR)
(frankenphp.c:894) and then tears the worker script down and restarts it — a
different control path with different output.

So: each case carries its own worker golden, captured from the official
container
, exactly as #4 captured the regular goldens. The equivalence claim
that survives is per-case and empirical: for each case, our worker response
matches upstream's worker response. Cases whose worker and regular goldens
happen to be byte-identical are the common outcome, not the assertion.

What to build

  • A per-case mode in corpus.toml ("regular" — the default and current
    behaviour — and "worker"), and a worker golden directory alongside
    tests/conformance/golden/. Do not repurpose the existing goldens.
  • Capture tooling: the official-container capture path conformance: golden HTTP corpus captured from the official FrankenPHP container #4 established, with a
    worker-configured Caddyfile (worker ./_executor.php, see
    vendor/frankenphp/testdata/Caddyfile:3-5, where it is commented out).
  • Replay: replay_frankenrust() starts frankenrust:bench with the worker
    knobs server: worker-mode configuration, boot split and dispatch #208 added to docker/frankenrust.Dockerfile.
  • lib/selftest.py gains a check that the worker corpus does not silently
    shrink, in the shape of the existing coverage check — and that every case
    present in one mode is present in the other or carries a written reason.

Acceptance

  • The worker leg replays with the frankenrust leg armed, and the run reports a
    real per-case pass/fail — not a skip, not a self-comparison.
  • exception carries its own worker golden and is not exempted.
  • bash scripts/gate.sh default passes, and its log distinguishes "worker leg
    ran, N cases" from "worker leg skipped".

Out of scope

Arming the frankenrust leg (#200), [targets.frankenrust] wiring (#143/#162),
skip-budget bounds (#189), and worker-mode benchmarking (#16).

Metadata

Metadata

Assignees

No one assigned

    Labels

    fr:followupFiled by an agent mid-taskfr:p2Normal -- the default when untriagedfr:readyClaimable by an agentfr:waitingReady, but waiting on an open dependency

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions