You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
gate: nothing checks that vendor/frankenphp/ is unmodified — the oracle is the one hard rule with no fail-closed check, and doc-citations rewards editing it #204
Hard rule 3 in orchestrator/prompts/shared.md:26 is "Do not modify vendor/frankenphp/. It is the oracle." Nothing enforces it.
Evidence
Every other hard rule in this project has a fail-closed gate step. This one has
none. The full scripts/gate.sh step list:
test-suite-intact <- rule 1, "never weaken the gate to pass it" (#58, #67, #68)
orchestrator-runnable <- self-modification survivability
doc-citations <- docs must cite upstream truthfully (#20)
dev-env / build / fmt / clippy / test / conformance / bench-smoke / bench-report-selftest
^ nothing in this list reads vendor/ as a thing to protect
$ grep -c vendor orchestrator/loop.py orchestrator/gh.pyorchestrator/loop.py:2 # both lines are "agent vendor" (codex/claude), not vendor/orchestrator/gh.py:0
guard_root_writes (#168) guards stray writes to the main checkout. A vendor
edit made inside the agent's own worktree is in scope for that worktree, passes
every gate step, and merges.
The incentive is real, not hypothetical
scripts/check_doc_citations.py resolves each vendor/frankenphp/<file>:<line>
citation against the real file and fails when the range is out of bounds. There
are two ways to make a failing citation pass: fix the doc (correct), or add
lines to the cited vendor file (wrong, silent, green).
The checker's own docstring anticipates exactly this — "a vendor bump that
shifts lines without going out of bounds, or a careless edit, fails the
gate" — but its only defence is the hand-listed ANCHORS table, which pins a
handful of citations. The other 545 files in the vendor tree are unprotected.
#137, open, extends this checker to the 285 upstream citations in Rust
comments. That multiplies the number of ways the check can fail, and so the
incentive, by roughly six, while leaving the escape hatch open.
Two things stop anyone noticing afterwards:
orchestrator/prompts/reviewer.md:37 tells the reviewer "Compare against vendor/frankenphp/". A reviewer reviewing a diff that edits vendor reads the edited oracle as ground truth, so the divergence it was asked to find is
gone.
tests/conformance/ compares us to a container built from published upstream,
not to the vendor tree, so it does not see a vendor edit either.
No vendor edit has happened yet — git log --oneline -- vendor/ is one commit,
the scaffold a5e88df. That is why this is p2 and not p0: it is a gate hole
with no trigger fired, not a demonstrated bad merge.
What to build
scripts/check_vendor_pristine.sh, in the house style of scripts/check_test_suite.sh. It must:
fail on a committed vendor change: compare git rev-parse HEAD:vendor/frankenphp
against a pin file;
fail on an uncommitted one: git status --porcelain -- vendor/ must be
empty (the gate runs in a dirty worktree, so the hash alone is not enough);
report the changed paths (git diff --name-only <pinned> HEAD -- vendor/
plus the porcelain output), not a bare hash mismatch;
support --selftest, run before the real check, in the exact two-command
shape test-suite-intact and doc-citations already use in gate.sh.
step "vendor-pristine" in scripts/gate.sh, placed with the other
toolchain-free checks (alongside test-suite-intact / doc-citations, before dev-env) so it runs in every profile, including bootstrap.
bash scripts/gate.sh bootstrap passes on a clean tree.
Appending one line to any file under vendor/frankenphp/ makes bash scripts/gate.sh bootstrapfail, naming that file. Paste the real
command output in your final message, then revert the edit.
A deliberate vendor bump is still possible, but only by editing .gate/vendor-tree in the same commit — one visible, reviewable line in the
diff instead of a silent 545-file blind spot.
Gate: bootstrap
Agent: codex
Hard rule 3 in
orchestrator/prompts/shared.md:26is "Do not modifyvendor/frankenphp/. It is the oracle." Nothing enforces it.Evidence
Every other hard rule in this project has a fail-closed gate step. This one has
none. The full
scripts/gate.shstep list:guard_root_writes(#168) guards stray writes to the main checkout. A vendoredit made inside the agent's own worktree is in scope for that worktree, passes
every gate step, and merges.
The incentive is real, not hypothetical
scripts/check_doc_citations.pyresolves eachvendor/frankenphp/<file>:<line>citation against the real file and fails when the range is out of bounds. There
are two ways to make a failing citation pass: fix the doc (correct), or add
lines to the cited vendor file (wrong, silent, green).
The checker's own docstring anticipates exactly this — "a vendor bump that
shifts lines without going out of bounds, or a careless edit, fails the
gate" — but its only defence is the hand-listed
ANCHORStable, which pins ahandful of citations. The other 545 files in the vendor tree are unprotected.
#137, open, extends this checker to the 285 upstream citations in Rust
comments. That multiplies the number of ways the check can fail, and so the
incentive, by roughly six, while leaving the escape hatch open.
Two things stop anyone noticing afterwards:
orchestrator/prompts/reviewer.md:37tells the reviewer "Compare againstvendor/frankenphp/". A reviewer reviewing a diff that edits vendor reads theedited oracle as ground truth, so the divergence it was asked to find is
gone.
tests/conformance/compares us to a container built from published upstream,not to the vendor tree, so it does not see a vendor edit either.
No vendor edit has happened yet —
git log --oneline -- vendor/is one commit,the scaffold
a5e88df. That is why this is p2 and not p0: it is a gate holewith no trigger fired, not a demonstrated bad merge.
What to build
scripts/check_vendor_pristine.sh, in the house style ofscripts/check_test_suite.sh. It must:git rev-parse HEAD:vendor/frankenphpagainst a pin file;
git status --porcelain -- vendor/must beempty (the gate runs in a dirty worktree, so the hash alone is not enough);
.gate/vendor-tree, following the.gate/min-test-filesconvention established by gate.sh's test-suite-intact check is blind to tests under crates/ #58. Seed it with the current value:
a6119f8613aa71d4fb890c43d4a19dc98c40a016;git diff --name-only <pinned> HEAD -- vendor/plus the porcelain output), not a bare hash mismatch;
--selftest, run before the real check, in the exact two-commandshape
test-suite-intactanddoc-citationsalready use ingate.sh.step "vendor-pristine"inscripts/gate.sh, placed with the othertoolchain-free checks (alongside
test-suite-intact/doc-citations, beforedev-env) so it runs in every profile, includingbootstrap.The selftest must be able to fail — see prompts: no prompt asks whether a new test can fail — six blocked rounds and two disarmed gate checks, all the same defect #192. At minimum: one negative case
that mutates a vendor-shaped file in a temp git tree and asserts non-zero
exit, and one positive case on a clean tree asserting exit 0.
Acceptance
bash scripts/gate.sh bootstrappasses on a clean tree.vendor/frankenphp/makesbash scripts/gate.sh bootstrapfail, naming that file. Paste the realcommand output in your final message, then revert the edit.
.gate/vendor-treein the same commit — one visible, reviewable line in thediff instead of a silent 545-file blind spot.
Out of scope
check_doc_citations.pyto Rust comments — that is gate: check_doc_citations.py scans only docs/*.md, leaving 285 upstream citations in Rust comments — including SAFETY-comment establishment points — unchecked #137.guard_root_writes— orchestrator: a straygit commitin the main checkout is invisible to guard_root_writes, and publish_main pushes it #190, orchestrator: a straygit reset --hard/git checkout .in the main checkout destroys uncommitted content and guard_root_writes cannot see it #191, orchestrator: guard_root_writes' post-stage capture can land inside a concurrent ff-merge and fabricate a root_write naming every merged file #193, orchestrator: guard_root_writes reverts a tracked path when only the index flipped — a straygit addin ROOT with no content write discards uncommitted work #194 own that.vendor/read-only on disk, or converting it to a submodule. The pinis sufficient and does not disturb the worktree machinery.