Draft: surviving-placeholder (orphan) report producer

[dividedby]

Draft proposal — surviving-placeholder (orphan) report producer

What it does. Adds tools/reportOrphans.mjs, an opt-in producer that enumerates the apply-time surviving-placeholder set: a ${...} whose leading identifier binds to no slot for its prompt, so it survives --apply as a raw ${NAME} into a live template literal and faults at boot (ReferenceError: NAME is not defined). It is the structured-data companion to auditMisbinds.mjs — that tool checks placeholders that DO bind (right name, wrong slot) and deliberately skips this set ("leak/uncomparable: other gates cover it"); this is meant to be that other gate.

It writes one JSON object to stdout and touches nothing in the normal --apply path:

{ "version": "2.1.x", "prompts": { "<promptId>": ["VAR", ...] } }

keyed per prompt id and variable, so a downstream consumer can map each finding to its ReferenceError: <VAR> is not defined signature without a live install.

Two things it does beyond a bare scan (and beyond your existing apply-path skip-guard in src/patches/systemPrompts.ts):

1. Covers the expression class, not just bare names. The crashing forms are interpolation expressions — ${!IS_TRUTHY_FN(PROCESS_OBJECT.env.X)&&..? ... }, ${OBJECT.member}, ${FN()}. It extracts the leading identifier of each expression (the name evaluated first, hence the one that appears in the ReferenceError). Your apply-path guard's regex requires the closing brace flush after the name, so it matches only the bare ${NAME} form and never sees these.
2. Per-prompt precision with a union fallback. The known-slot set is the prompt's own identifierMap values; for a tweakcc-own id absent from the prompts JSON it falls back to the union (which mirrors your loadIdentifierMapUnion). The per-prompt set is what catches IS_TRUTHY_FN on a prompt that has no such slot — the union alone would not, since IS_TRUTHY_FN is a valid slot on another prompt. Escaped literals are inert and not flagged.

It's a draft proposal — your call on the final shape and home. I built it as a standalone tools/ helper (sibling to auditMisbinds.mjs, version-independent) because that was the most reviewable surface to show the idea. But the home is entirely yours to pick:

• fold it into the apply path in systemPrompts.ts (the natural deep home — it already has the union and the per-prompt resolution in hand, so this could reuse loadIdentifierMapUnion directly instead of re-deriving the union),
• make it a field on driver.mjs report,
• or extend auditMisbinds output to emit the set it currently skips.

Happy to rework it into whichever fits — or to drop it entirely if keeping the boot-verify backstop plus the existing count is the answer you prefer.

The cost it puts on you. One new helper unit you would maintain (the leading-identifier extraction + the per-prompt/union orphan logic). It is opt-in — a separate tool that emits JSON to stdout — so there is zero impact on the normal --apply flow, and nothing in your release path has to call it. The helpers are version-independent and unit-tested (tools/reportOrphans.test.mjs, 15 cases: bare name, _FN(...), _OBJECT.member, the leading-operator boot-crash form, escaped \${...} not flagged, per-prompt vs union). The mis-bind audit and the four-zeros bar are untouched.

No merge pressure. This is a prepared draft for your consideration, not a request to land. Keeping it as the placeholder count plus boot-verify is a perfectly good answer.

Why we want it (the consumer side). This is the producer for our integration gate's orphan-report consumer. Today the consumer falls back to a boot-verify-derived proxy because no tool emits the surviving set keyed per prompt + variable. A real producer lets the gate map each finding to its exact ReferenceError signature statically.

---

Agreement with the recorded boot-verify evidence

Run against our overrides set with the 2.1.169 prompts JSON, the producer keys the prompt that boot-crashed on our 2.1.169 gate run exactly as boot-verify reported it:

$ node tools/reportOrphans.mjs data/prompts/prompts-2.1.169.json <overrides-dir>
...
  "tool-description-agent-usage-notes": [ "IS_TRUTHY_FN" ],
...

That override is pinned at an older ccVersion and interpolates ${!IS_TRUTHY_FN(PROCESS_OBJECT.env...)&&..? ... }; IS_TRUTHY_FN is not a slot for that prompt in 2.1.169, so it survives apply and faults — matching the live ReferenceError: IS_TRUTHY_FN is not defined we recorded. (Leading-identifier note: it reports IS_TRUTHY_FN, not the nested PROCESS_OBJECT, because that is the name evaluated first and the one in the actual ReferenceError.)

Verification

• tools/reportOrphans.test.mjs — 15 unit tests, all pass.
• Full suite: vitest run — 365 passed, 5 skipped (29 files); no regression. tsc --noEmit and eslint src clean.
• auditMisbinds still exits 0 against our set (the change is purely additive; your slot logic is untouched).
• Left for you: a real-install boot-verify on your machine — confirming that an override flagged by this producer is the one that crashes, and that a clean set produces an empty prompts object. The agreement above is against our recorded boot-verify set; it is sufficient for a draft but is not a substitute for your own live run.

🤖 Generated with Claude Code

[skrabe]

Closing — but the underlying observation was right, so credit where due: expression-position placeholders (${!FN(OBJ.env.X)&&…}) do dodge every bare-${NAME} gate here, and checking for that class surfaced a real stale rebind in an override set, now fixed (lobotomized-claude-code 968e32d, plus your #7 merged for the opus sets).

Not taking the tool itself, same reasoning as the golden snapshot in #6: it duplicates gates I already run (boot-verify catches this class live on every apply, and the leading-identifier check is a ~15-line addition to my existing validator rather than a 273-line unit to maintain), and per its own description it's the producer for your downstream consumer — that's a need on your side of the fence, so it makes more sense living in your maint repo against my published prompts JSONs. Thanks for the draft and for flagging the class.