Roadmap: leaf test broadening (tweakcc-fixed + lobotomized)

[dividedby]

Parent

PRD: release-adoption control plane — #2 (slice 2, roadmap)

What to build

Broaden leaf checks — unit tests (escape/regex logic) and golden extraction snapshots in tweakcc-fixed's vitest, plus per-override coverage on the lobotomized side — so a version bump can't silently break patches/prompts.

Per ADR 0001 this is leaf work, tracked against the leaf repos, not a tweakcc-maint deliverable. This issue is a tracking placeholder; the actual broadening lands as PRs to the leaf repos (the control plane prepares verified PRs, never direct-pushes).

Acceptance criteria

• Triaged: scope the coverage gaps and decide which land as leaf PRs.

Blocked by

None - roadmap, needs triage before it is AFK-ready.

🤖 Generated with Claude Code

[dividedby]

This was generated by AI during triage.

Human Brief

Category: enhancement
Summary: Broaden leaf-level checks (unit/regex + golden extraction snapshots in tweakcc-fixed; per-override coverage on lobotomized) so a version bump can't silently break patches/prompts.

Why not an AFK agent: per ADR 0001 this is leaf work, and the leaves are owned by a separate maintainer (skrabe). The control plane reads/runs leaves and prepares verified PRs — it never direct-pushes or mutates a leaf (cockpit guardrail). The deliverable is PRs to the leaf repos, which need the leaf owner's involvement and merge.

Current behavior:
Leaf test coverage exists but is uneven; the gate's runtime checks (Boot-verify, orphan detection) are the backstop for breakage that leaf unit tests could catch earlier and cheaper.

Desired behavior:
Scoped coverage growth landing as PRs to the leaves: escape/regex-logic unit tests and golden extraction snapshots in tweakcc-fixed's vitest, and per-override coverage on the lobotomized side.

Human decisions to make:

• Scope the actual coverage gaps in each leaf (read current leaf test suites first).
• Decide which gaps are worth leaf PRs vs. left to the integration gate's runtime altitude (ADR 0001 — don't pull leaf checks into the control plane).
• Coordinate the PRs with skrabe.

Acceptance criteria:

• Coverage gaps scoped per leaf with a decision on which become PRs.
• Agreed PRs prepared against the leaf repos (not merged by the control plane).

Out of scope:

• Adding leaf-level check logic into tweakcc-maint (ADR 0001 violation).
• Direct-pushing to any leaf.

[dividedby]

This was generated by AI during triage.

Agent Brief

Category: enhancement
Summary: Scope the leaf-check coverage gaps and prepare verified leaf test PRs that broaden tweakcc-fixed + lobotomized-claude-code so a version bump cannot silently break patches/prompts.

Current behavior:
Leaf test coverage is thin. Per ADR 0001 the escape/regex logic and prompt extraction live in the leaves, but there are no golden extraction snapshots and limited unit coverage, so a CC version bump can break a patch or prompt without a leaf test going red.

Desired behavior:

1. A written gap inventory (posted as a comment) enumerating the missing coverage: escape/regex unit-test gaps and golden extraction snapshots in tweakcc-fixed's vitest, plus per-override coverage on the lobotomized side.
2. Verified leaf test PR(s) that close the highest-value gaps — unit tests around the escape/regex logic and at least one golden extraction snapshot — prepared against the leaf repos.

Key interfaces:

• tweakcc-fixed vitest suite — add unit tests for the escape/regex helpers and golden snapshots of extractor output for a pinned in-matrix version.
• lobotomized-claude-code — per-override assertions (e.g. each override's declared variables resolve against its matched prompt's identifierMap).

Acceptance criteria:

• A gap-inventory comment lists the concrete missing tests, grouped by leaf, ranked by value.
• At least the top-ranked gaps are covered by new tests; pnpm test green on each touched leaf (no new failures).
• Golden extraction snapshot(s) exist for tweakcc-fixed against a pinned in-matrix CC version.
• Verified PR(s) prepared to the leaf repos. Do not merge — merge is skrabe's.

Out of scope:

• Changing patch/extraction logic itself (tests only; if a test reveals a real bug, file it separately).
• tweakcc-maint control-plane tests (this is leaf work per ADR 0001).
• Direct pushes to leaf main.

[dividedby]

Coverage-gap inventory (per leaf, ranked) + first PR

Scoped the two leaf suites at their current main. Headline: the two leaves are at opposite ends — tweakcc-fixed has a mature vitest suite with specific untested seams; lobotomized-claude-code has no test runner at all, but its highest-value check (per-override slot binding) already exists at the patcher altitude, so it should not get a parallel harness.

Leaf A — tweakcc-fixed (vitest present)

Ranked gaps:

1. [SHIPPED] Escape/identifier customization helpers — zero direct tests. The exported buildRegexFromPieces (regex-metachar + ${VAR}-literal escaping), extractUserCustomizations + buildHumanToRealMapping (round-trip / conflict / unmapped-skip), reconstructContentFromPieces, and the extractOriginalWhitespace/applyOriginalWhitespace pair. → test: broaden leaf coverage — escape/identifier units + 2.1.169 golden snapshot skrabe/tweakcc-fixed#6.
2. [SHIPPED] No golden over extractor output. Added a pinned digest over prompts-2.1.169.json (count, unique-id sha, identifier-value surface, dup-id groups, unmapped-identifier prompts) so a bump can't silently drop/rename/re-shape a prompt. → same PR.
3. [deferred — needs an export decision] Deep escape internals. escapeNonAsciiForRegex, hexAlt, buildSearchRegexFromPieces, applyIdentifierMapping, countPrecedingBackslashes are not exported — unit-testing them means a source change (export-for-test), which is skrabe's call, not test-only scope.
4. [lower value] I/O-bound seams (systemPromptDownload GitHub fallback/backoff, loadSystemPromptsWithRegex glob/union) — would need network mocking for modest payoff; leave to the integration gate's runtime altitude (ADR 0001).

Candidate findings surfaced by the golden (captured in the snapshot, not asserted away — flagged for skrabe on #6, not fixed here):

• 4 duplicate-id groups in 2.1.169 (system-reminder-cross-session-peer-message-authority-warning appears 4×).
• 3 skill-code-review-phase-* prompts whose identifier index has no identifierMap entry → reconstruct as UNKNOWN_<idx>.

Leaf B — lobotomized-claude-code (no harness) — recommend NOT standing one up

The brief's per-override coverage idea (each override's ${NAME} resolves against the matched prompt's identifierMap) is real, but it is already implemented: tweakcc-fixed/tools/auditMisbinds.mjs is the slot-binding gate — for every override, every placeholder must sit at the same identifierMap slot as upstream's authoritative map (skrabe confirmed on lobotomized#5; his bar is four-zeros + auditMisbinds=0). Adding a parallel vitest harness in the data-only lobotomized repo would duplicate it and pull a leaf check into the wrong place (ADR 0001).

The genuine residual gaps are patcher-altitude, and skrabe-owned, not lobotomized PRs:

• auditMisbinds is a manual showtime tool (needs an upstream dump), not a CI gate.
• The quote-vs-backtick escape check is still tribal knowledge — skrabe said he'll fold it into auditMisbinds (or a sibling) as a runnable gate.

Decision: Leaf-A gaps → PR (done, #6). Leaf-B per-override coverage → not a lobotomized PR; it's covered by auditMisbinds and the only forward work (gate-ify it + quote-context check) sits in tweakcc-fixed under skrabe. No further leaf PR prepared on the lobotomized side.

Acceptance criteria status

• Gap inventory, grouped by leaf, ranked by value.
• Top-ranked gaps covered by new tests; pnpm test green (343 passed, +17) on tweakcc-fixed.
• Golden extraction snapshot for a pinned in-matrix version (2.1.169).
• Verified PR prepared to the leaf — test: broaden leaf coverage — escape/identifier units + 2.1.169 golden snapshot skrabe/tweakcc-fixed#6 (not merged; merge is skrabe's). Lobotomized side resolved as covered-by-auditMisbinds (no PR needed).