Skip to content

QM-56: Hand the test-economy rules to every prompt that can write tests - #1433

Merged
pcapriolo-yc merged 1 commit into
factory-in-qmfrom
qm-56-s628008306
Sep 19, 2026
Merged

pcapriolo-yc merged 1 commit into
factory-in-qmfrom
qm-56-s628008306

Conversation

@pcapriolo-yc

Copy link
Copy Markdown
Contributor

Closes QM-56.

Changes

Why this matters: The factory's test-economy rules (fewer, better tests; the smallest test that proves the change) only reached the add-tests agent and the mutation-gate repair agent. But the agent that writes a run's code — implement on a normal pass, feedback-revise on a feedback re-run — usually writes the tests too, and it received no economy constraint at all. By the time the rules arrived, the bloat was already in the tree, and every later pass is additive rather than pruning. On QM-42 a feedback re-run produced 17 tests and 483 lines in a single file, including a 124-line fixture, before any agent had seen the rules.

What changes:

  • The feedback-revise and implement agent prompts now carry TEST_QUALITY_RULES and TEST_HOWTO, so every agent that can write tests is held to the same standard before it writes anything.
  • The review-kickback re-implement pass is covered by the same insertion, because it reuses the implement prompt and the rules sit outside the kickback branch.
  • The rule text itself is unchanged and still defined in one place; no new constant, flag, or conditional was added.
  • A guard in test/factory-layout.test.ts fails, naming the offending prompt, if feedback-revise, implement, or add-tests ever stops interpolating the rules.

Acceptance stories:

  • On a feedback re-run, the Factory's feedback-revise agent used to get no test-economy rules; it now gets them before it writes its first test.
  • On a normal pass and on a review kickback, the implement agent gets the same rules on both branches, where before it got none on either.
  • The add-tests and mutation-gate repair agents keep the rules they already had, and the proof agent is untouched.
  • A maintainer who removes the interpolation from any of the three prompts gets a failing test naming that prompt instead of a silently unenforced standard.

Test Plan

  • NODE_ENV=test ALLOW_UNSIGNED_TEST_IDENTITY=1 node --experimental-test-module-mocks --test test/factory-layout.test.ts
  • npm run typecheck && npm run lint
  • Manual: removed each of the three interpolations in turn, confirmed the guard failed naming that exact prompt, and restored the tree.

Proof it works

The new guard passes on this branch and fails with the offending label named when either insertion is reverted, so the rules are provably delivered to both newly covered agents. This change alters only the prompt text handed to background agents, so it has no visible surface and produced no screenshots or recordings.

@pcapriolo-yc
pcapriolo-yc merged commit e1306d4 into factory-in-qm Sep 19, 2026
16 checks passed
@pcapriolo-yc
pcapriolo-yc deleted the qm-56-s628008306 branch September 19, 2026 16:22
pcapriolo-yc added a commit that referenced this pull request Sep 19, 2026
* Ship the factory wrapper inside this repository

The software factory's instruction book (the sandbox wrapper, the three
work-ticket workflows, their prompts and skill, and the tools scripts) moves
from the private deployment repository into factory/. The loop's work stage
now fetches this repository at the commit the running core was built from
(GIT_SHA) into /workspace/qm-source and starts factory/.claude/io-coding-agent-js.sh
from it, so the wrapper and the loop always come from the same commit. A core
with no full build sha fetches main. Cold and warm sandboxes share one
fetch-then-checkout script.

Nothing specific to the monorepo the factory was carved out of remains:
the Rails app table and stack start path, the proof icon fonts, the
X-Current-User proof auth, the hard-coded publish project and GitLab bot
user ids, and the IO- ticket prefix in the linkback scan are gone or
configuration-driven. Browser proof runs only against a configured stack
(IO_PROOF_START_CMD / IO_PROOF_BASE_URL_CMD) and says so honestly otherwise.

factory/ is excluded from eslint, knip, and prettier: the workflows run inside
the Claude Workflow sandbox with its globals and are kept byte-stable.
test/factory-layout.test.ts pins the file inventory, executable bits, shell
syntax, the wrapper handshake, the tool preflight, and the absence of every
monorepo literal.

* Classify factory/ as implementation when no source dirs are configured

The mutation gate and the source helper fall back to a built-in directory
list when IO_SOURCE_APP_DIRS is unset. That list predates factory/, so a
change to the wrapper was unclassifiable and the gate failed closed. The
first factory-on-factory run found this and ledgered it.

* QM-56: Hand the test-economy rules to every prompt that can write tests (#1433)

Co-authored-by: Sprite <noreply@sprites.dev>

---------

Co-authored-by: Sprite <noreply@sprites.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant