Review prompts: swap the lens list for enumerable moves and an absence pass - #21
Closed
latekvo wants to merge 1 commit into
Closed
Review prompts: swap the lens list for enumerable moves and an absence pass#21latekvo wants to merge 1 commit into
latekvo wants to merge 1 commit into
Conversation
…e pass The depth fragments told the swarm to cover "the standard lenses - correctness, scope/simplification, edge cases and ripple effects". Those name kinds of bug, so an agent could report a lens swept and still have missed everything in it. On argent#574 a full sweep at max depth returned two test-scaffolding notes and a body claiming the concurrency, ripple and device-scoping lenses "held up throughout"; a human reviewing the same source an hour later found three real defects in exactly those lenses. The fragments now fan out over review moves - acts, not categories - matching the rewritten CLAUDE.md: claims vs code, nearest twin, non-happy paths, inputs, reachability both ways, and what outlives the call, plus a separate absence pass. The moves were induced from 231 real findings across 27 argent PRs, and one agent per move keeps the swarm inside the sizing limits rather than leaving the count implicit. Also fixed, all found while checking this change against its own gates: - softApprove's example was "Ran full E2E sweep across the review lenses. Returned perfectly clean." That is the exact unearned all-clear that fired on #574, and it contradicted CLAUDE.md's no-LGTM rule while claiming to be its one sanctioned exception. It now has to state the moves covered and the reproductions run, and bare impressions are named and banned. - finalPass asked for "massive swarms", contradicting the hard cap of 14 in CLAUDE.md. It now defers to those limits. - leaveReviews' no-duplicate rule had no floor, so a PR body that pre-lists known limitations read as prior coverage. An acknowledged limitation is now explicitly not a resolved finding. - The standard/deep blurbs still advertised "review lenses". Golden prompts regenerated with DIPLOMAT_GOLDEN_WRITE=1 swift run DiplomatCoreSmoke.
latekvo
marked this pull request as ready for review
July 28, 2026 16:53
latekvo
marked this pull request as draft
July 28, 2026 17:10
Owner
Author
|
bollocks |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The depth fragments fanned the swarm out over kinds of bug - "correctness, scope/simplification, edge cases (nulls, boundaries, errors, concurrency) and ripple effects on callers/docs/tests". That list is open-ended, and its coverage cannot be audited: an agent can report a lens swept and still have missed everything in it.
It did. On argent#574 a max-depth sweep returned two test-scaffolding notes plus a body claiming the concurrency, ripple/call-site and device-scoping lenses "held up throughout". A human reviewing identical source an hour later (the only intervening commit touched one test file) found three real defects, all in those lenses, all accepted and fixed: a writer-only lock whose readers can observe a truncated flow file and report a top-level PASS having replayed zero steps; a second device to services matcher disagreeing with the existing one on case-sensitivity and namespace set; and an
unmatchedfield reporting real device ids as typos.Approach
The fragments now fan out over review moves - acts a reviewer performs, one agent per move - matching the rewritten gates in CLAUDE.md:
claims vs code · nearest twin · non-happy paths · inputs · reachability both ways · what outlives the call
plus a separate absence pass, because roughly half of all real findings are things that are missing and you cannot grep for what does not exist. Absence only shows against a reference, so the pass names the four that work: a sibling that has it, prose that promises it, symmetry (spawn/reap, open/close, write/delete, set/reset), and mutation.
The moves are not invented. They were induced from a corpus of 231 real findings across 27 argent PRs: two agents given only the corpus, with no sight of any candidate framework, produced near-identical taxonomies. Three adversarial passes then broke the first draft, which scored only ~40-50% coverage and had no hook for absence at all. Mutation alone accounts for 33-41 of the 231 and had no home in the old lens list.
Naming one agent per move also keeps the swarm inside CLAUDE.md's sizing limits instead of leaving the count implicit.
Also fixed
Each of these was found by running this change against its own gates.
softApprove's example was the bug. It suggested posting "Ran full E2E sweep across the review lenses. Returned perfectly clean." - the exact unearned all-clear that fired on #574 - while also contradicting CLAUDE.md's no-LGTM rule it claims to be the one sanctioned exception to. It now has to state the moves covered and the reproductions run, and names bare impressions as banned.finalPassasked for "massive swarms", contradicting the hard cap of 14 in CLAUDE.md. Defers to those limits now.leaveReviews' no-duplicate rule had no floor, so a PR body pre-listing known limitations read as prior coverage. An acknowledged limitation is now explicitly not a resolved finding.standardanddeepblurbs still advertised "review lenses" after the fragments stopped using them.Verification
DIPLOMAT_GOLDEN_WRITE=1 swift run DiplomatCoreSmoketo regenerate; the three affected goldens are in the diff. Re-run without the env var:SMOKE OK, golden-prompt assertions pass across all 10 modes.swift build --product diplomat-coreclean.pytest linux/testswithDIPLOMAT_CORE_BINandQT_QPA_PLATFORM=offscreen: 195 passed, includingtest_golden_prompts.py(byte-for-byte Swift/Python parity) andtest_review_author.py. The mesh suite is excluded locally - it blocks ongetaddrinfoin a sandbox with no DNS - and is left to CI.review.jsonor the goldens.Notes for reviewers
Diplomat.app/Contents/Resources/core/review.jsonis a build artifact copied byscripts/build-app.shand is untracked; onlycore/review.jsonis edited here.