docs: add REVIEW.md review-severity rules - #69
Merged
Conversation
shigechika/ai-review v1.5.0 supports a repository-root REVIEW.md that overrides the reviewer's default focus and severity calibration. The contract, and the drafting hazards found while piloting it, are in the "Writing a REVIEW.md" section of that repository's README. Distilled from .github/copilot-instructions.md, carrying rules and severities only and citing that file's numbered focus items for the reasoning. Identifiers verified against the code first: SenderManager, set_version_tag and _is_plaintext_remote in sender.py, take_snapshot and stamp_version in runner.py, _extract_by_label in sites/cloudflare.py, and both ruff gates in ci.yml. The run_* containment rule leads because its blast radius is the surprising part: cli.py's loop does not wrap each call, so an escaping exception aborts every remaining site in the batch rather than failing one test. The rule also warns against assuming a uniform block shape, since ookla.py snapshots inside its own retry loop rather than a single finally — checking for one specific shape would produce a false finding there. This is a browser-driving CLI, not an MCP server, so MCP-flavored advice is suppressed outright. REVIEW.md is read at the base revision, so this pull request is still reviewed without it; the rules take effect from the next one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015NkkQZiesYXG7gEUdbYGcm
AI review (gpt-5.6-sol)Verdict: no blocking findings. Scope: full PR diff. All candidate findings this round were refuted by the verifier pass — no new findings. Findings ledger (all rounds)
Advisory per-push review (round 1) generated by ai-review — verify findings before acting. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a
REVIEW.md— the repository-specific review override supportedby
shigechika/ai-reviewsince v1.5.0. It overrides the reviewer'sdefault focus and severity calibration for this repository.
Part of the fleet rollout tracked in
shigechika/ai-review#52. Thecontract, and the seven drafting hazards found while piloting it on two
repositories, are documented in the "Writing a
REVIEW.md" section ofthat repository's README.
What is in it, and what deliberately is not
A severity and scope layer: which findings are blocking here, which
classes to report that the reviewer's default focus would otherwise
skip, and which are noise. The reasoning stays in
.github/copilot-instructions.mdandCLAUDE.md, both of which thereviewer already receives as guidance, and this file cites them rather
than restating them. That is a correctness point, not tidiness: the
override wins by design, so a duplicated explanation would let a stale
sentence here out-rank a corrected one there indefinitely.
Nothing is invented — it is distilled from the review material this
repository already had, and every identifier and CI claim was checked
against the code before being written down.
Applying the piloted hazards
The pilots produced thirteen findings against files that were supposed
to be straightforward distillations. The two that bite hardest, and how
this file handles them:
There is no checkout: it gets the diff, the changed files at head
(subject to count and byte caps, deny-listed ones dropped), guidance
at base, and a truncated PR description. So any coverage-style rule
here is judged from the diff alone — absent from the prompt is not
evidence of absent from the repository.
write to stdout" turns a deliberate CLI print into a blocking finding
on correct code, and since the override outranks guidance, the
guidance explaining the distinction cannot rescue it.
When this takes effect
REVIEW.mdis read at the base revision by design, so a pullrequest cannot rewrite the rules it is reviewed against. This pull
request is therefore still reviewed without it — the rules apply from
the next pull request onward. A review here that looks unchanged is
expected.
To confirm it is picked up afterwards, look for
::notice::REVIEW.md: sent N of M bytesand a non-zeroreview_override=in the context line of thereviewjob log.🤖 Generated with Claude Code
https://claude.ai/code/session_015NkkQZiesYXG7gEUdbYGcm