Skip to content

docs: add REVIEW.md review-severity rules - #69

Merged
shigechika merged 1 commit into
mainfrom
docs/add-review-md
Aug 13, 2026
Merged

docs: add REVIEW.md review-severity rules#69
shigechika merged 1 commit into
mainfrom
docs/add-review-md

Conversation

@shigechika

Copy link
Copy Markdown
Owner

Summary

Adds a REVIEW.md — the repository-specific review override supported
by shigechika/ai-review since v1.5.0. It overrides the reviewer's
default focus and severity calibration for this repository.

Part of the fleet rollout tracked in shigechika/ai-review#52. The
contract, and the seven drafting hazards found while piloting it on two
repositories, are documented in the "Writing a REVIEW.md" section of
that repository's README.

What is in it, and what deliberately is not

A severity and scope layer: which findings are blocking here, which
classes to report that the reviewer's default focus would otherwise
skip, and which are noise. The reasoning stays in
.github/copilot-instructions.md and CLAUDE.md, both of which the
reviewer already receives as guidance, and this file cites them rather
than restating them. That is a correctness point, not tidiness: the
override wins by design, so a duplicated explanation would let a stale
sentence here out-rank a corrected one there indefinitely.

Nothing is invented — it is distilled from the review material this
repository already had, and every identifier and CI claim was checked
against the code before being written down
.

Applying the piloted hazards

The pilots produced thirteen findings against files that were supposed
to be straightforward distillations. The two that bite hardest, and how
this file handles them:

  • Rules must be decidable from what the reviewer actually receives.
    There is no checkout: it gets the diff, the changed files at head
    (subject to count and byte caps, deny-listed ones dropped), guidance
    at base, and a truncated PR description. So any coverage-style rule
    here is judged from the diff alone — absent from the prompt is not
    evidence of absent from the repository.
  • Rules must be scoped to the code they govern. An unscoped "never
    write to stdout" turns a deliberate CLI print into a blocking finding
    on correct code, and since the override outranks guidance, the
    guidance explaining the distinction cannot rescue it.

When this takes effect

REVIEW.md is read at the base revision by design, so a pull
request cannot rewrite the rules it is reviewed against. This pull
request is therefore still reviewed without it
— the rules apply from
the next pull request onward. A review here that looks unchanged is
expected.

To confirm it is picked up afterwards, look for
::notice::REVIEW.md: sent N of M bytes and a non-zero
review_override= in the context line of the review job log.

🤖 Generated with Claude Code

https://claude.ai/code/session_015NkkQZiesYXG7gEUdbYGcm

shigechika/ai-review v1.5.0 supports a repository-root REVIEW.md that
overrides the reviewer's default focus and severity calibration. The
contract, and the drafting hazards found while piloting it, are in the
"Writing a REVIEW.md" section of that repository's README.

Distilled from .github/copilot-instructions.md, carrying rules and
severities only and citing that file's numbered focus items for the
reasoning. Identifiers verified against the code first: SenderManager,
set_version_tag and _is_plaintext_remote in sender.py, take_snapshot
and stamp_version in runner.py, _extract_by_label in
sites/cloudflare.py, and both ruff gates in ci.yml.

The run_* containment rule leads because its blast radius is the
surprising part: cli.py's loop does not wrap each call, so an escaping
exception aborts every remaining site in the batch rather than failing
one test. The rule also warns against assuming a uniform block shape,
since ookla.py snapshots inside its own retry loop rather than a single
finally — checking for one specific shape would produce a false finding
there.

This is a browser-driving CLI, not an MCP server, so MCP-flavored
advice is suppressed outright.

REVIEW.md is read at the base revision, so this pull request is still
reviewed without it; the rules take effect from the next one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015NkkQZiesYXG7gEUdbYGcm
@github-actions github-actions Bot added the ai-review Admitted for review by pr-gate.yml label Aug 13, 2026
@github-actions

Copy link
Copy Markdown

AI review (gpt-5.6-sol)

Verdict: no blocking findings.

Scope: full PR diff.

All candidate findings this round were refuted by the verifier pass — no new findings.

Findings ledger (all rounds)
  • R1F1 [dismissed/advisory] — Cited fixtures do not provide suite-wide WebDriver mocking

Advisory per-push review (round 1) generated by ai-review — verify findings before acting.

@shigechika
shigechika merged commit f07fe8f into main Aug 13, 2026
13 checks passed
@shigechika
shigechika deleted the docs/add-review-md branch August 13, 2026 01:55
@github-actions github-actions Bot removed the ai-review Admitted for review by pr-gate.yml label Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant