A comment-only edit is impossible in ANY of the 43 scripts/check-*.py checkers, so a false comment in one cannot be corrected

[localai-bot]

A comment inside a governance_checker can never be corrected on its own, because check-pr-size.py demands semantic mutation evidence that a comment-only edit cannot produce.

Found while landing #1578, correcting records that the floor repair for #1629 made stale.

Measured, not inferred

scripts/check-pr-size.py:372 carries a comment reading Measured: 31 of 31 red under the stub. After #1629's repair the suite is 32 cases and the same mutation measures FAILED (errors=32). Rewording that one comment was attempted and refused:

$ python3 scripts/check-pr-size.py --base 89925ad6f8... --head <comment-only commit>
ERROR: checker change 'scripts/check-pr-size.py' requires semantic mutation evidence in tests/scripts/test_check_pr_size.py
rc=1

The identical invocation against the parent commit exits 0, so the edit is the only difference.

Mechanism

check-pr-size.py classifies itself as a governance_checker. change_errors therefore requires any change to it to be paired with a tests/scripts/test_check_pr_size.py change that executable_evidence proves goes red against the BASE checker. A comment-only edit leaves BASE and HEAD semantically identical, so no test can distinguish them and no such evidence can exist. The required pr-size CI job runs exactly this invocation.

Consequence: every comment in that file is frozen until someone changes the checker's behaviour in the same commit. A stale comment in a governance checker is therefore permanently stale, and the only way to fix a factual error in one is to attach it to an unrelated semantic change — which is worse than leaving it.

This is the shape AGENTS.md ## Changing the rules or a checker warns about from the other direction: the guard is right to refuse a silent semantic weakening, but it currently cannot tell "weakened an assertion" from "fixed a typo in a comment".

Not fixed in flow

Teaching the guard to distinguish a comment-only or docstring-only diff from a semantic one changes what the gate accepts, so under ## Changing the rules or a checker it needs its own row, a red-before test and green-after evidence. Two candidate directions, neither chosen: compare the two versions with comments and docstrings stripped (an AST or token comparison) and skip the evidence requirement when they are identical; or keep the requirement and stop storing facts that go stale inside checker comments. The second is cheaper and matches ## Records, which already forbids storing a measurement of one file inside another.

The stale comment itself is being left in place in #1578 rather than smuggled in, and this issue records why.

Two smaller things in the same file, both pre-existing

• scripts/check-pr-size.py:370-371 — the comment lists the suite's entry points as scan_file / has_marker / drift_sites / stale_allowlist_entries / main, but the suite also calls scan_models, allowlisted_names, marker_reason and now MODEL_DIRS. The list was already incomplete before check-attention-rung-consistency's population floor is pinned at today's exact count, so every allowlisted row reds it by removing its naive call #1629's repair. The mechanism claim it supports (raises AttributeError) still holds. Same edit lock applies.
• scripts/check-pr-size.py:378 — SELF_CHECKER = "scripts/check-pr-size.py" is defined and never read anywhere in the file. Dead constant.

[localai-bot]

Correction: this is not one file. It is EVERY checker in the repository.

I filed this against scripts/check-pr-size.py's own comments. That was too narrow, and the difference matters.

scripts/check-pr-size.py:170 classifies by pattern, not by identity:

CHECKER = re.compile(r"scripts/check-[a-z0-9]+(?:-[a-z0-9]+)*\.(?:py|sh)\Z")

Every path matching that becomes a governance_checker, and change_errors then demands executable red-before evidence: the BASE checker plus the HEAD tests must go RED. A comment-only edit leaves base and head semantically identical, so no such evidence can exist for any of them. The repository currently has 43 scripts/check-*.py files plus the .sh checkers, and a factually wrong comment in any of them cannot be corrected on its own.

Measured on a real attempt, not inferred. Correcting three demonstrably false comments in scripts/check-attention-rung-consistency.py:

ERROR: BASE checker stayed green for 'scripts/check-attention-rung-consistency.py'; changed test is not semantic evidence
rc=1

classify_path('scripts/check-attention-rung-consistency.py') -> governance_checker.

The live cost, right now

Three comments in scripts/check-attention-rung-consistency.py are measured false and are shipping in #1578 unrepaired because of this rule. They assert that the \b in r"\bvt::Attention\s*\(" is what stops the pattern matching the fast rungs:

• :57-60, :91-94 — "The word boundary is load-bearing: without it this matches vt::AttentionDenseFlash( ... every FAST rung"
• :250-253 — same non-cause

The trailing \( is what excludes the fast rungs. vt::AttentionDenseFlash( does not match with OR without the \b. Removing the \b changes exactly one behaviour, a prefixed identifier such as xyvt::Attention(, and the two patterns were shown to disagree on exactly the 63 identifier characters over printable ASCII. With the \b removed the whole suite stays green (Ran 34 tests ... OK), so the companion claim that "the widening is caught in this suite" is false too.

The equivalent claims in the TEST file were repaired in #1578, because tests/scripts/test_*.py is not classified as a governance checker. So the tree now says one thing in the test and the opposite in the checker beside it — a direct consequence of this rule, and the sharpest possible illustration of the cost.

A prepared patch for the three checker-side comments exists and was deliberately not committed rather than ship a red pr-size.

Why the guard is still right, and what is actually wrong

Refusing a silent semantic weakening is correct and must not be removed. The defect is narrower: the guard cannot tell "weakened an assertion" from "fixed a false sentence in a comment", so it forces the two into the same lane and makes the second impossible.

Two candidate directions, neither chosen here, both needing a row, spec and red-before evidence under ## Changing the rules or a checker:

1. Compare base and head with comments and docstrings stripped (token-stream or AST comparison) and waive the evidence requirement only when the executable text is identical. ast.dump over both revisions is sufficient for .py and is already used as evidence elsewhere in this work; .sh would need a different mechanism or would keep the current behaviour.
2. Keep the requirement and stop putting falsifiable statements in checker comments at all. Cheaper, but it does not fix the 43 files that already have them, and it cannot be enforced.

Direction 1 is the one that actually discharges this. Note it must NOT waive the requirement when a docstring changes in a way the checker reads at runtime — a docstring is an AST node, so an AST comparison catches that correctly, whereas a naive comment-stripping regex would not.

[localai-bot]

The prepared checker-side patch, parked here so it outlives the scratchpad

These are the three measured-false comment repairs that could not be committed in #1578, because check-pr-size.py refuses a comment-only edit to any scripts/check-*.py. Recorded verbatim so whoever fixes this issue does not have to re-derive them. It applies to scripts/check-attention-rung-consistency.py at sha256 098b50255c8353aa798b1334bfd6be4e29013392deecf7e62fcca33aa41d17f6, which is the version that landed.

diff --git a/scripts/check-attention-rung-consistency.py b/scripts/check-attention-rung-consistency.py
index b29629571..3f12caa39 100755
--- a/scripts/check-attention-rung-consistency.py
+++ b/scripts/check-attention-rung-consistency.py
@@ -56,13 +56,11 @@ green with a live unmarked call:
 
 None exists in this tree, and the repository does not write attention calls that
 way, so this is a stated bound and not a live hole. Widening the regex is not the
-repair: `\bAttention\s*\(` also matches every fast rung's suffix-free form and
-would demand a marker beside exactly the calls this checker wants people to make,
-and no regex reaches a call through a function pointer at all. What closes it is a
-compiler-side population — the CUDA op registry, or a clang tooling pass over the
-real translation unit — which is a different instrument, not a longer pattern. A
-green here therefore means "no unmarked `vt::Attention(` call", never "no model is
-on the naive rung".
+repair: no regex reaches a call through a function pointer at all. What closes it
+is a compiler-side population — the CUDA op registry, or a clang tooling pass over
+the real translation unit — which is a different instrument, not a longer pattern.
+A green here therefore means "no unmarked `vt::Attention(` call", never "no model
+is on the naive rung".
 
 The validation logic is pure functions (`scan_file`, `drift_sites`,
 `stale_allowlist_entries`) so it is unit- and mutation-testable
@@ -90,10 +88,9 @@ MODEL_DIRS = (
 )
 ALLOWLIST = ROOT / "scripts/attention-rung-allowlist.txt"
 
-# `vt::Attention(` and nothing else. The word boundary is load-bearing: without it
-# this matches `vt::AttentionDenseFlash(`, `vt::AttentionDenseFast(`,
-# `vt::AttentionDenseFa2(` and `vt::AttentionCross(` — every FAST rung — and the
-# checker would demand a marker beside exactly the calls it wants people to make.
+# `vt::Attention(` and nothing else. The trailing `\(` is what excludes the fast
+# rungs; `vt::AttentionDenseFlash(` does not match with or without the `\b`. What
+# the `\b` excludes, measured: a leading identifier char, as in `xyvt::Attention(`.
 _NAIVE_CALL = re.compile(r"\bvt::Attention\s*\(")
 
 # The marker, in a `//` comment. A reason is required after the colon.
@@ -250,9 +247,8 @@ def main() -> int:
     sites = sum(len(v) for v in scanned.values())
     marked = sum(1 for v in scanned.values() for _, m in v if m)
     # The number a reader of a green actually needs: sites that carry NO reason and
-    # pass only because their stem is allowlisted. `sites - marked` is not it, because
-    # a marked call inside an allowlisted file counts in `marked`. This is the debt the
-    # green is hiding, so it is printed even when it is zero.
+    # pass only because their stem is allowlisted. This is the debt the green is
+    # hiding, so it is printed even when it is zero.
     excused = sum(
         1
         for path, sites_in_file in scanned.items()

[localai-bot]

Anchor correction: SELF_CHECKER is at :376, not :378

The body above cites the unread SELF_CHECKER constant at scripts/check-pr-size.py:378. Measured at PR #1578's head it is :376.

The issue was filed partway through that pull request and the file gained lines afterwards, so the anchor went stale inside the same pull request that wrote it — the recorded-line-anchor drift AGENTS.md warns about, arriving over a couple of hours rather than across releases. It is a small thing on its own and a useful reminder that an anchor is only true of the tree it was read on.

The .agents/issue-index.md row for this issue records the measured :376 and notes that this body says :378, so a reader who follows the link and finds a different number knows which one was measured rather than treating one of the two as an error.

The other anchors cited here were each re-read at that head and are correct: scripts/check-attention-rung-consistency.py:58-61, :93-96, :252-255, and scripts/check-pr-size.py:370-371.