gate(GATE-CONFLICT-MARKERS): refuse a tracked file that carries a merge conflict marker - #1450
Merged
Conversation
…eads Four record gates return rc 0 on a document carrying literal conflict markers. The measurement was re-derived on a detached scratch worktree rather than quoted: five lines spliced into the docs/STATUS.md capability table leave check-public-doc-tables, check-agent-record, check-doc-checkpoint and check-issue-index-append-only all green. The two range-scoped checkers were run over a scratch commit, because a working-tree mutation of a commit-reading checker returns 0 without reading the mutated bytes. This spec commits before the implementation, as the protocol requires for a semantic checker change. It records the detection rule, why the bare separator stays conditional, why the tree needs no allowlist, and the stop condition that refuses an allowlist every change would have to append to. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…flict marker Four record gates return rc 0 on a docs/STATUS.md carrying a spliced conflict hunk. Each measures its own budget, and none asks whether a merge tool wrote into the file, so a keyed row that is half one branch and half another satisfies every budget it is measured against. A person caught the real instance by reading git diff --stat. The new checker reads every tracked path once and refuses a line that starts with seven angle brackets and a space. A line of exactly seven equals signs is refused only inside an open hunk, because a bare rule of equals signs is a setext heading underline and five shipped files already carry one longer than seven. A gate that fires on ordinary work is the defect, not the discipline. The tree needs no allowlist. Every pattern is built by character repetition, so the checker and its suite carry no marker at column 0 and need no exemption from the gate they implement. An allowlist would be a file that every change must append to, which is the shared-file lock AGENTS.md forbids. The suite gained a case after its own mutation run: dropping the adjacency guard left every case green, because a file with no marker leaves the scan early and the guard never decided anything there. The new case asserts the report rather than the exit code, which is 1 either way. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
origin/main advanced to 9ecaf1b while this row was in flight. The merge is clean and touches no file this row owns: the incoming change is the CPU attention backend registry repair, which lands in src/, include/ and its own tests. The record gates and the row's own suite were re-run after the merge. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…ry (#1448) classify_path fails closed, and BENCH_EVIDENCE matches exactly one path segment, so the ten files of docs/bench-evidence/gdn-replayssm-w0-20260818/ have been unclassified since 2026-08-18. The sweep case in tests/scripts/test_check_pr_size.py classifies every tracked path, so that suite is red on origin/main today, verified in a detached worktree at 9ecaf1b. It sits in no preflight SUITES entry, so preflight stays green over it and the red surfaces only through the checker-evidence contract, where it reads as a defect in the change under review. Repaired here because registering a new checker requires an entry in CREATION_MUTATIONS, so this row could not prove its own evidence contract while the sweep was red. Fourth instance of the class after #856, #668 and #989, and repaired the same way: name the surface, do not widen a rule. The extension list is exactly what the directory carries and deliberately omits .md and .json. Those already classify as public_document through DOC, and the evidence arm is tested first, so admitting them would silently reclassify docs/bench-evidence/mxfp4-qwen/*. Classification of all 4811 tracked paths was captured before and after: exactly the ten move, and nothing else changes. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
origin/main advanced twice while this row was in flight, to 9ecaf1b and then to f07f96e. Both merges are clean and neither touches a file this row owns. The record gates, the row's own suite and the checker-evidence contract were re-run after each merge. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
… clean count to git A fresh review of #1450 returned PASS with two LOW findings, and both bear on the principle this row elevates: a gate that cannot say how many things it examined has not reported. git ls-files emits stages 1, 2 and 3 for a conflicted path, so a live merge made the checker read one file three times and report 9 findings in 1 file over 3 examined text files. The verdict stayed 1, so nothing was ever missed, but the count was wrong in exactly the state the gate exists for, and no case built an unmerged index. tracked_paths now deduplicates, and a new case makes git produce a real conflict and asserts the three stage entries before it asserts the count, so the dedupe is falsifiable rather than merely present. The shipped-tree case asserted an examined count above one thousand against a real 3733, so a scan that collapsed to markdown alone would have read green. It now asserts EQUALITY with git's own tracked text set, derived at read time from git grep plus the tracked files that have no lines. That stores no number, so it is not the drift lock the protocol forbids, and it agreed byte for byte with the binary heuristic on this tree. Three smaller repairs came from the same review. An unreadable file was filed as a finding, so an I/O error exited 1 and printed the merge remedy; it now takes its own list and exits 2, because a file this run could not read is a file it cannot call clean. The summary file count split each report line on its first colon and merged two paths that share a prefix. The creation-mutation comments claimed the disabled stub fails every case that runs the checker; it fails 16 of 21, and the five survivors assert only that an ordinary document exits 0. The spec records the unmerged index as NOT SEEN when it was written, rather than as a trade-off that was considered and dropped. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
origin/main moved three times while this row was in flight, to 9ecaf1b, then f07f96e, and now to the head this merge brings in. Every merge was clean and none touched a file this row owns. The record gates, the row's own suite, the checker-evidence contract and both trailer gates were re-run afterwards, so they EXECUTE on this tree rather than skipping because the base moved ahead. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
localai-bot
pushed a commit
that referenced
this pull request
Aug 20, 2026
…ew conflict-marker gate `origin/main` gained `check-conflict-markers` (#1450) after the review repairs were pushed. The merge is disjoint on files, and it is taken rather than left to the forge so the new gate runs against this branch's own tree here: a checker that lands mid-review is the case where "CI will catch it" is a guess. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
localai-bot
pushed a commit
that referenced
this pull request
Aug 20, 2026
`#1450` landed the conflict-marker gate and appended to `.agents/issue-index.md`, the one file this branch also touches. The merge is clean: that path carries `merge=union` in `.gitattributes`, so both appends land and no other row moves. Merged locally rather than left for the forge, which does not apply the driver and reports a phantom conflict on exactly this shape. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
localai-bot
added a commit
that referenced
this pull request
Aug 20, 2026
…bove its replacement (#1442) (#1449) Two public pages and one row record all told a reader the streaming-ON decode figure was void, after the measurement that replaced it had landed. ## The public cells `docs/BENCHMARKS.md:8`, the `ENG-EXPERT-STREAM` row, read "Streaming-ON decode **VOID** (#912 F1); re-measure owed" while line 9, the row directly under it, recorded 11.05 s/token at 4000 slots against a live cache with a decode-phase `exhausted` delta of 0, explicitly as replacing that VOID figure. `docs/STATUS.md:129` also carried the VOID. A reader got the answer that depended on which row they read first. The two cells never said the same thing, so the repair is stated per file: * `docs/BENCHMARKS.md:8` named no cause and its replacement was one line down. It keeps its `**VOID** (#912 F1)` and now reads "re-measured LIVE by `ENG-EXPERT-STREAM-DEVICE`" where it said "the next row". A positional pointer breaks the moment a row is inserted above it, and no checker validates one; the table holds 253 rows, measured with the checker's own `_table_rows` parser. * `docs/STATUS.md:129` never said a re-measure was owed, and its replacement is not adjacent: that figure is at `docs/STATUS.md:167`, 38 rows further down the SAME table, which runs 126-190 as one unbroken block. It keeps the cause verbatim, the step clock that had no caller, and gains the same named pointer. Neither cell repeats 11.05 s/token. `docs/BENCHMARKS.md:9` and `docs/STATUS.md:167` already carry it, and one number stated in two keyed rows is what lets the two drift apart. ## The row record said the same thing one level down `.agents/specs/expert-streaming.md`'s `## Owed` still read "Re-measure decode on a LIVE cache ... is void", with its "why it is open" naming an unreachable `dgx.casa`. So a fixed public cell routed a reader to the row record and the row record told them the measurement was still owed and blocked. The entry is struck as CLOSED by `ENG-EXPERT-STREAM-DEVICE` W0e and kept as a line, using the convention the sibling spec already uses, because the void's cause is what shaped the replacement: `EndStep()` had no caller, `protected_this_step` never cleared, and every slice from partway through token 3 came from the mmap, which is why W0e gates `exhausted` on its decode-phase DELTA rather than on a total. `.agents/specs/expert-stream-device-slots.md`'s `## Owed` listed issue 1387 as open while the same flow closed it. Struck, with the measured evidence beside it and with issue 573 named as the surviving half. ## Evidence for the figures The re-measure stopped being owed at `c805ccbb3`, recorded in `.agents/benchmark-record.md` under `ENG-EXPERT-STREAM-DEVICE W0e`: 11.05 is rep 2's median over steps 4 to 32, and rep 1's 11.22 sits 1.54% above it. The 8000-slot arm is not a second figure for the same thing; it measures 39.98 to 45.40 s/token and consumed all 30,625 MiB of swap, which that record reads as page-cache displacement and states as a reading of its columns rather than a proven mechanism. No CUDA number moves: `e67b2a4ba` makes `--device cuda` decode 32/32 steps at peak RSS 97.75 GiB, G0-CORRECT fails on a step-7 near-tie whose cause is not identified, and the row's own stop condition therefore leaves G0-SPEED VOID. ## What the pointer cost the cell The pointer clause grew 31 characters, from `; re-measure owed` at 17 to `; re-measured LIVE by ENG-EXPERT-STREAM-DEVICE` at 48. Two edits paid for it and both belong in this sentence, because the second alone does not cover it: the words "as 4 KiB faults" came out at 16 characters, and dropping bold from three supporting numbers and moving "streaming OFF" out of parentheses freed 13 more. 31 against 29 is the +2 that takes the cell from 215 on `main` to 217 here, inside the 220 budget. No fact was lost. The 4 KiB demand-fault mechanism survives verbatim at `.agents/benchmark-record.md:22198` and `.agents/specs/expert-streaming.md:880`, which is where this page's own legend sends a reader, and the index row discloses the displacement in full. Bold now marks the claim and the verdict. `docs/STATUS.md`'s cell measures 195, and that page's `oversized_cells` ratchet sits at 44 of 44, unmoved by this change. No gate catches this class, by design. The rows are keyed on different IDs, so `check-public-doc-tables.py` sees well-formed rows and `check-agent-record.py` sees individually consistent lifecycle states, and no checker here compares a claim in one keyed row against a claim in another. ## The index row is corrected in place `.agents/issue-index.md` is append-only. This row is corrected rather than superseded by a second row, and the reason is the rule's own reason: `check-issue-index-append-only.py` evaluates one aggregate diff against the merge base, and its docstring says a union merge duplicates an EDITED line, which needs that line to exist on both sides. Verified rather than assumed: the row appears in no ref but this branch, checked across all 513 remote heads, and `git show origin/main:.agents/issue-index.md | grep -c` returns 0 where the same command on `HEAD` returns 1, so the absence comes from an instrument proven to fire. The net diff against the merge base is one added line and no removed line. Appending a second row for the same issue would leave a duplicate key in an append-only log. This is flagged rather than done silently, and it is not a precedent: the argument holds only while no other branch carries the row. ## Gates Run from this worktree's own `scripts/`, exit status captured unpiped, after merging `origin/main` `4cde4762a`: `check-agent-record` 0, re-run after the merge; `check-public-doc-tables` 0, re-run after each edit; `check-now-current` 0; `check-conflict-markers` 0, the gate #1450 had just landed; `check-doc-checkpoint --base origin/main --head HEAD` 0; `check-issue-index-append-only --base origin/main --head HEAD` 0, with a detached scratch commit editing row 168, a row that DOES exist on `main`, as its positive control at 1 and the mutation proven applied by `git diff --stat`; `check-commit-trailers --range origin/main..HEAD` 0; `check-commit-style --range origin/main..HEAD` 0; `git grep -n '^<<<<<<< \|^>>>>>>> '` 1 with a two-line fixture as its positive control at 0. `agent-preflight.sh` reports one red, `test_cpu_x86_llamacpp_floor`, which exits 4 `NO_QUIET_WINDOW` where its case asserts 2, at one-minute load averages of 49.77, 52.96, 58.34 and 211.99 across four observations. That is the load dependence tracked in issue 618. A quiet re-run was attempted and could not be taken: the box was polled for 30 minutes and never fell below load 8, peaking at 126. The independence argument is structural instead: this branch changes five files, all `.agents/` or `docs/` Markdown, so neither the test nor anything it drives differs from `origin/main`. ## Issue 1387 It gets no product change here, and the reason is evidence rather than scope. Its record half already landed: the stale string "CPU keep-quant towers only" is absent from `docs/FEATURES.md` on `origin/main` and present at `5f4eb356e^`, which is the positive control for that grep. `5f4eb356e` (#1377) wrote the row and `e67b2a4ba` (#1427) refined it again, so the page names the staging device and both accepted residencies, keep-quant and keep-f16. Its checker half, whether a per-commit record gate should be satisfiable after the commit is published, is issue 573 and stays open under `ENG-RECORD-CONFLICT-SURFACES`. This branch touches no file under `scripts/`. Closes #1442 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
localai-bot
pushed a commit
that referenced
this pull request
Aug 21, 2026
…pair `origin/main` moved twice while the device-leakage repair was being gated: `9ecaf1bb3` when it started, `f07f96e1c` when the preflight base was first resolved, `aeba0de6f` when it was resolved again. Each time `git merge-base --is-ancestor` answered 1 and `scripts/agent-preflight.sh` took its `TRAILER_BEHIND` arm, so `commit-trailers` and `commit-style` reported nothing about this tree. A skip is not a green, so the base is taken again. This one carries #1322, #1450 and #1455 and DOES touch `src/` and `tests/`, unlike the two before it, so the CPU build and the full `ctest` are re-run against the merged tree rather than carried over. Four files auto-merged -- `.agents/issue-index.md` and the three `docs/` projections -- and each was verified by key rather than by the merge exit status: this branch's delta across the merge is byte-identical to its delta from the merge base for all four. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Claude:claude-opus-5 [ClaudeCode]
localai-bot
pushed a commit
that referenced
this pull request
Aug 21, 2026
…ocated every surface this row edits
GitHub reported this pull request `mergeable=CONFLICTING` / `mergeStateStatus=DIRTY`
after the last push, so it could not build `refs/pull/1415/merge` and NO workflow
ran at all: 0 check runs on `2c63cc87a`, while other pull requests were starting
normally. A repository whose Actions are healthy and a branch with zero runs is
the shape a conflict makes, not the shape a slow queue makes.
`docs/STATUS.md` and `docs/USAGE.md` both conflicted, and neither was resolved by
picking a side. Both were RELOCATED on `main`: `USAGE.md` is now a 231-line hub
whose per-model content moved to `docs/models/`, and `STATUS.md` is a 27-row
surface summary that no longer carries per-model rows at all. So both were taken
WHOLE from `main` and this row's scoped edits were re-applied BY KEY at their new
home, `docs/models/nemotron-3-5-lightning.md`:
* the `lm_head` arms row, which still read "**host** -- it refuses a non-CPU
queue by name", now states the device arm, that its numeric gate has NOT
run, and what still serves the host projection;
* the per-decode-token re-expansion table, re-attached above `## What has NOT
been measured` where the refused-arms table now ends.
That is 22 insertions and exactly ONE deletion in that file, with every unrelated
key byte-identical -- the relocation rule, not an automatic three-way merge.
Two of this row's edits are dropped rather than re-applied, deliberately. The
`STATUS.md` Nemotron row and the `USAGE.md` `NemotronHForCausalLM` refusal row do
not exist anywhere under `docs/` on `main` any more. Re-adding either would
re-create a row `main` removed, which is the duplicate a second relocation makes.
`docs/FEATURES.md` auto-merged and KEPT this row's cell, checked rather than
assumed.
`check-conflict-markers.py` (#1450, new on `main`) is 0 findings over 3820
tracked text files, and `check-device-leakage.py` still holds at `vt_ifdef` 32.
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Claude:claude-opus-5 [ClaudeCode]
localai-bot
added a commit
that referenced
this pull request
Aug 22, 2026
… handing out an address-keyed cache (#1415) Row `A2-Q2b`, issue [#810](#810), spec `.agents/specs/nemotron-h-a2q2b-realckpt-lmhead.md`. Measured first, built second. The row's premise was that NemotronH decode re-expands quantized weights on the host and that `lm_head` is a large share of it. That was arithmetic, and the brief that dispatched this row said so. It is now a count. **This description was rewritten after a fresh review returned FINDINGS.** The measurement and the seam design below survived review unchanged and were verified independently. Three claims that were in the previous description did not, and they are corrected in place rather than quietly dropped: the new test had never compiled, the routing checker was said to hold a branch it does not hold, and the Marlin arm was said to be unbuildable here when it builds in seconds. **A scoped re-review then found no correctness defect in the shipped code and verified all seven repairs, and a final round corrected what it did find — all of it evidence, records and process.** The evidence tables named a tree they were not measured on and now carry an `overlay` column with the un-overlaid baseline stated as its own row; the Marlin object figure is re-measured and anchored to a SHA; the shared process-static this row inherits is named; the #1421 index row's counts are corrected while the pull request is still unmerged and the row still editable; and `§4` of the spec stopped telling the next agent to take `flock` on a fleet device. No product code changed in that round. ## What the measurement says Tallied at `NemotronHOwned::DenseBf16` (`nemotron_h.cpp:395`), the single seam every host dequant in this architecture passes through, on the real 21 GB checkpoint (revision `29f2d174`) through the production ABI driver `examples/nemotron_h_gen`. One decode step, T=1, `top_k` 6, 23 MoE layers, 369 dequant calls: | group | shape | calls | elements | % | |---|---|---|---|---| | routed expert `up_proj` | `[1856, 2688]` | 138 | 688 472 064 | 22.36% | | routed expert `down_proj` | `[2688, 1856]` | 138 | 688 472 064 | 22.36% | | shared expert `down_proj` | `[2688, 3712]` | 23 | 229 490 688 | 7.45% | | shared expert `up_proj` | `[3712, 2688]` | 23 | 229 490 688 | 7.45% | | `lm_head` | `[131072, 2688]` | 1 | 352 321 536 | 11.44% | | mamba `out_proj` (FP8) | `[2688, 4096]` | 23 | 253 231 104 | 8.23% | | mamba `in_proj` (FP8) | `[10304, 2688]` | 23 | 637 034 496 | 20.69% | | **TOTAL** | | **369** | **3 078 512 640** | **100%** | `138 == 6 x 23` exactly, which is what identifies this as the decode shape rather than a prefill aggregate. Three findings, and the first two are corrections: 1. The dispatching estimate put `lm_head` at 131072 x 4096 = 537e6 elements and ~43% of the population. `hidden_size` is 2688. It is 352 321 536 elements and 28.35%. 2. The widely quoted "1.24e9 elements / 2.49 GB per token" names a REGIME, not a total. It is not the host arm's 3 078 512 640. It is `mamba + lm_head` = 1 242 587 136 = 2.485 GB, the residue after A2-Q2a moved the MoE arm to the device, matching to four significant figures. 3. `lm_head` is the last one. Against the A2-D1 discriminator on `dgx:gpu0` (device mamba ON 1.554 s/token and 108.2x vs vLLM, OFF 10.319 s/token and 718.1x), the mamba arm is worth 6.64x and `lm_head` is on the host in every leg. Once the mamba arm lands, `lm_head` is 352 321 536 of 352 321 536, i.e. everything that remains. It is also the largest single re-expansion in the model by 12.7x, so its 704.6 MB transient bf16 buffer is the one that matters on a unified-memory box that reboots rather than OOM-kills. So the estimate's share was wrong, its direction was right, and the case is stronger after the discriminator than before it. ## The residency decision, taken explicitly The spec's `## 5. Owed` required this row to choose rather than default: "A2-Q2a routed around it by never calling either `MarlinDenseResidentFor`; `lm_head` must do the same or say why not." That accessor keys its repack cache on the weight's ADDRESS ([#984](#984)), and NemotronH is exactly the second-consumer condition an address key cannot survive. Rather than fork the seam as `qwen3_5.cpp` did, or inherit the defect as routing through it plainly would, the seam is EXTENDED: `MatmulNvfp4W4A16D` and `MatmulNvfp4MarlinD` accept a caller-owned `MarlinDenseResident*`, defaulting to the existing cache so every current caller is byte-for-byte unchanged. NemotronH owns its resident in a `ResidentSlot` on the weights, which is the property [#237](#237) added `ResidentSlot` for and the same call A2-Q2a made for the MoE arena. [#984](#984) is left exactly as it was for every other caller, where its own two-engine red-before belongs. ## What moved `NemotronHPagedForward`, the production path, now returns a device-resident `ForwardLogits` on an NVFP4 checkpoint, which is every released one, and downloads `final_normed` only when a consumer needs it. `NemotronHDeviceForward` deliberately keeps the host projection. It is the operand the numeric gate compares against, it is what makes A2-R's token gate attributable, and it has no production caller. The spec required this row to disclose where that property ends: it ends in the paged forward. ## The allowlist entry is narrowed, not removed Its one clause is met. But `check-runner-routing-consistency.py` resolves a cross-TU delegate only for the `Class::ForwardDevice` shape, and NemotronH's device forward is a free function in another TU, so the model still classifies HOST although `NemotronHPagedForward` assigns both `fl.device_tensor` and `fl.device_storage`. Filed as [#1410](#1410) and left to its own row: it changes checker semantics, which `AGENTS.md` routes away from an in-flow fix, and widening a classification to turn a red gate green is precisely the move that section slows down. ## The review repair ### The new test had never compiled, so the red it claimed could not have existed `tests/vllm/models/test_nemotron_h_moe_device.cpp` used `NemotronHHostWeights` unqualified and never declared it. Measured on the same command: | tree | `compile_rc` | errors | |---|---|---| | PR head `29b1128e`, this file | 1 | 9 | | merge base, same file | 0 | 0 | | repaired, default arm | 0 | 0 | | repaired, `-DVT_MARLIN_NVFP4=1` | 0 | 0 | The consequence is larger than the declaration. The file had never compiled on any platform, so **the red-first result this row claimed for its synthetic numeric gate did not exist and could not have existed**. That gate is CUDA-only and still has never executed; the spec now records `NEVER RUN` rather than restating a red nobody saw. The test target is deliberately registered with no CUDA guard, and that is what surfaced this — a case that skips at run time still has to parse on every CPU build. ### The production source asserted a protection that does not exist A comment at the device branch claimed the allowlist entry was removed and that "the routing checker, not a comment, is what now holds this branch in place: put the host projection back and it goes red by name". All three claims were false, and they contradicted this description and the allowlist file in the same commit. Reproduced here on the repaired tree: baseline `check-runner-routing-consistency.py` exits 0 with "3 host-logits off-framework (3 allowlisted)"; after deleting the whole `if (DeviceLmHeadEligible(...)) { ... return fl; }` block (`git diff` moves by 1 insertion and 14 deletions, so it applied) the checker exits 0 with **byte-identical output**; the file was restored to an identical sha256. **Nothing automated holds that branch.** The checker is not widened to make it — that changes checker semantics and [#1410](#1410) owns it with its own red-before. The comment now says what is true, the allowlist entry's own text is corrected the same way, and the reachability deletion mutation is recorded PENDING a CUDA window rather than claimed. Every case that can enter the device branch is `MarlinW4A16Selects`-gated and therefore CUDA-only, so that mutation cannot run on a CPU box. ### A silent dequant fallback that no token gate can see `DeviceLmHeadEligible` restated the shared dispatcher's three selection clauses and dropped `MarlinW4A16Enabled()`. Under an explicit `VT_NVFP4_MARLIN=0` the model said eligible while `MatmulNvfp4W4A16D` took its naive redundant-dequant arm — the same logits, while re-uploading the whole `[131072, 2688]` operand on **every decode step**, because `LmHeadNvfp4View` hands out a stack temporary that `ResidentNvfp4`'s weight-keyed cache can never hit. The repair is structural rather than a patched clause. The three clauses now live once, in `dense_nvfp4::MarlinW4A16Selects`, and both the dispatcher and the model call it, so they cannot drift. `DeviceLmHeadD` additionally refuses BY NAME if that same predicate is false on the operand it is about to hand over, which catches an eligibility answer taken against a different queue or dtype. That is deliberately the predicate and not the seam's `fallback_gemms` counter: `MutableW4A16Stats()` is a plain non-atomic process-wide static, so a counter window in production would refuse a correct run whenever anything else took a fallback GEMM concurrently, and a false refusal is worse than the silence it replaces. The counter is the right instrument in a test, where single-threadedness is a property of the harness, and the synthetic gate asserts it there. It is demonstrably armed rather than assumed: `test_qwen3_forward.cpp:559` — the assertion itself, not the `TEST_CASE(` line at `:497` — already asserts on CPU that it reaches exactly `5 * num_hidden_layers` when the dispatcher does fall back. **And the counter is not the only process-wide static on this route, so this description should say what the row does NOT decline.** `dense_nvfp4::DenseMarlinWorkspace` (`dense_nvfp4_gemm.h:506`) is a process-static DEVICE allocation — `static void* ws` behind a `static std::mutex`, sized from `MarlinDeviceSms` and keyed on nothing, not even the device index — shared by every caller of `MatmulNvfp4MarlinD` (`:539`) and `GateUpFusedMarlinD` (`:700`). It has three consumers today (the shared dense route in `dense_attn_block.h`, MiniMax-H3, and the compressed-tensors NVFP4 scheme); NemotronH's `lm_head` becomes a fourth. It is PRE-EXISTING — it arrived at `80d1da096` and its definition is byte-identical at this row's merge base and at its head — and nothing here addresses it. In a change whose thesis is refusing to inherit the sibling process-static defect ([#984](#984)), saying nothing about the one it does inherit was an asymmetry a reviewer is entitled to see stated. The cost the fallback would carry is 198.18 MB per call — 176.16 MB of packed codes plus 22.02 MB of group scales — arithmetic that checks to the byte against the `[131072, 2688]` geometry. The behavioural red for this class needs CUDA and is PENDING with the rest. ### The Marlin arm builds here, and saying otherwise overstated the blocker The previous description said "this branch has not built the Marlin path at all". `include/vt/cuda/marlin_repack.h` includes only `<cstdint>`, `<cstddef>` and `<vector>`, so the host side needs no CUDA toolkit. On this box, which has no `nvcc`: ``` c++ -std=c++20 -I include -I src -isystem third_party -DVT_MARLIN_NVFP4=1 \ -Wall -Wextra -Werror -c -o nhd_marlin.o \ src/vllm/model_executor/models/nemotron_h_device.cpp -> rc 0, 0 errors, 0 warnings, a 1 268 552-byte object ``` **That byte count is anchored to the merge commit on this branch**, gcc 13.3.0, no `nvcc`, and compilation is deterministic here — two runs of the identical command, identical `sha256 42d670b6...`. The anchor is the correction. The figure this description carried until now, 1 269 696, was accurate when written and accurate at the reviewed head `29b1128e3`, but the translation unit moved twice afterwards — 1 272 808 at `fedf78d86`, 1 268 552 from `bff2b7b2f` on — and an evidence block naming no SHA cannot tell a reader which of the three it means. The measured series across seven commits is in the spec. The size is incidental either way; what the block asserts is `rc 0`, zero errors and zero warnings on a toolchain with no CUDA, and that holds at every one of those commits. Both arms are compiled and both are compiled `-Werror`. What genuinely needs `nvcc` is the Marlin **kernel** and every **execution** of the device path; those stay PENDING and are what the table below records. ### Reachability, and a CPU vehicle that already existed Both synthetic cases build `NemotronHHostWeights` by hand, which proves the class works and not that anything reaches it, and the deletion mutation for the device arm needs CUDA. But this row also refactored the part of `NemotronHPagedForward` that every CPU step runs, and `tests/vllm/models/test_nemotron_h_paged_forward.cpp` already drives that function through a real `GPUModelRunner`. It gains a case: three gathered rows against one request, so the returned row count must follow `logits_indices` and never `num_reqs`; the trace's copy of `final_normed` fed back through the production host entry point must reproduce the returned logits bit for bit; and the same logits must come back with the trace off. With `trace->capture` set, `DownloadF32(final_normed)` ran twice on the host arm. One download now serves both consumers on every path. That duplicate is not observable from outside the function — both copies are of the same unchanged buffer — so it is repaired structurally and no assertion pretends to catch it. ### Records `spec §4.3` did not exist; the requirement is under `## 5. Owed`. The allowlist's `## Gates` did not either; the heading is `## 3. The gate`. `docs/FEATURES.md` still listed the device `lm_head` as owed while `docs/USAGE.md` said it was device; both now say implemented and never executed, which is the honest state. The appended issue-index row for [#1410](#1410) cited `## Owed` where the heading is `## 5. Owed` — cosmetic, since the row passes on its owning-row disjunct, but the index is append-only and this is the only window to correct it. [#1421](#1421) is filed rather than fixed in flow: two NVFP4 W4A16 upstream anchors are wrong at the pin, verified against a checkout of `vllm-project/vllm` at `5559679229bc961848b121ccdeaa8fa5d79bec98`. `kernels/linear/__init__.py:879-881` is the tail of `init_wfp8_a16_linear_kernel`, an FP8 helper sharing the `a16` vocabulary, and the forced-Marlin line it quotes is at 922-924; `qwen3.py:271-274` is the `class Qwen3ForCausalLM(...)` line and `packed_modules_mapping` is at 275-278. Both predate the merge base, and both are cited from nine code and test sites over SEVEN files plus eight record sites over FIVE record files (`parity-ledger.md` carries three). The index row said "six files" while its own parenthetical enumerated seven, and "six record files" where there are five; `.agents/issue-index.md` is append-only and a landed row can never be edited, so an unmerged pull request is the only window to correct that, and this one does. The set still needs re-deriving at the pin rather than a sed. This branch relocates one of those citations and deliberately leaves its VALUE unchanged, so the issue owns every site uniformly instead of half of them looking reviewed. ## Gate status, stated as pending rather than as passed Every number here is from a run on a CPU box with no CUDA toolkit, `RelWithDebInfo`, `-Wall -Wextra -Werror`. **Read the `overlay` column first.** This tree cannot construct a `GPUModelRunner` at all — [#1371](#1371) throws there — so every green below was taken with [#1392](#1392 production fix applied to the working tree, never committed here and reverted byte-for-byte afterwards. The previous version of this table said "a run on this tree" and reported the §12 case at `run_rc 0 / SUCCESS!`, which is false of the tree it named: un-overlaid, that case is already red. | binary | overlay | `run_rc` | cases | assertions | verdict | |---|---|---|---|---|---| | `test_nemotron_h_moe_device` | none | 0 | 4, 4 passed | 4 | `SUCCESS!` — and **all four SKIP**, both A2-Q2b cases included | | `test_nemotron_h_paged_forward`, whole binary | **none** | 1 | 13, 2 passed, 11 failed | 18 | `FAILURE!` — [#1371](#1371), not this row | | `test_nemotron_h_paged_forward`, whole binary | **#1392** | 0 | 13, **13 passed** | 3269 | `SUCCESS!` | | the new §12 case alone | **none** | **1** | 1, **0 passed, 1 failed**, 12 skipped | **0** | **`FAILURE!` — throws #1371 before its first assertion** | | the new §12 case alone | **#1392** | 0 | 1, 1 passed, 12 skipped | 13 | `SUCCESS!` | **The `moe_device` row is not a pass, and it is the honest state of the synthetic gate.** The binary builds and exits 0, but every case takes the `TryCudaQueue` skip on a GPU-less box, so those 4 assertions are the skip notices themselves. The numeric gate examined nothing. **The paged-forward red is [#1371](#1371 All 11 failures throw the identical `No valid attention backend for device type 0 from {FLASH_ATTN: [head_size not supported]}` at `GPUModelRunner` construction, and 10 of them are cases this row never touched. Overlaying [#1392](#1392 production fix in the working tree — never committed here, reverted afterwards — turns the same binary green. Worth recording: doctest printed `assertions: 18 | 18 passed | 0 failed` while 11 cases were throwing, so the assertion line alone would have read as a pass. ### The red-first, on the cases that can run Each mutation went into a scratch copy of `nemotron_h_device.cpp`, was proven applied by `git diff --stat`, was built before being run (a mutation that fails to build proves nothing), and was restored to an identical sha256. **Every row below was also run with [#1392](#1392) overlaid**, and that is not a detail. Un-overlaid, the §12 case is already `run_rc=1` on the UNMUTATED tree, so an M1 or M2 red measured there would prove nothing whatever. The overlay is what lets the vehicle report a green in the first place; only then can a mutation take it away. | mutation | overlay | applied | `compile_rc` | `run_rc` | verdict | |---|---|---|---|---|---| | — (unmutated control) | **#1392** | — | 0 | 0 | **GREEN**, `SUCCESS!`, 13 assertions | | — (unmutated control) | **none** | — | 0 | **1** | already **RED** — #1371, which is why the rest is overlaid | | **M1** — `n_out` -> `R` at the host projection | **#1392** | 1 ins / 1 del | 0 | **1** | **RED**, `FAILURE!` | | **M2** — never fill `trace->final_normed` | **#1392** | 1 ins / 2 del | 0 | **1** | **RED**, `FAILURE!` | | **M4** — restore the exact pre-repair two-download shape | **#1392** | 3 ins / 8 del | 0 | 0 | **GREEN — reported, not hidden** | Re-derived rather than restated, on a clean tree at the merge commit, overlay and mutations reverted to byte-identical sources (`nemotron_h_device.cpp` back to `sha256 abf6e21f...`): §12 alone un-overlaid gives `run_rc=1`, `1 | 0 passed | 1 failed | 12 skipped`, `assertions: 0`, `THREW exception: No valid attention backend for device type 0 from {FLASH_ATTN: [head_size not supported]}`; overlaid, `run_rc=0`, 1 passed, 13 assertions; the whole binary overlaid, 13/13, 3269 assertions; M1 `compile_rc=0 run_rc=1` throwing `gathered row count does not match hidden_size` at `nemotron_h.cpp:1029`; M2 `compile_rc=0 run_rc=1` on `REQUIRE( 0 == 288 )`. M4's cell needs no separate attestation — un-overlaid every run of this case is red, so a GREEN is only reachable with the overlay and its verdict entails the column. M1 is the red the `n_out` rename exists for: with the request count substituted, the returned row count is 1 where the gather asked for 3. M2 arms the trace-operand assertion. Both showed the same trap — doctest printed `assertions: 2 | 2 passed | 0 failed` on M1 while the case was failing, because a `REQUIRE` throws rather than counting. M4 staying green is a result, not a gap. The duplicate `DownloadF32` copies the same unchanged buffer twice and produces identical bytes, so nothing observable from outside the function can distinguish it. It is repaired structurally and no assertion pretends to catch it. **The device arm's own red-first does not exist on a CPU box and is not claimed.** `MarlinW4A16Selects` is false on a CPU queue, so the device branch, the `fallback_gemms` assertion and the reachability deletion mutation are all unreachable here. ### A gate this row was already failing, and cannot repair in place `scripts/check-doc-checkpoint.py` is red on this branch, and it was red at the reviewed head `29b1128e` with the identical two errors (`--base 96ed834 --head 29b1128`): ``` ERROR: commit 1c62d99: changed user_usage but did not update docs/USAGE.md ERROR: commit 8fa900a: changed .agents/benchmark-record.md: measurement recorded but did not update docs/STATUS.md ``` The previous description said this gate had one real failure and that it was repaired. It was not, and `.github/workflows/ci.yml:519` runs the same `--base/--head` invocation, so the lane is red for this reason independently of [#1371](#1371). The second error names a real gap and it is now closed: this row moves a lifecycle state and records a measurement, and `docs/STATUS.md` said neither. It does now, in 192 characters, because that page carries a shrink-only ratchet on oversized cells and the first attempt took it from 44 to 45. The first error, and the historical form of the second, cannot be closed by a later commit. The checker iterates `commits_in_range` and judges each commit on its own contents, so the obligation belongs to `1c62d9974` and `8fa900a62`. Discharging it means rewriting commits that are already the reviewed base, which resets this pull request's CI approval and moves the head a fresh reviewer was asked to look at. **That is a scheduling decision, so it is recorded as owed and raised for the operator rather than taken by a repair pass.** Run over this repair's own commits (`origin/main..HEAD`), the checker reports these two and nothing else. ### The spec sent the next GPU window through the wrong mutex `§4` step 1 read "Take `$GPU_LOCK` with a blocking `flock` and wait". `AGENTS.md` requires a fleet device to be claimed through `rc`, and `dgx:gpu0` is one. The text predates that rule, but this branch is the wrong place to leave it standing: it ADDS `scripts/nemotron-h-a2q2b-gpu-gate.sh` for exactly that window, the script correctly takes no mutex of its own, and the contradicting instruction sat beside it in the same file — and every PENDING leg above is waiting on that window, so the next reader of `§4` is the person about to open it. Two mutexes that do not exclude each other are worse than one: the fleet cannot see `$GPU_LOCK`, so a `flock` over `ssh` does not exclude a concurrent `rc` holder and the controller reports the box free while somebody is on it. On 2026-08-17 that pair voided a whole speed axis (`minimax-music3.md` §13.10). Step 1 now claims through `rc run -d dgx:gpu0 --max-runtime <N>h --` with the gate script as the payload, and step 2's headroom check moves inside the lease and names the script's own PRECONDITION 1. ### The trailer gates had stopped running At the previous head, `origin/main` was not an ancestor, so `scripts/agent-preflight.sh --fail-on-skip` took its `TRAILER_BEHIND` arm and SKIPPED both `commit-trailers` and `commit-style`. A skip is not a green. **It then happened five more times during the device-leakage repair**, because `origin/main` moved under it at `9ecaf1bb3`, `f07f96e1c`, `aeba0de6f`, `01854663c` and `c8d926ea8`. The ref only moves on a fetch, so the fix is ordering: merge, then do not fetch again, then gate, then push. At the pushed head both gates RUN: `BASE_SHA=c8d926ea8` non-empty, `ANCESTRY_STATUS=0`, `RANGE_COUNT=22`, `RANGE_STATUS=0`, **zero `SKIP` tokens anywhere in the report**, 85 gates `ok`, and the only failure is `doc-checkpoint range` with the two commits above. The skip arms are falsified individually rather than read off a green line, because `agent-preflight.sh` prints a summary only on failure. Each is armed over exactly this range, proven by a detached scratch commit with a period-terminated subject and no trailer block, which turns `check-commit-style.py` and `check-commit-trailers.py` red with the specific messages. Both files that both sides of the merge touched are records, resolved by shape rather than by whatever the three-way merge produced. `.agents/issue-index.md` is a genuine append-only log carrying `merge=union`, and both sides append at the tail, so all four new rows survive — verified additive, 469 + 2 lines on each side to 471, no line removed or altered. `docs/USAGE.md` is a keyed record: the merged file is byte-identical to `origin/main`'s version with this branch's scoped edit re-applied, so every key neither side owns is unchanged. ### The DSR ratchet was red, and no review round ever saw it run `device-leakage` never COMPLETED while this pull request was under review, so its verdict was an input to none of the three passes. It completed after the third and failed: `vt_ifdef` **35 against a baseline of 32**, `rc 1`. Three `#ifdef VT_MARLIN_NVFP4` sites had been added to the device-agnostic shared layer, which is exactly what the ratchet exists to stop. They were located by running `scripts/check-device-leakage.py --report` at the failing head `7a3909187` and diffing the per-file table against `9ecaf1bb3`, rather than by reading the diff for guards. | # | site at `7a3909187` | what the guard decided | resolution | |---|---|---|---| | 1 | `include/vllm/model_executor/models/dense_nvfp4_gemm.h:768` — inside `MarlinW4A16Selects` | **nothing** | **removed** | | 2 | `src/vllm/model_executor/models/nemotron_h_device.cpp:883` — around `LmHeadNvfp4View` | **nothing** | **removed** | | 3 | `src/vllm/model_executor/models/nemotron_h_device.cpp:985` — `DeviceLmHeadD`'s body | `ResidentIn` and a complete `dense_nvfp4::MarlinDenseResident` | **`DSR-ALLOW(A2-Q2b)`** | **No baseline was changed.** `AGENTS.md` forbids making a red gate green by widening an assertion, and a baseline bump is that. `scripts/device-leakage-baseline.json` is untouched at 32, and the per-file table is now byte-identical to `origin/main`'s. **(1) is the case the checker's own message describes.** `MarlinW4A16Selects` is a SELECTION wearing a build guard, and every term it reads exists in every build: `MarlinW4A16Enabled()` is declared above the guarded region, and `vt::OpRegistered` is the op/provider table's own answer to whether the Marlin arm is realized for a device. The flag and the registration are one condition, not two — `CMakeLists.txt`'s single `if(VLLM_CPP_MARLIN)` block adds `src/vt/cuda/cuda_moe_marlin.cu`, whose file-scope `Registrar` holds the tree's only `RegisterOp(OpId::kMoeGroupedGemmNvfp4Marlin, …)`, and defines `VT_MARLIN_NVFP4=1` in that same block. A build without the macro registers nothing, so the query already resolves false on exactly the builds the `#ifdef` excluded. This is the call `nemotron_h_device.cpp`'s `moe_on_device` selection had already made, in a comment that says so. **(2) was measured rather than reasoned.** `LmHeadNvfp4View` names nothing the Marlin build adds — `Nvfp4Weight` comes from `qwen3_5_weights.h` and `OwnedBytes` from the loader, both unconditional, and no `vt::cuda::` symbol appears in it. The claim that its external linkage at `namespace vllm` scope is what makes an unused definition harmless where its only call site is compiled out is proven by the mutation that removes that property: adding `static` turns the same CPU compile RED at `rc 1`, `error: 'vllm::Nvfp4Weight vllm::LmHeadNvfp4View(...)' defined but not used [-Werror=unused-function]`. The file was restored to an identical sha256 (`9719ea70…`) afterwards. **(3) is TYPES-not-behaviour and takes the checker's documented escape hatch.** `DSR-ALLOW` is not a baseline change: the site is excluded from the count but COUNTED AND PRINTED on every run, so the exemption is visible in CI output rather than invisible in the diff. `DeviceLmHeadD`'s body names two symbols that do not EXIST without the guarded arena region — the `ResidentIn` template, defined inside it, and `dense_nvfp4::MarlinDenseResident`, which the header declares unconditionally and defines only under `VT_MARLIN_NVFP4`, so the reference cannot bind to an incomplete type. It is the same class and the same stated reason as the five sibling guards A2-Q2a and A2-P already carry in this file, and its `#else` refuses by name. The SELECTION for this arm stays a runtime op-table query. **Measured on this tree**, a CPU build with `VT_MARLIN_NVFP4` absent from `build/compile_commands.json` — positive control: 1020 `VLLM_CPP` hits in the same file, so the grep is not silently wrong — which is the configuration that exercises both removals, because it is the arm the deleted `#else` branches used to serve. | what | before (`7a3909187`) | after | `rc` | |---|---|---|---| | `check-device-leakage.py` `vt_ifdef` | 35 | **32** | 1 → **0** | | `DSR-ALLOW` exemptions in force | 20 | **21** | — | | `scripts/device-leakage-baseline.json` | 32 | **32, untouched** | — | | per-file table vs `origin/main` | +1 header, +2 model TU | **identical** | — | | `nemotron_h_device.cpp`, `nemotron_h.cpp`, `qwen3_5.cpp` at `-Wall -Wextra -Werror` | — | compile | **0** | Each repair is individually load-bearing, proven by reverting it alone in a scratch worktree and re-running the gate. Every mutation was verified applied by `git diff --stat` and restored to an identical sha256, with the unmutated control green immediately before and after. | mutation | applied | `vt_ifdef` | `rc` | verdict | |---|---|---|---|---| | — (control) | — | 32 | 0 | `ratchet holds` | | **M-B** — restore the guard on `MarlinW4A16Selects` | 4 ins | **33** | **1** | **RED**, `DSR REGRESSION` | | **M-C** — restore the guard around `LmHeadNvfp4View` | 2 ins | **33** | **1** | **RED**, `DSR REGRESSION` | | **M-D** — delete the `DSR-ALLOW(A2-Q2b)` line | 1 del | **33** | **1** | **RED**, `DSR REGRESSION` | | — (control, after restore) | — | 32 | 0 | `ratchet holds` | The two pre-existing allowlist entries the report also prints, `deepseek_v4_device.cpp [kcuda] x8` and `platform.cpp [dev_cast] x1`, are byte-identical at `9ecaf1bb3` and here. This change moves one bucket and nothing else. **`docs/USAGE.md` rides in the same commit, because it has to and because it is true.** Any commit touching `include/vllm/` is `user_usage` to `scripts/check-doc-checkpoint.py` and owes the surface in that same commit. `doc-checkpoint range` is already red on this branch for `1c62d9974` and `8fa900a62`, which an operator decision covers; a third error of the identical kind would be new damage rather than inherited, so the header hunk was recommitted with the USAGE edit beside it. The edit is not written to feed the gate: the NemotronH arms table enumerates what selects the host `lm_head` projection and omitted `VT_NVFP4_MARLIN=0`, a user-settable knob that selects an arm and appears nowhere else in USAGE, and this change is precisely what makes that knob reach the model's eligibility test in every build rather than only where the guard compiled it in. `check-doc-checkpoint.py --commit` is `OK` on that commit, with the still-red `1c62d9974` as the positive control that the checker is armed. **`origin/main` moved four times during this repair** — `9ecaf1bb3`, `f07f96e1c`, `aeba0de6f`, `01854663c` — and each time `git merge-base --is-ancestor` answered 1, which is the arm that SKIPS both trailer gates without failing. Each was merged rather than worked around, and every auto-merged keyed record was resolved by key on every merge: the branch delta across the merge byte-identical to the branch delta from the merge base, checked file by file with a positive control on the comparison itself. ### The three red CPU jobs were inherited, and the merge cleared them `build-test-cpu` and both `sanitize-cpu` arms failed on this pull request, all three on `test_nemotron_h_paged_forward` — [#1371](#1371), which [#1392](#1392) fixed on `main` after the review rounds. `sanitize-cpu` is NOT a no-baseline job, so that had to be triaged by WHICH test failed rather than assumed. The prediction is verified rather than asserted. With `origin/main` merged, the same binary is green on this tree with NO overlay: `13 | 13 passed | 0 failed`, `assertions: 3269 | 3269 passed | 0 failed`, `Status: SUCCESS!`, `run_rc 0` — the same numbers the spec's tables previously recorded only WITH #1392 applied to the working tree and reverted. **The `#1392` overlay is therefore retired**; the historical rows in the spec keep their `overlay` cells, because they describe the tree they were measured on and rewriting them would make them false. ### `main` brought four red suites with it, and they are attributed both ways The third sync carried `4712dac40` (`VT-ACT-ROUND-POLARITY`, [#1322](#1322) via [#1347](#1347)), and with it four red CPU suites this branch does not touch. Both `ctest` runs are on this box, no overlay of any kind. | tree | base | `ctest` | result | |---|---|---|---| | this row + `9ecaf1bb3` | before `4712dac40` | 567 | **`100% tests passed, 0 tests failed out of 567`**, `rc 0` | | this row + `01854663c` | after `4712dac40` | 569 | `rc 8`, **4 failed**: `test_minimax_music3_ar`, `test_ltx2_text_encoder`, `test_muse_glimmer_text`, `test_muse_glimmer_text_fallback` | Those four are [#1458](#1458), filed from another flow before this control was run. Inherited, not caused, and proven in both directions on the same tree rather than argued — each mutation verified applied by `git diff --stat` and restored to an identical sha256: | control | change | `compile_rc` | `run_rc` (music3 / ltx2 / glimmer) | verdict | |---|---|---|---|---| | **A** | revert THIS row's two source files to their pre-repair `7a3909187` content | 0 | **1 / 1 / 1** | still RED — **not this row's** | | **B** | revert `src/vt/cpu/cpu_ops.cpp` alone to `4712dac40^` | 0 | **0 / 0 / 0**, `37/37`, `27/27`, `24/24` | GREEN — **`4712dac40` is the cause** | Control A answers the attribution question on its own; control B is here because naming a cause is more useful to the next reader than clearing oneself. Neither repairs anything, deliberately: #1458 needs `VT-ACT-ROUND-POLARITY` to decide whether its kernel or four bf16 error floors that were never re-derived are the wrong side, which is that row's oracle work rather than a small and clear in-flow fix. **That window has since closed.** [#1458](#1458) was fixed on `main` and closed, and this branch has merged the repair, so the four are not expected on the current head. The controls stay recorded because a red that was attributed and then disappeared is still the reason two `ctest` runs in this description disagree, and deleting the evidence would leave that unexplained. ### `documentation-checkpoint` is the same two commits, and it is not new It went red for the same reason `device-leakage` did: it had never COMPLETED during review. Its whole failure set is `1c62d9974` and `8fa900a62`, the two an operator decision covers and which cannot be repaired in place — `check-doc-checkpoint.py` iterates `commits_in_range` and judges each commit on its own contents, so a later commit cannot close either, and discharging them means rewriting the reviewed base. The job's three invocations were reproduced locally at this head with CI's own arguments (`--base` the PR base `63d87805c`, `--head` HEAD): | invocation | result | |---|---| | `check-doc-checkpoint.py` | `rc 1` — **exactly `1c62d9974` and `8fa900a62`, nothing else** | | `check-now-current.py` | `rc 0`, `OK: .agents/NOW.md is a current, in-budget resume digest` | | `check-role-discipline.py` | `rc 0`, `OK: every change on main arrived on a task branch` | **This repair's own commits are clean, and that is not an accident.** The first draft of the header fix added a THIRD error of the identical kind, and it is the reason `docs/USAGE.md` rides in the same commit as the `include/vllm/` hunk rather than in a follow-up: the checker's obligation is per-commit, so a follow-up cannot discharge it. `check-doc-checkpoint.py --commit 65ab066` is `OK`, with the still-red `1c62d9974` as the positive control that the checker is armed against exactly that class. ### Nothing ran at all for a whole push, and the cause was a conflict rather than a queue The push before this one produced **zero check runs** on `2c63cc87a` while other pull requests were starting normally. `gh pr view` named it: `mergeable=CONFLICTING`, `mergeStateStatus=DIRTY`. GitHub could not build `refs/pull/1415/merge`, so no `pull_request` workflow had anything to run against. A healthy Actions queue plus a branch with zero runs is the shape a conflict makes, and reading it as a slow queue would have cost the whole verdict. The conflict was `docs/STATUS.md` and `docs/USAGE.md`, and neither was resolved by picking a side, because both had been **relocated** on `main`. `USAGE.md` is now a 231-line hub whose per-model content moved under `docs/models/`, and `STATUS.md` is a 27-row surface summary with no per-model rows at all. So both were taken WHOLE from `main` and this row's scoped edits re-applied BY KEY at their new home, `docs/models/nemotron-3-5-lightning.md` — 22 insertions and exactly ONE deletion there, every unrelated key byte-identical. One of this row's edits is **dropped rather than re-applied**, deliberately, and the other was never dropped at all — the claim that both were is corrected below. The `STATUS.md` Nemotron row is genuinely gone: merged `main`'s `STATUS.md` is a 98-line surface summary that enumerates no model at all. `Nemotron` returns 0 hits there, and so do the positive controls `Kimi` and `Laguna`, while the file plainly greps. Re-adding it would re-create a row a relocation removed, which is the duplicate a second relocation makes. The `USAGE.md` `NemotronHForCausalLM` refusal row is the opposite case, and the earlier claim that it "no longer exists anywhere under `docs/`" was **wrong**: `aee6c48d6` (#1491) did not delete that row, it **RELOCATED** it to `docs/reference/model-loading.md` — a file that did not exist at this branch's pre-merge base, which is precisely why the earlier pass read it as gone. It is still on `main`, and this row now updates it. `docs/FEATURES.md` auto-merged and kept this row's cell — checked, not assumed. `check-conflict-markers.py` (#1450, new on `main`) reports 0 findings over 3820 tracked text files. Run locally from the ORIGINAL merge base, `check-doc-checkpoint.py` also reports `d995c52f0`, `af25bd251` and `995ed1ccd` — each verified an ancestor of `origin/main`, none in this branch's own range, all arrived with the merge. CI does not see those, because it resolves `--base` to the CURRENT merge base: **its whole failure set is `1c62d9974` and `8fa900a62`**, the two an operator decision covers, and nothing else. This row's own commits stay clean either way: `check-doc-checkpoint.py --commit` is `OK` on `65ab06636` and on the merge itself. ### Two public rows still said the head runs on the host, and this change is what made them false Following the relocation above to its consequence: `docs/reference/model-loading.md:188` still read "`lm_head` and FP8 Mamba2 projections run on the host". That sentence was true before this row and is false after it, which is the test for whose obligation it is. Left alone, the tree would land with `docs/models/nemotron-3-5-lightning.md` calling the head a device arm and this row calling it a host arm, on the same merge commit. Only the clause this row falsified moves. `lm_head` reaches the device **on the paged forward** and is carried as implemented-and-unmeasured, matching what the model recipe and the spec's `## Now` already say. The FP8 Mamba2 projections are a **different unit** (A2-Q1, #1289, still held DRAFT) and still run on the host, so that clause is kept and only re-worded to stop sharing a verb with the head. The pending-token-gate and no-GGUF clauses are untouched, because both are still accurate. Sweeping by CLAIM rather than by file, at the MERGED tree, found a **second** site the scoped review did not reach, because it is not a refusal row. The checkpoint registry at `docs/USAGE.md:230` lists per-checkpoint arms, and its "Supported arms" cell read "host FP8 Mamba2 and NVFP4 head". That "NVFP4 head" is this same `lm_head`, so the same test applies and the same one clause moves. That cell is the surface the "say which weights, and from where" rule owns, so a supported-arms cell putting the head on the host is exactly the contradiction this finding is about. The sweep carried its own controls, because a null grep only proves the terms wrong. Every `docs/**.md` and `README.md` line asserting a host placement for a head or for logits is 11 hits, 2 of them the sites repaired here (positive control); the same pipeline with a nonsense term returns 0 (negative control). The other 9 were read rather than pattern-matched, and none is stale: `BENCHMARKS.md:16` is this row's own host-re-expansion ATTRIBUTION and is what motivates the device arm; `BENCHMARKS.md:276` is Laguna GB10 ATS weight residency; `ENVIRONMENT.md:144,187,212` are Qwen3.5/Qwen3.6 levers; `server.md:88` is the CPU-only `prompt_logprobs` full-logits route on the shared runner, not this model; `model-loading.md:144` is the FP8 scalar-scale guard; and two are historical `docs/superpowers/plans/` documents, which are not projections. **No gate can catch either repair.** `check-doc-checkpoint.py` asks whether SOME `docs/USAGE.md` edit accompanied a commit of the changed class. It never asks whether a sentence inside it is still true. ### Every check has COMPLETED on the current head, and the remaining reds are all inherited `ci` run `32537115264` on `99f9f672a`: **25 checks, 0 pending** — 14 pass, 6 fail, 5 skipped by design on a pull request (`attest`, `baseline-summary`, `manifest`, `promote`, `publish`). | check | on `b7d89b43e` | on `99f9f672a` | whose | |---|---|---|---| | **`device-leakage`** | SUCCESS | **SUCCESS** | **this row's.** The restructure survives the 38-commit merge at `vt_ifdef` 32, and `scripts/device-leakage-baseline.json` is byte-identical to `main`, so the row adds no new allowance | | `build-newest-gcc` | FAILURE | **SUCCESS** | was **`main`'s**, and it cleared exactly as predicted once `main` carrying #1581 was merged. CI builds the MERGE commit, so the red came from the base side; #1565 is closed | | `build-test-cpu`, `sanitize-cpu (address,undefined)`, `sanitize-cpu (thread)` | SUCCESS | **FAILURE** | **`main`'s**, [#1608](#1608) and [#1602](#1602) — see below | | `build-test-vulkan`, `build-test-cpu-arm64`, `cuda-fat-build`, `cuda-arch-features`, `vulkan-spirv-freshness`, `agent-record`, `pr-size`, `commit-protocol-tag`, `last-gated-commit`, `plan`, `verify` x2 | SUCCESS | SUCCESS | — | | `documentation-checkpoint` | FAILURE | FAILURE | `1c62d9974` + `8fa900a62` ONLY — the two an operator decision covers, verified against the job log rather than assumed | | `windows-msvc-cpu`, `windows-msvc-vulkan` | FAILURE | FAILURE | the two no-baseline PR-only jobs, [#584](#584) | The three CPU legs all fail on **`282 - test_runner`**, and the failure is inherited rather than this row's. The failing case, `runner: initialize_kv_cache refuses a non-multiple-of-16 block size`, has **0** occurrences at `b7d89b43e` — the head where all three legs were SUCCESS — and **1** at the merge base `5453e571d`. It was introduced by `e2a9e035d` (#1273), which **is** among the 38 commits the merge brought in, and this row's range touches **0** files in that test (control: it does touch the two `tests/vllm/models/test_nemotron_h_*.cpp` files, so the query discriminates). It is also **not a data race**, which matters because `sanitize-cpu (thread)` is the lane that would expose one. All three legs report the SAME deterministic assertion — `test_runner.cpp:1557`, `20 test cases / 19 passed / 1 failed`, `544 assertions / 543 passed / 1 failed` — and **0** sanitizer findings in any of them (`WARNING: ThreadSanitizer` = 0, `ERROR: AddressSanitizer` / `runtime error:` = 0). A race cannot make a non-sanitized `build-test-cpu` fail identically. The assertion expects `"Block size must be a multiple of 16"` but the attention registry refuses first, with `"No valid attention backend for device type 0 from {CPU_ATTN: [block_size not supported], FLASH_ATTN: [block_size not supported]}"` — which is precisely what #1602 describes. Nothing is suppressed here and nothing is pragma'd, because there is nothing of this row's to suppress. ### Gate table | leg | state | |---|---| | seam extension, device arm, production wiring | DONE; built on BOTH arms, `-Werror`, 0 warnings | | the CPU-reachable half, gated through a real `GPUModelRunner` | DONE and RUN, with M1/M2 red-first | | routing allowlist narrowed, [#1410](#1410) filed | DONE | | synthetic device `lm_head` numeric gate | COMPILES; CUDA-only; **NEVER RUN** | | `nvcc` build of the Marlin kernel, real-checkpoint numeric leg, token identity, reachability deletion mutation, the `VT_NVFP4_MARLIN=0` fallback red | PENDING a `dgx:gpu0` window, job queued | A reviewer should treat every CUDA leg as unrun. The spec's `## 6. Now` carries the same tables, so the pending state lives with the row rather than only in this description. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code] --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Four record gates return rc 0 on a
docs/STATUS.mdcarrying a spliced mergeconflict. Each measures its own budget, and none asks whether a merge tool wrote
into the file at all, so a keyed row that is half one branch and half another
satisfies every budget it is measured against. An earlier revision of #1414's
branch carried exactly that file, the full record gate set ran, and it reported
clean; a person caught it by reading
git diff --stat.This adds one tree-scoped checker, in one place rather than four, and wires it
where the pre-push record gates already run.
The rule, and why the separator is conditional
A line that starts with seven
<and a space opens a hunk, and a line thatstarts with seven
>and a space closes one. Either fails, anywhere in a trackedtext file. A line of exactly seven
=fails only when a start marker opened ahunk above it.
The separator stays conditional because a bare row of
=is legal markdown: itis the setext heading underline and it is a horizontal rule. Two independent
narrowings keep the gate off ordinary text, and both were measured rather than
assumed. Exactly seven: five shipped files already carry a rule of eight or
more
=(three logs underdocs/bench-evidence/gdn-replayssm-w0-20260818/andtwo tokenizer corpora under
tests/parity/goldens/), so a^=+rule would havebeen red on arrival. Adjacency: even a line of exactly seven is silent unless
a real marker opened a hunk.
The diff3
|||||||marker is deliberately absent. A diff3 conflict still carriesthe start and end markers, so nothing escapes, and a line of seven
|is aplausible empty row in a repository whose records are wide markdown tables.
No allowlist, and no per-change registration
Every pattern is built by character repetition rather than written as a literal,
so the checker and its suite carry no marker at column 0 and need no exemption
from the gate they implement. The suite builds its fixtures in temporary git
repositories. Nothing in the repository has to register itself with this gate,
and no file accumulates a list that every change must append to. AGENTS.md
§ Records forbids that surface: if N concurrent pull requests edit file F, F is
a lock.
A document that quotes a conflict marker on purpose indents it. That remedy is
printed in the failure message, and it is per instance and in the text, not a
registry entry.
RED-BEFORE
The defect was re-derived rather than quoted, on a detached scratch worktree at
b537a5344. Five lines were spliced into thedocs/STATUS.mdcapability table:a start marker, a duplicated keyed row, a separator, the same row again, and an
end marker. The mutation asserted its own application against the file's sha256
before and after, and
git diff --statreaddocs/STATUS.md | 5 +++++.The two range-scoped checkers ran over a detached scratch commit, not over
the working tree. A working-tree mutation of a commit-reading checker returns 0
because the checker never reads the mutated bytes, and that 0 looks exactly like
a gate that cannot detect the defect.
The suite was written before the checker existed and was seen failing:
Ran 17 tests,FAILED (failures=17), rc 1.GREEN-AFTER
Same tree, same splice, the new gate:
On the clean tree of this branch:
3739 tracked text files examined, out of 4817 tracked paths: 1077 binary, 1
symlink and 0 absent were skipped, and each class is counted and printed rather
than dropped in silence. (It read 3733 of 4811 before the last merge of
main;both figures are the same run on different trees.) A gate that cannot say how
many things it examined has not reported, so an examined count of zero exits 2
instead of printing a clean verdict, and a file the run could not READ exits 2
as well rather than being called clean. The report also names the resolved root,
because a checker resolves its root from its own path and a run from a linked
worktree can otherwise print OK about the shared checkout.
The clean-tree case does not assert a floor. It asserts EQUALITY with git's own
tracked text set, derived at read time from
git grep -I --name-only -e ''plusthe tracked files that have no lines at all. Both sides are re-derived on every
run, so nothing is stored and nothing drifts; the two agreed byte for byte.
The suite:
Ran 21 tests,OK,cases run: 21, rc 0. The count is asserted bythe runner itself, so a suite that ran zero cases exits 2 rather than printing
OK.
Cost: 0.29 s of CPU for the whole tree, against 0.9 s before the scan was moved
off decoding 142 MB of text to bytes. Wall time on this box ranged 1.17 s to
4.60 s under load from two sibling sessions, which is why the spec's stop
condition bounds CPU and not wall.
The gate is armed, proven by mutation
Each mutation was applied in a scratch worktree under
/dev/shm, never in thereviewed tree. Each asserted that its bytes actually changed, printed the new
sha256, compiled the result before running anything, and restored the file
byte-identically afterwards.
scan_textreturns nothing)open_at is not Nonedroppedreturn names)Each mutation reds the case that owns it. M1 to M3 were re-run after the
refactor, so no earlier commit's mutation proof was silently disarmed by a later
one.
M2 is the finding this pull request is most worth reading for. On the first
run it stayed GREEN: the adjacency guard was in the source and no case could
tell whether it was there. A file with no marker leaves the scan early, so the
guard never decided anything in any fixture, and a new gate carrying a branch no
mutation can falsify has the same defect it was added to fix.
test_a_separator_after_the_hunk_closes_is_not_namedwas written to close it. Itgives a file a closed hunk and then a legal setext underline below it; the exit
code is 1 either way, so the case asserts the report, and it carries its own
positive control (three named lines) before it claims a fourth is absent.
M4 is green on purpose and is recorded as such. The whole-buffer reject is a pure
optimization that decides nothing the per-line pass would decide differently, and
its docstring says so, so a later reader cannot mistake it for the semantic
guard.
The four existing checkers are unchanged
Byte-identical to
origin/main, verified by sha256 rather than by eye:All four still pass on this tree, tree-scoped and range-scoped, at rc 0. This
adds a gate; it alters no existing gate's semantics.
Repairs from the fresh review
The review returned PASS with two LOW findings. Both are repaired here, because
both bear on the principle this change elevates.
An unmerged index triple-counted.
git ls-filesemits stages 1, 2 and 3 fora conflicted path, so during a live merge the checker read one file three times
and reported
9 findings in 1 file; examined 3 tracked text files. The verdictstayed 1, so nothing was ever missed, but the count was wrong in exactly the
state the gate exists for, and no fixture built an unmerged index.
tracked_pathsnow deduplicates withdict.fromkeys, which keeps git's order.test_an_unmerged_index_is_counted_oncemakesgit mergegenuinely fail andasserts the three stage entries before it asserts anything else, so the dedupe
is falsifiable rather than merely present.
The clean-tree floor was 27% of the real count.
> 1000against a real 3733would have stayed green over a scan that collapsed to markdown alone. It is now
equality with git's own text set, as described above.
Three accuracy repairs came with them. An unreadable file was filed as a
finding, so an
OSErrorexited 1 and printed "Resolve the merge beforecommitting"; it now takes its own list and exits 2, because a file this run
could not read is a file it cannot call clean. The summary file count split each
report line on its first colon and merged two tracked paths sharing a prefix.
And the creation-mutation comments in
check-pr-size.pyclaimed the disabledstub fails every case that runs the checker: measured, it fails 16 of 21,
and the five survivors assert only that an ordinary document exits 0, or never
run the checker at all. The mutation is still detected; the sentence was wrong,
and it is corrected in both places.
The spec's
## Risksnow records the unmerged index as not seen when thespec was written, rather than as a trade-off that was weighed and dropped.
Where it runs
CHECKERSinscripts/agent-preflight.sh, deliberately first, so a reader whosees it fail knows to stop reading the verdicts below it;
SUITESfor themutation suite; and a step in the
agent-recordjob of.github/workflows/ci.yml, so a pull request is gated even when preflight wasnever run. The workflow was re-parsed with a loader that rejects duplicate
mapping keys: 17 jobs, one registered step. PyYAML accepts workflows GitHub
rejects, and a duplicate key yields zero jobs.
One in-flow repair, tracked separately as #1448
Registering a new checker requires an entry in
CREATION_MUTATIONS, so thischange necessarily edits
scripts/check-pr-size.py, and a change to that filemust prove its own HEAD checker/test pair passes. It could not:
test_every_tracked_and_current_change_path_is_classifiedsweeps every trackedpath, and ten files of
docs/bench-evidence/gdn-replayssm-w0-20260818/have beenunclassified since 2026-08-18, because
BENCH_EVIDENCEmatches exactly one pathsegment. That suite is RED on
origin/main, verified in a detached worktree at9ecaf1bb3, and it is in no preflightSUITESentry, so preflight is green overit. It surfaces only as
ERROR: HEAD checker/test pair failed for 'scripts/check-pr-size.py', which reads as a defect in the change under review.Repaired here, per AGENTS.md § Every change starts from an issue: a
BENCH_EVIDENCE_RUNpattern that names the surface rather than widening a rule,restricted to the extensions the directory carries and deliberately excluding
.mdand.json, which already classify aspublic_documentand would besilently reclassified because the
evidencearm is tested first. Classificationof all 4811 tracked paths was captured before and after: exactly those ten move,
and no other path changes class. Issue #1448 tracks it and is not closed by this
body's keyword, so it can be closed by hand after review confirms the repair.
Fourth instance of that class after #856, #668 and #989.
Gates
All re-run after the last merge of
origin/main0adeb8b0e, so the base is anancestor of HEAD and the trailer gates EXECUTE rather than skip.
python3 scripts/check-conflict-markers.py— rc 0, 3739 files examined.python3 tests/scripts/test_check_conflict_markers.py— rc 0, 21 cases,OK.python3 scripts/check-agent-record.py— rc 0, re-run after the merge.python3 scripts/check-public-doc-tables.py— rc 0.python3 scripts/check-doc-checkpoint.py --base origin/main --head HEAD— rc 0.python3 scripts/check-issue-index-append-only.py --base origin/main --head HEAD— rc 0.python3 scripts/check-pr-size.py --base origin/main --head HEAD— rc 0,OK: every explicit path class is within its review budget.python3 scripts/check-commit-trailers.py --range origin/main..HEAD— rc 0over 7 commits, EXECUTED rather than skipped.
python3 scripts/check-commit-style.py --range origin/main..HEAD— rc 0, samerange.
One accounted-for red, and it is not this change's:
test_cpu_x86_llamacpp_floorreportsNO_QUIET_WINDOWexit 4 where the caseexpects 2. That is #618's load dependence, measured on this box at load average
122-129 and later 157-225, driven by sibling agent sessions. This branch touches
no benchmark harness, and the same suite re-run on the same sources at load
average 2.5 reported
Ran 10 tests,OK, rc 0.The full
scripts/agent-preflight.shon this head, after the merge: 85 gatesok, ZERO skipped, and rc 1 for that one suite alone.commit-trailersandcommit-styleboth readokrather thanSKIP, which is the point of mergingmainfirst: a skipped gate reports nothing about the tree, and the harnessrefuses the green banner over one.
windows-msvc-cpuandwindows-msvc-vulkanare red on every pull request(#584, #968) with no
mainbaseline. This diff carries no C++.Closes #1417
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]