From 522a458d71406a6d9942fd58fe5b6a32f81a9cf4 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Wed, 19 Aug 2026 07:47:02 +0000 Subject: [PATCH] fix(ENG-RECORD-ANCHOR-RATCHET): drop the live OK count, which was false the moment it landed (#1324, #1325) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit At `af87251c5`, `docs/BENCHMARKS.md:14` and `docs/STATUS.md:100` published `844` correct citations for this row. At the row's own merge commit `678fc672c` the tree reads `ok=847`: three good citations landed on `main` between verification and merge, so the figure was already wrong when it shipped. `.agents/engine-matrix.md` and `.agents/specs/record-anchor-ratchet.md` carried the same number. Chasing it to 847 fixes nothing. `ok` moves on every merge that adds or removes a citation, so 847 is false again at the next landing, and a live measurement stored in a document is what AGENTS.md §Records forbids by name: "Never store a measurement of one file inside another file. A number that changes after each edit couples every pull request to lines that it does not own." The count is therefore DROPPED rather than anchored to a SHA. Both shapes are defensible; this one was chosen for three reasons. First, `scripts/record-anchor-baseline.json` already made this decision for the machine-read surface, and says so in its own `_comment`: the OK count "is printed by --report but kept out of this file on purpose". Dropping it from the human-read surfaces makes the two agree, instead of stating a rule in one file and breaking it in four others. Second, no information is lost. The same BENCHMARKS cell already carries "832 of 867 in-scope citations (96.0%)" anchored at `8daa67b39`, so the reader keeps the sense of scale, from a figure that cannot drift. Third, anchoring would preserve a number no gate reads (#1324) in a projection document, where a reader takes what is written as current. Dropping removes the class rather than restating it in a form that still needs a human to notice. What remains stated is the rot -- 38, 32 stale, 6 broken, 32 in range -- which `check-agent-record.py` gates against the baseline in both directions, so it cannot move without a commit that says so. The BENCHMARKS cell falls from 219 to 203 characters against the 220 cap. Two findings from the same review are filed and NOT fixed here, because each is a semantic checker change owing its own spec, red-before mutation and fresh reviewer. #1324: this row's public figures are read by no gate. Measured at `af87251c5`, restoring the wrong pre-#851 values (rot 39, 7 broken, 828 OK) into the BENCHMARKS cell leaves `check-public-doc-tables.py` exit 0 AND `check-agent-record.py` exit 0, while the same run derived `stale=32, broken=6` beside them. The ratchet reads only the `code` and `tests` cells of matrix rows, so `docs/` prose sits outside `RECORD_ANCHOR_FIELDS` by construction. Rounds four and five of the #851 review were each spent on this class and each caught by a human reading; the `844` above is the third instance, and it landed. #1325: `scripts/record-anchor-baseline.json` stores a `"total"` that no code reads. `load_record_anchor_baseline` returns the two buckets only, and `write_record_anchor_baseline` refuses a raise against their `sum()`, so mutating `"total": 38` to `39` is inert and every gate stays green. The row's own budget file therefore carries the shape the row exists to name. Both are listed under `## Owed` in the row's spec and appended to `.agents/issue-index.md` naming `ENG-RECORD-ANCHOR-RATCHET` as the owning row. Gates at this head: `check-agent-record.py --report` exit 0, `ok=847, stale=32, broken=6 -> rot 38`, agreeing with the baseline's `"stale": 32, "broken": 6`; `check-public-doc-tables.py` exit 0. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code] --- .agents/engine-matrix.md | 2 +- .agents/issue-index.md | 2 ++ .agents/specs/record-anchor-ratchet.md | 35 +++++++++++++++++++++++++- docs/BENCHMARKS.md | 2 +- docs/STATUS.md | 4 +-- 5 files changed, 40 insertions(+), 5 deletions(-) diff --git a/.agents/engine-matrix.md b/.agents/engine-matrix.md index 49f6326fe..776be0970 100644 --- a/.agents/engine-matrix.md +++ b/.agents/engine-matrix.md @@ -222,7 +222,7 @@ claims it. | `ENG-RELEASE-WINDOWS` | Native Windows x86_64 pre-alpha release extension: one adaptive MSVC/UCRT CPU bundle with AVX2 executed in CI and one Vulkan preview bundle, both deterministic ZIPs and authenticated by the existing release handoff | T0 | vLLM has no Windows release path; runtime behavior remains pinned to vLLM `555967922`. Platform substrate reference: llama.cpp `src/llama-mmap.cpp:520-590` @ `237ad9b961f009ae19ac29dbce4cd0c1251f94b3`; Win32 API is the OS authority | W14 Win32 portability/MSVC CPU, W15 deterministic ZIP/PE packaging + Vulkan, and W16 ten-tuple prerelease workflow/version/docs implemented for one PR | Linux portability/release mutation gates are local evidence only. Native `windows-2022` MSVC `/W4 /WX`, extracted runtime/ISA smokes, merged-SHA ten-tuple dry run, `v0.0.3-pre.1` publication, attestations, and exact 32-asset audit remain pending; no Windows ZIP exists yet | [windows-binary-release.md](specs/windows-binary-release.md); [#117](https://github.com/mudler/vllm.cpp/issues/117) | `ACTIVE` | `CLAIM-ENG-RELEASE-WINDOWS` | | `ENG-RELEASE-CONTAINERS` | Published OCI container images on GHCR, built by GitHub Actions: the same staged server bundle as `ENG-RELEASE-BINARIES`, shipped from one package `ghcr.io/mudler/vllm.cpp` with the lane in the tag — `:-cuda` / `-vulkan` / `-cpu`, the moving `:latest-cuda` / `:latest-vulkan` / `:latest-cpu`, and a bare `:latest` aliasing the cpu lane, with `ENTRYPOINT vllm-server`. Lanes `cuda` (one fat image covering every supported SM), `vulkan`, `cpu` (adaptive baseline); `rocm` blocked-preview, tracking its binary channel. Version tags are immutable; every `latest-` moves. Each lane is a `linux/amd64` + `linux/arm64` multi-arch manifest built on native runners — aarch64 is first-class here because GB10 (sm_121a), Thor (sm_110) and Orin (sm_87) are all arm64. The image contains the bundle and nothing else: no weights, no Python, no PyTorch, no compiler, no build tree. BOUNDARY: the GPU driver and container runtime stay on the host and are never bundled; Metal and MLX are NOT-CONTAINERIZABLE (no macOS container runtime and no Metal passthrough exists) and remain static-binary-only lanes, recorded as a permanent boundary rather than pending work. No image, workflow, registry package or pull is claimed to exist. | T0 | release image lanes `.buildkite/release-pipeline.yaml:34-170` and the published-image dependency boundary `docker/Dockerfile.cpu:262-290` @ `555967922` | `docker/Dockerfile` (cpu/vulkan/cuda targets calling the release scripts); `docker/healthcheck.sh`; `release/container-matrix.json`; `scripts/check-container-matrix.py`; `scripts/check-container-workflow.py`; `scripts/validate-container-image.py`; `scripts/container_tags.py`; `.github/workflows/containers.yml`; SIGTERM handler `src/vllm/entrypoints/openai/server_main.cpp` (`SignalShutdown`, all three `listen()` sites); the pre-existing `docker/Dockerfile.arm64` is an unrelated CPU bench cross-check | issues `#170`, `#312`, `#394`; `tests/scripts/test_check_container_matrix.py` 31/31; `test_check_container_workflow.py` 29/29; `test_check_cuda_fat_gencode.py` 7+4 subtests. **GB10 2026-08-11 (`promaxgb10-4ad8`, `sm_121a`, CUDA 13.3): arm64 cuda image 1.71 GB, 673/673 objects, ten-SM gencode audit PASS, and a REAL GPU boot -- `/health` 200, `/version` 200, in-container healthcheck, clean SIGTERM, `--gpus all`, host driver 580.159.03 injected.** cpu amd64 783 MB gated locally; cpu+vulkan amd64 green on hosted CI **arm64 cuda lane RUNTIME-VERIFIED on GB10 2026-08-11** -- the first accelerator-hardware evidence for any lane. Four defects were removed to get there, each found by building rather than reading: the CUDA 12.9 base could not compile `sm_110`, the BuildKit cache mount outlived its toolchain (both #366), Marlin gencode had drifted from the feature table and failed the audit on 14 correctly-compiled TUs (#394, blocking BOTH cuda tuples project-wide), and the validator could only ever produce build evidence because its boot smoke never passed `--gpus`. **NOT established: nothing is published to GHCR; amd64 cuda is unbuilt; the published arm64 image is SBSA (`targets/sbsa-linux`), so Tegra -- Thor `sm_110`, Orin `sm_87` -- is untested and NOT covered** **ORIN (Tegra) 2026-08-11: the SBSA image RUNS on Jetson AGX Orin `sm_87` (L4T R36.4.3, Docker 27.5.1) -- Qwen3-0.6B (rev `c1899de2`) loads and GENERATES via `/v1/completions`, tegrastats GR3D 95-97% during decode vs 14-15% idle.** Tegra needs `--runtime nvidia --gpus all`: `--gpus` alone is refused by the hook and `--runtime` alone mounts no driver | [container-images.md](specs/container-images.md); issues [#170](https://github.com/mudler/vllm.cpp/issues/170), [#312](https://github.com/mudler/vllm.cpp/issues/312), [#394](https://github.com/mudler/vllm.cpp/issues/394) | `ACTIVE` | `CLAIM-ENG-RELEASE-CONTAINERS-W1-W7` | | `ENG-DOCS-SITE` | Publish the 11 `docs/*.md` as a browsable GitHub Pages site at `https://mudler.github.io/vllm.cpp/` WITHOUT a second copy of the prose. A Hugo site at `website/` mounts `../docs` READ-ONLY and derives everything else from what is already in the files: each page title from the file's first `# H1`, the sidebar order from `website/data/nav.yaml`, and links through a Goldmark render hook (internal `.md` → site URL; the 139 `../.agents/**` and `../AGENTS.md` escapes → GitHub blob URLs, since the protocol tree is deliberately NOT published). **No file under `docs/` is modified, moved, renamed, or given front matter**, so `check-doc-checkpoint.py` and every protocol path reference keep working and there is no second surface that can drift — the whole point of the row. Custom lean layouts, NO theme and NO submodule: off-the-shelf docs themes read titles, weights and menus out of front matter this design deliberately does not have, so each would need its title partial, menu and link hook overridden anyway, and hugo-book additionally floors at Hugo 0.158 against the 0.146.3 pin CI and the local toolchain share. Hard prerequisite inside the repo: `classify_path` in `scripts/check-pr-size.py` FAILS CLOSED on `website/**` (verified: raises `ValueError: unclassified repository path`), so the classifier must learn the path or the PR cannot pass the project's own size gate. Hard prerequisite outside it: GitHub Pages must be enabled with the source set to GitHub Actions — the workflow is inert otherwise. A marketing landing page is explicitly OUT of scope (`README.md` stays the front door), as is any restructuring of `docs/`; the custom domain is parked behind the pending vLLM trademark question | T1 | NO vLLM analogue — upstream's docs are a separate mkdocs site and nothing in this row mirrors upstream *behavior*, so it carries no parity obligation. The STRUCTURAL reference is LocalAI's `.github/workflows/gh-pages.yml` (two Hugo sites merged into one Pages artifact), reduced to the docs half | read-only mount `website/hugo.toml:29`; title-from-H1 `website/layouts/partials/title.html:10`; link rewriting `website/layouts/_default/_markup/render-link.html:27`; guard `scripts/check-site.py:70`; deploy `.github/workflows/gh-pages.yml` | `tests/scripts/test_check_site.py:51,56,66,80,89,97` (6 mutation cases: clean tree, H1 stripped, doc absent from nav, nav entry with no file, duplicated entry, missing nav file); build evidence 14 pages with `docs/bench-evidence` + `docs/superpowers` absent from `public/` and no `href` ending in `.md`; 48 protocol links rewritten in `docs/status/`. NO published page is claimed: GitHub Pages is not yet enabled on the repository, which is the recorded stop condition holding this row at `GATING` | [gh-pages-docs-site.md](specs/gh-pages-docs-site.md); issue [#224](https://github.com/mudler/vllm.cpp/issues/224) | `READY` | `CLAIM-ENG-DOCS-SITE` | -| `ENG-RECORD-ANCHOR-RATCHET` | **The record's `path:line` citations were range-checked and never reported.** `check-agent-record.py` parsed BOTH forms: markdown links, and bare `` `file.cpp:123` `` through `RAW_LOCAL_ANCHOR_RE` since `ee511ca8a`. On a missing file or an out-of-range line `local_line_anchors` runs `continue`, so the bad anchor never reaches the caller, and `is_code_anchor` then answers with **any**, so one good sibling covers the rest. There was no symbol test and no report, and **32 of the 38** offenders are IN RANGE, so range-checking could not have found them. Measured at `8daa67b39`: **832 of 867** in-scope citations (**96.0%**) were already parsed and range-checked, and the **35** new to parsing sit under `.agents/`, `docs/` and `website/`; `EVIDENCED_STATES` omits `ACTIVE`/`READY` entirely and is deliberately NOT widened, because requiring an anchor there raises 85 errors across 53 rows. Even the fraction it saw was only range-checked, never checked to CONTAIN the symbol named beside it — every stale anchor found in the 2026-08-13/14 campaign was in range. LANDED as a device-leakage-shaped ratchet over a recorded baseline, never a bulk cleanup: the backlog is fixed by whoever next touches each row | T1 | none — this is our own record surface; the discipline mirrors AGENTS.md §Records ("cite the `file:line` you ported from") | parser + classifier + ratchet in `check-agent-record.py`: `BARE_CITATION_RE` `scripts/check-agent-record.py:1204` (the bare form), `cell_citations` `scripts/check-agent-record.py:1258` (both forms, with the adjacent-symbol rule), `classify_citation` `scripts/check-agent-record.py:1315` (OK / STALE / BROKEN), `RECORD_ANCHOR_STATES` `scripts/check-agent-record.py:1202` (gap 3: `ACTIVE` and `READY` join the count), `check_record_anchors` `scripts/check-agent-record.py:1438` (the two-way gate); budget in `scripts/record-anchor-baseline.json` | `RecordAnchorRatchet` `tests/scripts/test_agent_record.py:1397` — 10 cases, RED-first, including `test_one_good_link_does_not_cover_a_rotted_bare_citation` `tests/scripts/test_agent_record.py:1465`, the `any()` shape the rot hid in. Five mutants red it: report-only, `EVIDENCED_STATES` restored, links-only, first-citation-only, range-only. Measured baseline **38** (32 STALE + 6 BROKEN) over 844 OK; gate wired in `scripts/agent-preflight.sh` and the `agent-record` CI job (`--report`) | [record-anchor-ratchet.md](specs/record-anchor-ratchet.md) | `ACTIVE` | `CLAIM-ENG-RECORD-ANCHOR-RATCHET` | +| `ENG-RECORD-ANCHOR-RATCHET` | **The record's `path:line` citations were range-checked and never reported.** `check-agent-record.py` parsed BOTH forms: markdown links, and bare `` `file.cpp:123` `` through `RAW_LOCAL_ANCHOR_RE` since `ee511ca8a`. On a missing file or an out-of-range line `local_line_anchors` runs `continue`, so the bad anchor never reaches the caller, and `is_code_anchor` then answers with **any**, so one good sibling covers the rest. There was no symbol test and no report, and **32 of the 38** offenders are IN RANGE, so range-checking could not have found them. Measured at `8daa67b39`: **832 of 867** in-scope citations (**96.0%**) were already parsed and range-checked, and the **35** new to parsing sit under `.agents/`, `docs/` and `website/`; `EVIDENCED_STATES` omits `ACTIVE`/`READY` entirely and is deliberately NOT widened, because requiring an anchor there raises 85 errors across 53 rows. Even the fraction it saw was only range-checked, never checked to CONTAIN the symbol named beside it — every stale anchor found in the 2026-08-13/14 campaign was in range. LANDED as a device-leakage-shaped ratchet over a recorded baseline, never a bulk cleanup: the backlog is fixed by whoever next touches each row | T1 | none — this is our own record surface; the discipline mirrors AGENTS.md §Records ("cite the `file:line` you ported from") | parser + classifier + ratchet in `check-agent-record.py`: `BARE_CITATION_RE` `scripts/check-agent-record.py:1204` (the bare form), `cell_citations` `scripts/check-agent-record.py:1258` (both forms, with the adjacent-symbol rule), `classify_citation` `scripts/check-agent-record.py:1315` (OK / STALE / BROKEN), `RECORD_ANCHOR_STATES` `scripts/check-agent-record.py:1202` (gap 3: `ACTIVE` and `READY` join the count), `check_record_anchors` `scripts/check-agent-record.py:1438` (the two-way gate); budget in `scripts/record-anchor-baseline.json` | `RecordAnchorRatchet` `tests/scripts/test_agent_record.py:1397` — 10 cases, RED-first, including `test_one_good_link_does_not_cover_a_rotted_bare_citation` `tests/scripts/test_agent_record.py:1465`, the `any()` shape the rot hid in. Five mutants red it: report-only, `EVIDENCED_STATES` restored, links-only, first-citation-only, range-only. Measured baseline **38** (32 STALE + 6 BROKEN); gate wired in `scripts/agent-preflight.sh` and the `agent-record` CI job (`--report`) | [record-anchor-ratchet.md](specs/record-anchor-ratchet.md) | `ACTIVE` | `CLAIM-ENG-RECORD-ANCHOR-RATCHET` | | `ENG-RECORD-CONFLICT-SURFACES` | Retire the shared record surfaces that make concurrent PRs conflict by construction. MEASURED at `origin/main` `d928e2c3` with `git merge-tree --write-tree` over every open PR: **16 of 29 conflict (55%), and 13 of the 16 conflict in bookkeeping files ONLY**, with no product code involved — `.agents/coordination.md` in 8, `.agents/NOW.md` in 5, `.agents/roadmap_v1.md` in 4, `scripts/check-public-doc-tables.py` in 4, `docs/STATUS.md` in 4, and any `src/`/`tests/` path in just 3. Three defects, each of which GUARANTEES rather than risks a collision. (1) `.agents/NOW.md` is a fixed-size shared buffer at EXACTLY 6000/6000 chars (`check-now-current.py:31`), so adding a row requires evicting another and every PR is a read-modify-write of one global — and the conflict is the LUCKY outcome, since a clean three-way merge would apply both evictions and both additions, silently dropping live rows and blowing the very budget the checker defends. (2) `STATUS_RATCHET = {"chars": 243245}` (`check-public-doc-tables.py:557`) is a hardcoded byte count of a DIFFERENT file that may only fall, so a PR owing `docs/STATUS.md` one lifecycle line must delete unrelated prose from another row to pay for it and edit the checker too; the checker's own comment at `:331` already records the failure (*"a ratchet pinned to the byte turns every concurrently merged row's one-line status edit into a spurious failure"*) and answered it with slack instead of removing the coupling. (3) `.agents/coordination.md`'s active-claims table is insert-at-one-anchor: the six ROCm GDN PRs (#334 #336 #341 #343 #345 #348) are ONE author's sequential stack that conflicts on nothing else, each appending a ~1,500-char row — the PR description, transcribed into a file every other claim also writes. It also contradicts the protocol it serves: `AGENTS.md` holds that *"History is git"* and *"There is no state log"*, yet both claims tables ARE state logs duplicating `gh pr list`, `row/` branch names and issue state; the argument that refuses a waiver registry applies unchanged to a claims registry. Precedent twice over — `policy.csv` retired in `0f3e44ee`, per-class line budgets retired 2026-08-10 because the gate fired on ordinary work. The exonerated surfaces share ONE property, one writer per file: `.agents/specs/.md` (one file per row, **zero conflicts** in the sample), the `*-matrix.md` inventories, and the append-only `.agents/benchmark-record.md`. SCOPE: remove `STATUS_RATCHET` and the doc-gating global counters while KEEPING the per-cell/per-paragraph caps (local, so they couple nothing); remove the active-claims table and derive claims from open PRs and branch names; drop `NOW.md`'s byte budget; order the roadmap's keyed tables by ID so distinct keys stop colliding at one anchor; and record the invariant — **no surface that every PR must write** — in `AGENTS.md`. No product source, kernel or gate semantic moves | T0 | NO vLLM analogue — this is local protocol machinery, so the mirror rule does not apply and no upstream `file:line` exists to port from. Governed instead by `AGENTS.md` §"Changing the rules or a checker", which requires a spec, a red-before test or mutation, and green-after evidence | - | - (spec-before-code: the red-before suites are named in the spec's Tests section — `tests/scripts/test_check_public_doc_tables.py`, `tests/scripts/test_check_now_current.py`, a mutation case per removed rule proving the obligation survives in the retained caps and `check-doc-checkpoint.py`, and a `git merge-tree` merge-shape regression that must be RED before the `NOW.md`/roadmap work and GREEN after) | [retire-shared-record-surfaces.md](specs/retire-shared-record-surfaces.md); issue [#364](https://github.com/mudler/vllm.cpp/issues/364) | `READY` | `CLAIM-ENG-RECORD-CONFLICT-SURFACES` | | `ENG-TRAILER-MERGE-ARTIFACTS` | The trailer gate rejects CORRECT commits because of paragraph placement, and that is why `main` is red on `agent-record`. `check-commit-trailers.py` reads trailers via `git interpret-trailers --parse`, which treats ONLY the final paragraph as the block; GitHub appends `Co-authored-by:` as a SEPARATE trailing paragraph on a squash merge, so a complete correct block becomes invisible and the gate reports it missing. MEASURED: piping `git show -s --format=%B dbd0d51c` into `git interpret-trailers --parse` prints nothing but the co-author line, and 13 of the last 30 commits on `main` fail the check -- unnoticed only because those runs were cancelled (#274), which HID the defect rather than causing it. FIX: fuse consecutive trailing TRAILER-SHAPED paragraphs before parsing. Nothing is relaxed -- the block must still exist, the marker must still sit above it, each declaration must still appear exactly once, and an AI co-author is still forbidden; the block is merely FOUND where the merge tool left it. A prose paragraph still terminates it. REJECTED IN FLIGHT and recorded because it is the more instructive half: a first attempt also collapsed identical duplicate trailers to fix the multi-commit-squash shape, which relaxes the uniqueness rule an existing test already pins. Rewriting that assertion to suit the change is what AGENTS.md forbids, and the distinction is real -- a doubled block is genuinely malformed and fixable at source, whereas the co-author case is a correct commit defeated by the parser. Reverted in full. SCOPE LIMIT, stated rather than implied: this fixes ONE of five observed shapes. `f64f2b71` (bot co-author) is a REAL violation the parse had been hiding and now correctly fails; `87308dea` (GitHub's `---------` separator), `b8293c88` (squash doubled the block) and `b580452d` (merge button, no trailers) stay red by design. Closing those is a merge-method change, not a checker change | T0 | NO vLLM analogue -- local protocol machinery, so the mirror rule does not apply and there is no upstream `file:line` to port from. Governed by `AGENTS.md` §"Changing the rules or a checker" | `scripts/check-commit-trailers.py:60` (`join_trailing_trailer_paragraphs`, `_is_trailer_paragraph`, and the fused `parsed_trailers`) | `tests/scripts/test_check_commit_trailers.py:1` 21 cases -- the RED-BEFORE appended-co-author case plus four GUARDS that keep the fusion bounded (doubled block still fails, contradictory declarations still fail, a no-trailer merge message still fails, prose after the block still fails), all four green before and after; closure [parity-ledger.md#L941](parity-ledger.md#L941) | [trailer-merge-artifacts.md](specs/trailer-merge-artifacts.md); issue [#406](https://github.com/mudler/vllm.cpp/issues/406) | `DONE` | `157080c8` | | `ENG-FORGE-COAUTHOR` | The forbidden-AI-trailer rule was catching ATTRIBUTION rather than an authorship claim, which is why bot-opened PRs red `main` on merge. GitHub composes the squash message itself and appends the account that opened the PR — `Co-authored-by: localai-org-maint-bot <...@users.noreply.github.com>` — and most PRs here are opened by a bot, so nearly every squash trips the AI-identity check. Real instance `f64f2b71`, invisible until #406 repaired the parse, which is why it reads as a new failure and is not one. The rule exists so an AI cannot claim it WROTE the code, and that stays; GitHub is recording who pressed the button, and the AI-involvement claim is already carried separately by `AI-Assisted` and `Assisted-by` in the same block. FIX: accept a `Co-authored-by` at a GitHub account noreply address even when the name matches an AI identity token, keyed on the FORGE'S OWN DOMAIN rather than the name so the exemption cannot be borrowed. A hand-written `Co-authored-by: Claude ` still fails; `Signed-off-by` is excluded from the exemption entirely, because a sign-off is a legal assertion about provenance rather than attribution. `AGENTS.md` records the same distinction in the same change so prose and checker cannot drift | T0 | NO vLLM analogue -- local protocol machinery, so the mirror rule does not apply and there is no upstream `file:line` to port from. Governed by `AGENTS.md` §"Changing the rules or a checker" | `scripts/check-commit-trailers.py:38` (`FORGE_ACCOUNT_EMAIL` and the forbidden-trailer skip) | `tests/scripts/test_check_commit_trailers.py:1` 25 cases -- the RED-BEFORE forge-bot case plus THREE guards that matter more than the relaxation because this LOOSENS a rule: a hand-written AI co-author still fails, `Signed-off-by` at the same noreply address still fails, and a human co-author still passes; all three green before and after. Real commit `f64f2b71` re-verified per commit | [forge-coauthor-attribution.md](specs/forge-coauthor-attribution.md); issue [#418](https://github.com/mudler/vllm.cpp/issues/418) | `ACTIVE` | `CLAIM-ENG-FORGE-COAUTHOR` | diff --git a/.agents/issue-index.md b/.agents/issue-index.md index d4e6bcb0e..cd4a8313f 100644 --- a/.agents/issue-index.md +++ b/.agents/issue-index.md @@ -421,3 +421,5 @@ rather than merged. `scripts/check-agent-record.py` gates both. | [#1291](https://github.com/mudler/vllm.cpp/issues/1291) | `ENG-CUDAGRAPH-BREAK` | W3 of the break-point capture seam: the three remaining PLAIN BATCHED decode drivers migrate onto `vt::GraphCaptureScope` + `vt::BreakableGraph` — `Qwen3MoeDecodeGraph` (`qwen3_moe.cpp`), `VoxtralDecodeGraph` (`voxtral.cpp`) and `DeepseekV2DecodeGraph` (`deepseek_v2.cpp`) — one commit each, each landable alone, each with its own RED-first G2 gate. Four of the nine drivers are now on the seam and the six batched-driver `VLLM_CPP_CUDAGRAPH` reads the spec's `## Our baseline` item 1 counted are down to TWO, both in `qwen3_5.cpp` (W4). The two per-model rollback switches STAY (`VT_QWEN3MOE_CUDAGRAPH`, `VT_DEEPSEEK_CUDAGRAPH`): each is an A/B lever for exactly one driver, not a copy of the shared one. **Each driver owes its own gate because nothing else can see the difference** — a driver that kept its hand-rolled `BeginCapture`/`EndCaptureGraph` pair produces IDENTICAL logits, an IDENTICAL backend log and an identical `replay_count()`, so `segments_captured` and `replays` are the only observables that separate "captured a graph" from "captured a graph THROUGH THE SEAM". Red-first on four assertions each (`test_qwen3_moe_decode_graph_seam` 222/226, `test_voxtral_decode_graph_seam` 224/228, `test_deepseek_v2_decode_graph_seam` 224/228, all exit 1), 3/3 green after; the G2 mutation restoring each pre-W3 driver file (25/102, 23/92 and 25/94 lines, each compiled clean, each restored and verified by sha256) reddens ONLY its own gate and leaves `test_breakable_graph` 216/216 and W2's `test_qwen3_decode_graph_seam` 231/231 GREEN. The two swapped seams every such gate needs now live ONCE, in `tests/vllm/models/decode_graph_seam_harness.h`, with W2's file including it: three more copies inside `tests/` would have reproduced the duplication this row removes from `src/`, and two copies of a harness diverge invisibly because both files stay green while measuring different things. **G1 IS DELIVERED, retiring the item W1 and W2 both carried as owed.** `tests/vllm/models/test_decode_graph_seam_g1_cuda.cpp` drives each migrated driver COLD, CAPTURE and THREE consecutive replays and compares every step bit for bit against the driver's own EAGER arm, which is selected by `max_num_reqs == 0` (`PadToCaptureSize` returns -1 and `Step` falls out to its plain forward) so both arms are ONE binary on ONE device rather than two builds, each with its OWN device KV cache so neither can read the other's writes. Measured on `thor:gpu0` through an `rc` lease — NVIDIA Thor sm_110, driver 595.78, nvcc 13.0.88, source `c905bb536`, CUDA-ON build for arch 110, 32 `.cu.o` objects, binary resolving `libcudart.so.13` and `libcublasLt.so.13`, build 489 s at `-j4`: **3 cases, 1600 assertions, exit 0**, `5 steps x 100 logits, 0 differing, 4 replays` for each driver. The ASSERTION COUNT carries that claim and the status line does not: with no CUDA backend every case skips and the same file prints `Status: SUCCESS!` over `assertions: 0`, which the file's own header names as a skip wearing a pass. Bounded honestly: the models are the synthetic tiny ones the CPU forward gates use rather than a checkpoint, and W2's `Qwen3DenseDecodeGraph` is covered by shared-seam ARGUMENT and not by one of the three cases. **W3 ALSO FOUND A GATE THAT COULD NOT FAIL, and closing it is part of the stage.** The three gates assert `breaks_registered == 0` to hold the capture to vLLM's decode arm (`FULL_AND_PIECEWISE`, `vllm/config/compilation.py:63,65-66,630-632` @ `5559679229`). That is a TAUTOLOGY for these models: the counter moves only when a `vt::GraphBreak` registers into a splitting scope, and the ONE production break point in the tree is W1's, in `qwen3.cpp` — W2's driver runs through it, none of W3's three does. Measured rather than reasoned: flipping `kFull` to `kPiecewise` in `qwen3_moe.cpp`, one token, compiled clean and left that driver's whole gate GREEN at 226/226, and the same flip was equally invisible in the other two. The mode was UNOBSERVABLE from outside a driver — the scope is a `Step` local, the container is private to the driver's `Impl`, and a token gate cannot see a segment count — so `vt::GraphBreakStats` gains `full_scopes` and `piecewise_scopes`, counted in `GraphCaptureScope`'s constructor on the ACTIVE path only (an inert scope makes no backend call in either mode, so counting it would report a mode that never reached a backend), cleared by `ResetGraphBreakStats()`, gated in `tests/vt/test_breakable_graph.cpp` with the inert arm as the control that stops them degenerating into "scopes constructed". Re-measured after the fix, the same one-token flip now REDS each of the three gates on exactly those two assertions (226/228, 228/230, 228/230, all exit 1). **NO break point is registered in these three models, and that is a decision rather than an omission**: under `kFull` a break point takes the same pass-through arm it takes outside a scope, so registering one would land machinery no gate can exercise; establishing each model's break-point set is what the PIECEWISE arm needs, that arm is blocked on replay-safe closure inputs (W4), and W6 is where the eligibility predicate moves. **The async device-token decline, stated per driver rather than left as an omission.** `VoxtralDecodeGraph` needs none: its only construction site is `VoxtralGenerateGreedy`, the single-sequence multimodal greedy loop, which is not reached from `runner.cpp` and has no asynchronous device mirror. The other two are a NEW FINDING, filed [#1305](https://github.com/mudler/vllm.cpp/issues/1305) with W4 as owner and NOT fixed in flow, because adding a decline trades a shipped default-ON capability away on a measurement W3 cannot make and the fix `qwen3.cpp:1106`'s own comment names is `StepDevInputs` as a seam capability. **G5's ROCm/Tenstorrent arm is NOT discharged**, and the owner moves from W3 to W5 with the reason stated: the fleet carries no ROCm and no Tenstorrent device, so it is BLOCKED on hardware rather than unattempted. What W3 can say instead is that the seam's CUDA arm now runs on TWO architectures, sm_110 here and sm_121a for W1's exit criterion. Still NO throughput claim anywhere: the prefill refutation stands (3.8% host idle, >96% GPU-busy, 92.5% glue) and `kFull` keeps each migrated step's shape the one it already had | feature | | [#1305](https://github.com/mudler/vllm.cpp/issues/1305) | `ENG-CUDAGRAPH-BREAK` | Three decode-graph registrations route an ASYNCHRONOUS step into a host-vector replay with NO `device_token_ids` decline. Found while migrating two of them onto the break-point capture seam ([#1291](https://github.com/mudler/vllm.cpp/issues/1291)) and NOT caused by that migration — the shape is present at its base commit `5d9fe332c`. `src/vllm/model_executor/models/qwen3.cpp:1106` returns `std::nullopt` whenever `ModelForwardInput::device_token_ids` is non-null, on a measured four-concurrent battery (`depth-1 graph ON PASS 78/78`, `depth-2 graph OFF PASS 82/82`, `depth-2 graph ON FAIL, slots 1-3 degenerate`, #323/#1179), because `Step()` replays against persistent HOST `token_ids` while the asynchronous combine has patched the DEVICE ids and deliberately left the host vector stale for decode rows (`src/vllm/v1/worker/gpu/runner.cpp:1265-1309,1523`). That comment calls the hazard "latent for EVERY classic-dense model, since the graph is default-ON". `qwen3_moe_registry.cpp:107`, `deepseek_v2_registry.cpp:106` and `glm4_moe_lite_registry.cpp:125` (the last constructs the DeepSeek driver) admit a pure-decode step to a driver with the same host-vector replay and carry NO such check; `grep -c StepDevInputs` returns 0 for `qwen3_moe.cpp`, `deepseek_v2.cpp` and `voxtral.cpp` against 41 for `qwen3_5.cpp`, which is the divergence the row's spec already records as the cost of nine drivers. `VoxtralDecodeGraph` is NOT affected: its only construction site is `VoxtralGenerateGreedy`, which the runner does not reach and which has no device mirror. NOT fixed in flow, for two reasons stated rather than assumed: adding a decline is a behaviour change that trades a shipped, default-ON capability away, and W3 has no four-concurrent battery for these two models to justify it; and the correct fix is already assigned — `StepDevInputs` (`qwen3_5.cpp:3894`) as a SEAM capability, which is `## Work breakdown` W4 of [eng-cudagraph-break.md](specs/eng-cudagraph-break.md) and the only version that reaches these registrations at all. Owner: **W4**, with [#1179](https://github.com/mudler/vllm.cpp/issues/1179) as the standing tracker. What would settle it: run that same battery against Qwen3-Coder and DeepSeek-V2-Lite with `VT_ASYNC_DEVICE_MIRROR` live and the decode graph ON — either it reproduces the depth-2 degeneration, or the reason it does not has to be found and written down rather than assumed | bug | | [#1308](https://github.com/mudler/vllm.cpp/issues/1308) | — | ENV-AGNOSTIC: the campaign's derivation query `git grep -cIE 'dgx\.casa\|nas_share\|192\.168\.\|thor:gpu0'` names a HOST, and a hard-coded default names a PATH, so the query cannot see the defect [#1190](https://github.com/mudler/vllm.cpp/issues/1190) was opened to remove. Measured at `5c8671c50` over `scripts/` and `tools/`, which is `ENV-AGNOSTIC-W1-TOOLING`'s ownership: the four campaign patterns match 19 files and 22 hits, and widening to `/home/mudler`, `~/venvs/vllm-oracle`, `~/work/vllm.cpp`, `cutlass-4.5.0` and `cutlass_probe` matches 73 files and 174 hits. Twelve of those lines in six shell scripts are the exact `${KEY:-}` shape the campaign's worked example removed from `scripts/dgx-bringup.sh`, and `scripts/upstream-inventory.py:38-40` carries the Python form `os.environ.get("VLLM_SOURCE", str(Path.home() / "_git/vllm"))`. `ENV-AGNOSTIC-W1-TOOLING` converts `scripts/regen-triton-aot.sh`, because the campaign row wrote that one debt into the file's own text at `:20-23` rather than into a count, and leaves the rest: two of them read as provenance on a first pass and must not be swept blind, since `scripts/cpu-x86-llamacpp-floor.sh:33` states that "every recorded leg used these defaults verbatim" and `scripts/dgx-gdn-packed-bridge-ab.sh:4` dates its prerequisites, which is the same instrument-identity argument that keeps `scripts/mtp-k-gt-1-neartie-gap.py` literal. The blind spot is campaign-level: `ENV-AGNOSTIC-W3-CODE` and `ENV-AGNOSTIC-W4-RECORDS` derive their sets from the same query, so the campaign's claim that its five waves partition 227 files is a claim about the query's 227 and not about the tree. Owed by [`env-agnostic-w1-tooling.md`](specs/env-agnostic-w1-tooling.md) under `## Owed` | bug | +| [#1324](https://github.com/mudler/vllm.cpp/issues/1324) | `ENG-RECORD-ANCHOR-RATCHET` | This row's own figures in `docs/BENCHMARKS.md` and `docs/STATUS.md` are read by NO gate. Measured at `af87251c5`: restoring the wrong pre-[#851](https://github.com/mudler/vllm.cpp/pull/851) values (`rot 39`, `7 broken`, `828 OK`) into the BENCHMARKS cell leaves `check-public-doc-tables.py` exit 0 AND `check-agent-record.py` exit 0, while the same run derived `stale=32, broken=6` beside them; tree restored byte-for-byte by sha256. The ratchet reads only the `code` and `tests` cells of matrix rows, so `docs/` prose is outside `RECORD_ANCHOR_FIELDS` by construction, and `check-public-doc-tables.py` measures shape and never a value. THREE instances, none caught by a gate: rounds four and five of the [#851](https://github.com/mudler/vllm.cpp/pull/851) review, each caught by a human reading, and the landing itself, which published `over 844 OK` where its own merge commit `678fc672c` reads `ok=847` (three good citations landed on `main` between verification and merge). NOT fixed in flow: a repair is a semantic checker change owing a spec and red-before evidence, and it has a design decision first — assert the derived figures against the page, which makes `docs/` a new lock of the shape [#364](https://github.com/mudler/vllm.cpp/issues/364) argues against, or stop publishing derived values and leave the derivation in `--report`, which is what `record-anchor-baseline.json` already chose for itself in its `_comment`. What IS fixed in flow is the drift: the live `OK` count is dropped from all four surfaces that carried it. Closest sibling [#667](https://github.com/mudler/vllm.cpp/issues/667) (same class in `.agents/model-matrix.md`), distinct from [#911](https://github.com/mudler/vllm.cpp/issues/911) (spec bodies) and [#1287](https://github.com/mudler/vllm.cpp/issues/1287) (the ratchet's own false negatives). Also under `## Owed` in [`record-anchor-ratchet.md`](specs/record-anchor-ratchet.md) | bug | +| [#1325](https://github.com/mudler/vllm.cpp/issues/1325) | `ENG-RECORD-ANCHOR-RATCHET` | `scripts/record-anchor-baseline.json` stores a `"total"` that no code reads. `load_record_anchor_baseline` in `scripts/check-agent-record.py` returns `{bucket: int(data["buckets"][bucket]) for bucket in RECORD_ANCHOR_BUCKETS}` with `RECORD_ANCHOR_BUCKETS = ("stale", "broken")`, and it is the file's only reader; `check_record_anchors` iterates those two buckets, and `write_record_anchor_baseline`'s refusal compares `result.total > sum(previous.values())` — the buckets, not the stored `total`. Measured at `af87251c5`: mutating `"total": 38` to `39` (a file whose total disagrees with `32 + 6`) leaves `check-agent-record.py` exit 0, tree restored byte-for-byte by sha256. So this row's own budget file carries the exact shape the row exists to name: a recorded figure no gate reads, sitting beside the figures that are read and presenting as if it were checked. `--write-baseline` compounds it by printing `-> 38`, which reads as the value it stored and is the one no later run consults. TWO candidate resolutions, deliberately not chosen here because choosing belongs to the fixing row: read it and assert `total == stale + broken` on load, or drop the field and derive it at read time, which is the shape AGENTS.md §Records prefers. Either is a semantic change to `check-agent-record.py` owing a spec and a red-before case in `tests/scripts/test_agent_record.py` `RecordAnchorRatchet`. Distinct from [#1287](https://github.com/mudler/vllm.cpp/issues/1287) and [#1270](https://github.com/mudler/vllm.cpp/issues/1270), neither of which reaches the unread field. Also under `## Owed` in [`record-anchor-ratchet.md`](specs/record-anchor-ratchet.md) | bug | diff --git a/.agents/specs/record-anchor-ratchet.md b/.agents/specs/record-anchor-ratchet.md index ecd5a52da..6da3c1aa9 100644 --- a/.agents/specs/record-anchor-ratchet.md +++ b/.agents/specs/record-anchor-ratchet.md @@ -158,7 +158,23 @@ Re-derived at this head after merging 161 commits of `main`, not carried from any earlier number. The population tables are in `## The defect` above and are not repeated here. -**Rot: 38 — 32 `STALE`, 6 `BROKEN`, against 844 `OK`.** +**Rot: 38 — 32 `STALE`, 6 `BROKEN`.** + +**The `OK` total is not written down, here or anywhere else.** `--report` +derives it at read time, and it moves on every merge that adds or removes a +citation, so a document that states it becomes false at the next landing. +`record-anchor-baseline.json` refuses to store it for that reason, in its own +`_comment`. Four prose surfaces stated it anyway: this spec, the +`ENG-RECORD-ANCHOR-RATCHET` row in `engine-matrix.md`, `docs/BENCHMARKS.md` and +`docs/STATUS.md`. All four said `844` where the tree at `678fc672c` reads `847`, +because three good citations landed on `main` between the last measurement and +the merge. AGENTS.md §Records names this shape directly: *never store a +measurement of one file inside another file*. The count is therefore dropped +rather than re-derived. What remains stated is the rot, which is gated against +the baseline in both directions and cannot drift in silence, and the +SHA-anchored population figures at `8daa67b39`, which give the same sense of +scale without a live number. [#1324](https://github.com/mudler/vllm.cpp/issues/1324) +tracks the gate that would have caught the drift. The set is the 40 the fresh review verified by hand, minus one. `KERNEL-ATTN-MLA-SPARSE` cites `include/vllm/v1/attention/backend.h:271` for @@ -289,6 +305,23 @@ exact invocations, each of which genuinely fails when the row regresses: - **Out of scope**: rewriting existing citations. The ratchet lets them be fixed by whoever next touches the row. +## Owed + +Both were found by this row's own review and neither is fixed here, because each +is a semantic checker change and owes its own spec, red-before mutation and +fresh reviewer. + +- [#1324](https://github.com/mudler/vllm.cpp/issues/1324) — this row's figures + in `docs/BENCHMARKS.md` and `docs/STATUS.md` are read by no gate. Restoring + wrong values into the page leaves `check-public-doc-tables.py` and + `check-agent-record.py` both green, measured at `af87251c5`. The drift that + landed with the row is repaired; the gap that let it land is not. +- [#1325](https://github.com/mudler/vllm.cpp/issues/1325) — the `"total"` key in + `scripts/record-anchor-baseline.json` is written and never read. + `load_record_anchor_baseline` returns the two buckets only, and + `write_record_anchor_baseline` refuses a raise against their `sum()`, so + corrupting `total` to disagree with them is inert. + ## Now `ACTIVE` — the parser, the classifier, the baseline and the cases are diff --git a/docs/BENCHMARKS.md b/docs/BENCHMARKS.md index 5b14066b2..6ea4e3a9a 100644 --- a/docs/BENCHMARKS.md +++ b/docs/BENCHMARKS.md @@ -11,7 +11,7 @@ | **Container images (ACTIVE; arm64 cuda verified on GB10 + Orin 2026-08-11)** | `ENG-RELEASE-CONTAINERS` ([spec](../.agents/specs/container-images.md)) | cpu amd64 783 MB; cuda arm64 **1.71 GB**. GB10 `sm_121a`: `/health`+`/version`+SIGTERM on `--gpus all`. Orin `sm_87` (Tegra): Qwen3-0.6B **generates**, GPU **GR3D 95-97%** | n/a | | **Developer/row protocol** | Contribution entry point; `ENG-NOW-DERIVED` #374 @`dbd0d51c` | Entry-point gates retained. #374 W1-W5 DONE; benchmark/runtime/parity `VOID`; row specs now carry `## Now` | n/a | | **NemotronH paged forward** (`MODEL-NEMOTRON-H-ABI-A2P`, [#810](https://github.com/mudler/vllm.cpp/issues/810)) | **No speed number, by the unit's own rule** ([spec](../.agents/specs/nemotron-h-a2p-paged-forward.md) §5) | **A3 host gate PASSES 96/96 `STRICT PASS`.** GB10 read 4/24; cause and fix [#1157](https://github.com/mudler/vllm.cpp/issues/1157), sm_121a re-run pending a lease | CPU gate 12/12. Load 209.0 s, peak RSS 20 142 392 KB | -| **Record-anchor ratchet** (`ENG-RECORD-ANCHOR-RATCHET`, #632) | **No number owed:** a record checker. At `8daa67b39`, **832 of 867** in-scope citations (**96.0%**) were already parsed; no symbol test and no report ran. Rot **38** (32 stale, 6 broken) over **844** OK, **32 in range** | +| **Record-anchor ratchet** (`ENG-RECORD-ANCHOR-RATCHET`, #632) | **No number owed:** a record checker. At `8daa67b39`, **832 of 867** in-scope citations (**96.0%**) were already parsed; no symbol test and no report ran. Rot **38** (32 stale, 6 broken), **32 in range** | | **LoRA runtime W2** (`LORA-RUNTIME`, #278) | **No number owed:** correctness-only; a grid PENDS the W7 model gate | | **ARCH audit: ABI is text-only** | 4 capabilities (H3 video, Laguna, Kimi-Linear, DeepSeek-V4) reachable only from `examples/`, none registry-backed. No gate asks whether a CONSUMER can reach a capability. Documentation only | | **CUDA-graph break seam W1** (`ENG-CUDAGRAPH-BREAK`, [#1192](https://github.com/mudler/vllm.cpp/issues/1192)) | **No number owed, and none taken:** coverage and correctness row, no throughput gate declared | Capability, not a rate: mid-forward capture re-begin holds on a leased GPU; scoped forward matches eager, 500 logits, 0 differing. Probe committed, recipe and sha256 in the [record](../.agents/benchmark-record.md) | diff --git a/docs/STATUS.md b/docs/STATUS.md index cfd9443fd..29c3a3089 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -97,8 +97,8 @@ Both forms were already parsed; a failing anchor was dropped and covered by `any()`, and no symbol test ran. 32 of the 38 offenders are in range. `check-agent-record.py` now classifies each citation OK, STALE or BROKEN and gates the rot against `scripts/record-anchor-baseline.json` in both directions. -Backlog **38** (32 stale, 6 broken) over 844 correct; `ACTIVE` and `READY` rows -join the count. No product behavior changed. +Backlog **38** (32 stale, 6 broken); `ACTIVE` and `READY` rows join the count. +No product behavior changed. Supported-model registry guard (2026-08-06): the public per-architecture list in [FEATURES](FEATURES.md) is CI-bound to the C++ registry by