feat(KERNEL-QUANT-CIQ-GEMM-ROCM): keep-quant GEMM providers on kROCM

.agents/issue-index.md

@@ -527,3 +527,6 @@ rather than merged. `scripts/check-agent-record.py` gates both.
 | [#1563](https://github.com/mudler/vllm.cpp/issues/1563) | `GATE-SQUASH-SEPARATOR` | **A markdown `---` horizontal rule anywhere in a pull request body silently voids the trailer block, and `check-commit-trailers.py` blames the trailers instead of the framing.** Found 2026-08-21 writing the body for PR [#1550](https://github.com/mudler/vllm.cpp/pull/1550) ([#1542](https://github.com/mudler/vllm.cpp/issues/1542)). `parsed_trailers()` shells out to git's trailer parser, and **git treats a line of exactly `---` as the start of the patch section**, so everything after the first one is not part of the message and a trailer block below it is invisible. Reproduced with no repository state: a body of `subject / prose / --- / more prose / FOLLOWING_AGENTS_PROTOCOL / the three trailers` reports `[trailers] Following-Agents-Protocol must appear exactly once` and `[attribution] AI-Assisted must appear exactly once`; `sed -i '/^---$/d'` on that same file reports `OK: commit trailer contract`, and the `---` is the only difference. **The MESSAGE is the defect, not only the behaviour**: `Following-Agents-Protocol` appears EXACTLY ONCE in the body while the checker says it must appear exactly once, so a reader counts occurrences, finds one, counts again and dumps bytes before thinking to test the parser's own framing. `_strict_errors` already computes `_paragraphs(body)[-1]` correctly as the three trailers verbatim, so the checker holds the information needed to say "the trailer paragraph is present but git could not parse it; a `---` line at line N ends the message". Worse, the neighbouring `FOLLOWING_AGENTS_PROTOCOL must appear exactly once as a separate paragraph before the trailer paragraph` check stays SILENT, so the two errors that fire both point away from the cause. **Beyond one confusing message**: the repository sets `squash_merge_commit_message = PR_BODY`, so the body IS the landed commit message, and a body carrying a `---` lands a commit whose trailers `git interpret-trailers` cannot see, on a branch that is never force-pushed. Same permanent-damage shape AGENTS.md records for the `---------` separator GitHub wrote under `COMMIT_MESSAGES`, arriving from the AUTHOR side rather than the forge side. `scripts/agent-pr-body.py --pr <N>` DOES catch it and caught it here before the merge; the exposure is a body never passed through that command, which AGENTS.md notes is not a gate and cannot be one because it reaches the network, while the CI guard reads the frozen `pull_request` payload and so does not re-read a body edited after the final push. NOT FIXED HERE: it changes a checker's semantics and its message, so under `## Changing the rules or a checker` it needs its own row, a red-before test and green-after evidence. Two candidate repairs, neither chosen: name the `---` line, or strip patch-section framing before parsing so a markdown rule is inert -- the second changes what the contract accepts and is the larger decision. Suggested minimum: `tests/scripts/test_check_commit_trailers.py` gains a case pinning the reproduction above | bug |
 | [#1454](https://github.com/mudler/vllm.cpp/issues/1454) | `SPEC-MTP-GGUF` | **`test_qwen3_5_gguf_mtp.cpp` reported `Status: SUCCESS!` with `assertions: 0` on every CI run, and its one arithmetic guarantee was a tautology.** Both cases opened `if (path == nullptr) return;` on `VLLM_MTP_GGUF_MODEL`, and a bare `return` from a doctest case is a PASS: re-derived on a clean Release build at `947e5f648`, unset, the file printed `test cases: 2 \| 2 passed \| 0 failed \| 0 skipped`, `assertions: 0`, `Status: SUCCESS!`, exit 0, and printed nothing else. The variable is set nowhere in `.github/workflows/`, so that was the state of every run. Second defect in the same file: the comment at `:52` stated `num_hidden_layers + depth == block_count` and the line under it asserted `CHECK(c.num_hidden_layers > 0)`, true of every valid model. MEASURED, not argued: mutating `src/vllm/model_executor/models/qwen3_5_gguf_weights.cpp:889` to `c.num_hidden_layers = block_count;` compiled clean and left the file at 2/2 cases, 0 assertions, `SUCCESS!`, exit 0. FIXED IN FLOW. The invariant is now pinned **HERMETICALLY** on KV-only synthetic GGUFs carrying no weight bytes, so CI checks it every run rather than never - 65/1 (the shipped Qwen3.8-27B pair), 25/1 (the Qwen3.5-2B reference this suite was developed against) and 28/3, the third arm separating `- nextn` from `- 1` - plus a head-less arm asserting the key is NOT published, which is the half `NumMtpLayers` cannot express because it answers 1 for an absent key. The two env-gated cases stay, now skipping with a `MESSAGE` naming the variable as `test_gguf_mmproj_reach.cpp` does, and the live one re-derives the invariant from the file's own `block_count` kv. Unset 4 cases / 18 assertions / `SUCCESS!` / rc 0; live on `Qwen3.8-27B-Q4_K_M.gguf` 4 / 38 / `SUCCESS!` / rc 0. Both mutants now red (9/18 and 5/18, exit 1), compiled clean, restored against a pre-taken sha256. **The production line is CORRECT and was not touched**: `block_count - nextn` landed `1a4db5c3c`, the `mtp_num_hidden_layers` republication `493327b4e`. Related but distinct: [#821](https://github.com/mudler/vllm.cpp/issues/821) W2 (`0adeb8b0e`) pins the same arithmetic for the 27B artifact on a committed manifest in `tests/vllm/models/test_qwen38_27b_gguf_manifest.cpp`, and that gate DOES catch both mutants - so the invariant was not globally unpinned, it was unpinned in this row's own file | bug |
 | [#1434](https://github.com/mudler/vllm.cpp/issues/1434) | `GATE-DOC-CHECKPOINT-STATES` | **`scripts/check-doc-checkpoint.py` could not see `PARTIAL`, so 118 state cells could move with no gate observing them.** `STATES` (`:56-66`) is the whole definition of what a lifecycle state IS for the gate that enforces AGENTS.md's `docs/STATUS.md` / `docs/BENCHMARKS.md` / spec `## Now` triple, and `row_states` drops any row it cannot match. `lifecycle_moves` and `moved_rows` then iterate the AFTER map, so leaving the matched set is silent by construction. Re-derived at `947e5f648` (the report measured `63d87805c`): `PARTIAL` **118** cells and `ANCHOR-BACKFILL` **73**, against `DONE` 77 and `BLOCKED` 9 — `PARTIAL` is the second most used state in the matrices and the gate was blind to it. Over the seven tables `ROW_TABLES` actually reads, the resolved population goes from **153 rows to 226**, a 47.7 % widening. Two of the transitions the report names behave differently from its description, measured with scratch commits at `947e5f648` on an unmodified checker: `READY -> PARTIAL` rc **0** and `PARTIAL -> READY` rc **0** are the real blind spots, while the report's suggested `PARTIAL -> ACTIVE` already reds — by accident, reporting **`added as ACTIVE`** for a row that has existed for months, because it is absent from the BEFORE map. FIXED IN FLOW for `PARTIAL` only. **`ANCHOR-BACKFILL` is deliberately excluded**: `.agents/feature-matrix.md:14-17` defines it as a property of the RECORD (*a legacy implemented row without exact code, test and real-spec anchors*), `docs/STATUS.md` carries no such term and would have nothing true to write on a `DONE <-> ANCHOR-BACKFILL` move, and `REQUIRED["lifecycle"]` cannot demand the spec's `## Now` alone — so admitting it would demand a public-document edit with nothing to say, which is the exact shape `check-doc-checkpoint.py:4-17` records as the reason the file was rewritten (16 of 20 red CI runs, six hardcoded escape hatches). One row's resolved state moves and the move is a REPAIR: `KV-BLOCK-POOL` says `` `PARTIAL` (not `DONE`) `` in its prose and the last-match heuristic believed the parenthesis, resolving `DONE`. No pinned counter moves — `check-gate-commands.py` has its own `GATED_STATES` and `RUNNABLE_BASELINE` is keyed on matrix rows, `UNOWNED_HIGH_WATER` is unmoved because this row names an owner, and no matrix row or public document changes — which was measured, not assumed, because this is the [#1376](https://github.com/mudler/vllm.cpp/issues/1376) ratchet shape. Remainder listed under `## Owed` in [doc-checkpoint-lifecycle-states.md](specs/doc-checkpoint-lifecycle-states.md): `ANCHOR-BACKFILL` moves, `.agents/sglang-matrix.md` never entering `ROW_TABLES`, a row that leaves the matched set entirely, and a new row added directly as `PARTIAL` | bug |
+| [#1586](https://github.com/mudler/vllm.cpp/issues/1586) | `BACKEND-ROCM` | Adopt the ROCm 7.14 container toolchain (first TheRock production release, gfx1100 supported) and open the gfx1100 optimization campaign; baseline recorded in the issue: build 586/586 green on `rocm-dev:7.14.0`, focused gate 4/5 with the `MoeSiluMul` bf16 exactness failure at `tests/vt/test_backend_cross_device.cpp:2063` | feature |
+| [#1587](https://github.com/mudler/vllm.cpp/issues/1587) | `KERNEL-QUANT-CIQ-GEMM-ROCM` | The ROCm backend has no quantized-weight GEMM provider, so every GGUF k-quant on an AMD card computes off device; upstream pins a `csrc/rocm` W4A16 family (`gptq_gemm_rdna3` gated `VLLM_ROCM_GFX1100`) we cannot reach yet because no GPTQ consumer exists. W1 registers the `kMatmulBTQuant`/`kMatmulBTQuantGrouped` keep-quant providers mirroring the CUDA sibling; W2 (owed) ports the upstream family behind a loader consumer. Spec [`kernel-quant-ciq-gemm-rocm.md`](specs/kernel-quant-ciq-gemm-rocm.md) | feature |
+| [#1588](https://github.com/mudler/vllm.cpp/issues/1588) | `BACKEND-ROCM` | Characterize Qwen3.5-0.8B CPU against ROCm numerics on gfx1100: the backend matrix records the all-native run with its numerical characterization open; owns the `MoeSiluMul` bf16 exactness failure found in the 7.14 baseline | verification |

.agents/specs/kernel-quant-ciq-gemm-rocm.md

@@ -0,0 +1,160 @@
+# KERNEL-QUANT-CIQ-GEMM-ROCM — keep-quant GEMM providers on kROCM
+
+- Issue: [#1587](https://github.com/mudler/vllm.cpp/issues/1587)
+- Base: `e2a9e035d` (upstream/main)
+- State at commit: `SPIKE` accepted; W1 implementation rides this pull request
+- Pull request shape: one pull request for spec and implementation
+  (developer decision 2026-08-21)
+
+## Scope
+
+The ROCm backend registers roughly 44 ops and has no quantized-weight GEMM
+provider. A search for `MatmulBTQuant`, `kMatmulBTQuant`, and `vec_dot` over
+`src/vt/rocm/` and `include/vt/` returns nothing. Every GGUF k-quant weight
+on an AMD card therefore computes off device today.
+
+Two waves, one row:
+
+1. **W1 (this change).** `kROCM` providers for `OpId::kMatmulBTQuant` and
+   `OpId::kMatmulBTQuantGrouped`: the GGUF Q8_K-family keep-quant GEMM,
+   mirroring the CUDA sibling's contract. Registering the provider flips
+   `GgufQuantComputeAvailable()` true on the platform, so every GGUF
+   k-quant model reaches it with zero model-code edits.
+2. **W2 (owed, see `## Owed`).** The upstream `csrc/rocm` W4A16 GPTQ/AWQ
+   family (`wvSplitK_int4_g`, `gptq_gemm_rdna3`,
+   `gptq_gemm_rdna3_wmma`, `moe_gptq_gemm_rdna3`). This project cannot
+   reach those kernels yet: GPTQ and AWQ checkpoints have only a host
+   dequant path (`awq_gptq_dequant.cpp`) and no W4A16 consumer. Porting
+   them before a consumer exists would land dead code.
+
+Out of scope: FP8 on gfx1100 (upstream refuses it on this arch;
+`supports_fp8()` is gfx9 or gfx12x only), Triton-on-ROCm families, and any
+loader work for AWQ/GPTQ checkpoints.
+
+## Upstream anchors
+
+Pinned vLLM `555967922`:
+
+- `csrc/rocm/torch_bindings.cpp` names the whole HIP quant-GEMM surface:
+  `LLMM1`, `wvSplitK`, `wvSplitKrc`, `wvSplitK_int4_g`, `wvSplitKQ`, and
+  the `VLLM_ROCM_GFX1100`-gated `gptq_gemm_rdna3`,
+  `gptq_gemm_rdna3_wmma`, `moe_gptq_gemm_rdna3`.
+- `csrc/rocm/q_gemm_rdna3.cu:1-40` (header) records the RDNA3 hardware
+  facts W2 inherits: wave32 geometry, no native packed fp16/bf16 atomic
+  add (emulated with `global_atomic_cmpswap_b64`), `v_dot2_f32_f16` for
+  fp16, fp32-widened accumulate for bf16, and the WMMA forward at
+  `M >= 16`.
+- `vllm/platforms/rocm.py` `supports_fp8` excludes gfx1100.
+
+Classification per `.agents/porting.md`: W1 has **no upstream
+counterpart** — vLLM has no GGUF keep-quant device path anywhere. It is
+derived from our own CUDA sibling plus the ggml CPU reference semantics,
+and it is recorded as such in `porting-inventory.md` section 9. W2 is a
+1:1 port of the pinned files.
+
+## Local anchors
+
+- `include/vt/ops.h:176` `kMatmulBTQuant`; `:184` `kMatmulBTQuantGrouped`;
+  `:1629` the `MatmulBTQuant` entry signature.
+- `src/vt/ops.cpp:186-211` validation and dispatch through
+  `GetOp(OpId::kMatmulBTQuant, q.device.type)`.
+- `src/vt/cpu/cpu_quant_gemm.cpp:302-310` the CPU registrar — the exact
+  oracle.
+- `src/vt/cuda/cuda_quant_dot.cu:1-18` the oracle chain (kernel wiring,
+  per-block dot, activation quant); `:1814` the provider; `:1990-1993`
+  the registrar whose registration flips the loader default.
+- `src/vllm/model_executor/model_loader/gguf_keep_quant.cpp:75-78`
+  `GgufQuantComputeAvailable()` — the reachability flip.
+- `src/vt/rocm/rocm_ops.hip:101` the kROCM registration pattern;
+  `src/vt/rocm/rocm_backend.hip:328` the unified-memory bool that decides
+  CPU-reference fallthrough on APUs versus discrete cards.
+
+## Design
+
+W1 adds `src/vt/rocm/rocm_quant_dot.hip`, structured like
+`cuda_quant_dot.cu`:
+
+- Quantize each activation row to `Q8_K` on the device, then run an
+  integer dot against the compressed weight blocks per output element.
+  Integer arithmetic is exact, so the provider gates **bit-exact**
+  against the CPU provider — the same bar the CUDA sibling meets.
+- First-wave weight types: `Q4_K`, `Q5_K`, `Q6_K`, `Q2_K`, `Q3_K`. The
+  IQ codebook types (`IQ2_XXS`, `IQ3_XXS`) join when their tables port
+  cleanly; `Q8_0`/`Q4_0` activations fall back to the CPU provider over a
+  drained queue, exactly as `cuda_quant_dot.cu:1834` does.
+- Geometry sized for wave32 on gfx1100; the CUDA warp-per-output shape
+  carries over with wave-size adjustments. Geometry is a performance
+  concern only; correctness comes from the exact integer core.
+- Memory: gfx1100 is a discrete card, so weight blocks must be
+  device-resident. The unified-memory assumptions in the model paths do
+  not hold here. The provider requires device pointers and relies on the
+  existing weight-staging path; the implementer verifies
+  `needs_weight_staging()` reports true for `kROCM` so loaders stage
+  blocks once. If staging needs model-path edits beyond the platform
+  seam, that is a stop condition below.
+- Registration follows `rocm_ops.hip:101`. From registration onward,
+  `GgufQuantComputeAvailable()` is true on `kROCM` and the GGUF loader
+  routes keep-quant towers to the device. A rollback env kill switch
+  mirrors whichever flag the CUDA side exposes.
+
+## Risks
+
+- R1: weight residency on a discrete card. The DeepSeek-V4 and Qwen3.5
+  paths stage weights through `ResidentWeight` gated on
+  `needs_weight_staging()`; if that predicate is CUDA-only, W1 grows the
+  platform-seam fix and says so.
+- R2: wave32 geometry differences make the first build slower than the
+  CPU tier on some shapes. That is a recorded measurement, not a
+  correctness failure; the provider stays default-on only if it wins or
+  ties, else ships behind the kill switch with the numbers in the spec.
+- R3: hipcc `-O0` device code starts a hostcall listener that can deadlock
+  at exit (#132). Builds set a `CMAKE_BUILD_TYPE`; the container baseline
+  uses Release.
+- R4: IQ codebook tables grow `.rodata`; deferring them keeps W1 small.
+
+## Tests
+
+Red-first, in the same change:
+
+1. Extend the quant-dot operator tests with `kROCM` arms: per-type
+   bit-exact equality against the CPU provider on random and boundary
+   inputs, the `K % 256` refusal, the grouped variant's expert-index
+   contract, and the unsupported-dtype CPU-fallback arm. Capture the red
+   before the provider exists (`OpRegistered(kMatmulBTQuant, kROCM)` is
+   false and keep-quant stays off).
+2. Focused gate: `ctest -R 'rocm|cross_device|quant'` inside the
+   `rocm-dev:7.14.0` container under the host GPU mutex.
+3. Model-level smoke: one small GGUF checkpoint (Qwen3.5-0.8B Q4_K_M,
+   fetched under the recorded authority) decodes end to end on gfx1100
+   with keep-quant routed to the device, token-identical to the same
+   build forced onto the CPU provider.
+
+## Gates
+
+Correctness gate: bit-exact versus the CPU `kMatmulBTQuant` provider on
+the declared types, plus the model smoke above. The pinned-vLLM ROCm
+oracle does not cover GGUF keep-quant (`BACKEND-GATE-ROCM-VLLM` stays
+`INVENTORIED`), so vLLM parity for this wave is out of reach by
+construction and said so. Performance axes are measured and recorded; no
+throughput floor is claimed in W1.
+
+## Evidence
+
+- Container baseline on 7.14: build 586 of 586 targets green; focused
+  gate 4 of 5 with the `MoeSiluMul` bf16 exactness failure recorded on
+  [#1586](https://github.com/mudler/vllm.cpp/issues/1586).
+- This row appends its measurements to `## Outcome` when it reaches DONE.
+
+## Stop conditions
+
+- `NEEDS_DECISION`: weight staging requires edits to model forward paths
+  rather than the platform seam.
+- Stop and report if bit-exactness cannot be reached; the integer-dot
+  premise would be violated, which means the port is wrong somewhere.
+
+## Owed
+
+- W2: the upstream `csrc/rocm` W4A16 family port together with the loader
+  consumer that makes it reachable. Stays owed unless it lands in this
+  pull request.
+- `porting-inventory.md` section 9 entry for the W1 derivation.

CMakeLists.txt

@@ -1588,7 +1588,8 @@ if(VLLM_CPP_HIP)
     src/vt/rocm/rocm_gdn_postconv.hip
     src/vt/rocm/rocm_gdn_scan.hip
     src/vt/rocm/rocm_gdn_fused.hip
-    src/vt/rocm/rocm_ops.hip)
+    src/vt/rocm/rocm_ops.hip
+    src/vt/rocm/rocm_quant_dot.hip)
   if(VLLM_CPP_HIP_ARCHITECTURES)
     set_source_files_properties(
       src/vt/rocm/rocm_backend.hip
@@ -1609,6 +1610,7 @@ if(VLLM_CPP_HIP)
       src/vt/rocm/rocm_gdn_scan.hip
       src/vt/rocm/rocm_gdn_fused.hip
       src/vt/rocm/rocm_ops.hip
+      src/vt/rocm/rocm_quant_dot.hip
       PROPERTIES HIP_ARCHITECTURES "${VLLM_CPP_HIP_ARCHITECTURES}")
   endif()
   # Prefer the absolute path inside ${ROCM_PATH}/lib, fall back to the bare name,