perf(GFX1100-TG200): raise Qwen3.5-4B Q4_K_M gfx1100 decode to 76.6 tok/s

.agents/claims/CLAIM-GLM53-FLASH-W5B1.md

@@ -0,0 +1,5 @@
+# CLAIM-GLM53-FLASH-W5B1
+
+| Claim | Row IDs | Agent | Worktree / remote dir | Branch | Owned scope | State | Last update |
+|---|---|---|---|---|---|---|---|
+| `CLAIM-GLM53-FLASH-W5B1` | `MODEL-MM-glm5-next-glm5-next-for-conditional-generation` (`ACTIVE`) | Claude Code (opus-5), helper role — a fresh implementer working from the committed spec | local linked worktree `/home/mudler/_git/vllm.cpp-glmw5b1`, base SHA `df024dce466fcfde9b3d2fd40e55d2c25b48e96e`. CPU only: no `rc` lease was taken, no GPU was used, no `ssh` to a fleet device was attempted and no file mutex was needed. No checkpoint download and NO materialising load: `/mnt/nas_share/rc/ckpt/GLM-5.3-Flash-UD-Q2_K_XL/` was not opened at all, because W5c already measured that a materialising load on this box stops at 8.09 GiB RSS in uninterruptible CIFS I/O (O22) and this wave needs no artifact — its substrate is the synthetic `glm5next` GGUF miniature W5c gates its loader against. The `transformers` `v5.16.1` oracle was installed into a throwaway venv under the session scratchpad with `--system-site-packages` so the resident torch was reused rather than re-downloaded, and its `modeling_glm5_next.py` was verified to hash `2092bbb4efa2a8087b74f4a4da37635c503fe1df9ae73f1e6e8342af8b4b8e8b` — the value W3 and W5c both recorded — READ OFF THE INSTALLED MODULE and not off a downloaded copy | `row/MODEL-MM-GLM53-FLASH-W5B1`, issue [#2324](https://github.com/mudler/vllm.cpp/issues/2324) | Owns W5b-1 of [glm5-next-flash.md](../specs/glm5-next-flash.md) `## Work breakdown`: the `Glm5NextTextAttention` block and the `OwnedTensor` -> host f32 bridge. That is the new `src/vllm/model_executor/models/glm5_next_attn.{h,cpp}` and `glm5_next_bridge.{h,cpp}`, `tests/vllm/models/test_glm5_next_attn.cpp` and `test_glm5_next_bridge.cpp`, `tests/vllm/models/fixtures/gen_glm5_next_attn_goldens.py` and its emitted `.inc`, four CMake registrations, this claim, one appended `.agents/issue-index.md` row, and the spec's `### W5b` split, `### W5b-1`, `### W5b-2`, `## Owed` O25 and `## Now`. **EXCLUDES the rest of W5b and says so rather than narrowing silently**: the decoder layer, the mHC stream threading, `Glm5NextTextModel::Forward` and the binding to `MakeGlm5NextKVCache` are W5b-2's, still tracked by [#2241](https://github.com/mudler/vllm.cpp/issues/2241), which this pull request references and does NOT close. The split is not a size decision: W5b-1 answers to `transformers` v5.16.1 and the llama.cpp #27752 container and needs no cache over it, while W5b-2 answers additionally to `MakeGlm5NextKVCache` and the `[T, hc_mult, hidden]` manifold and is where reachability lands. EXCLUDES the KDA arm (W2), the DSA indexer (W3) — `SelectIndexerTopk` is CALLED and not touched — the mHC bricks (W4), the MoE (W5), the weight tower (W5c), the vision tower (W6) and the converter (W7). EXCLUDES `deepseek_v4_dsa.cpp` and `mla_attention.{h,cpp}`, which are the documented wrong reuse and W3's surface respectively, both untouched. EXCLUDES any routing of the experts through `layers::MlpGateUpMethodBase` / `vt::MergedGemmGroup`: O19 / [#2260](https://github.com/mudler/vllm.cpp/issues/2260) records that doing so makes `MoeGateUpSwiGLUGroupedCuda` throw for this 101.14 GiB-resident model, and the bridge has NO overload taking an expert bank. EXCLUDES any parity-pin advance and the `.agents/model-matrix.md` row: the row's lifecycle state does not move | `ACTIVE` | 2026-08-29 — landed both deliverables. **RED captured FIRST on the same tree, in one build**, from the plausible wrong port: with `expand_kv` reading `k_b` UNTRANSPOSED, the all-masked row filled with `-inf` instead of `finfo.min`, `IndexerRoleFor` never reporting `shared`, and the bridge's ceiling not checked, `test_glm5_next_attn` read 9/14 cases and 63/150 assertions failed and `test_glm5_next_bridge` read 3/8 and 3/55. That red also found TWO defects in the tests rather than the product — the refusal golden carried huggingface_hub's wrapper class name, and the bridge's shape case moved a dim `q_b_proj` also depends on, so it threw on the wrong tensor — both repaired before green. Green is 14/14 + 160 assertions and 8/8 + 56 assertions, both exit 0. **Eighteen negative mutations, each sha256-proved applied, built and restored byte-for-byte; SEVENTEEN kill their gate.** The eighteenth is recorded as an EQUIVALENT mutant with its reason in O25, not as a pass: `host_f32_bytes` computed from the dims instead of from the buffers is indistinguishable while `DecodeShaped` refuses a shape disagreement, and the test now pins the sum against the buffers themselves — an earlier version pinned it against the predictor and that mutation passed it. Two more findings came out of the same run and are repaired: the fixture could not distinguish `min(l+1, n-1)` from a wrapping `(l+1) % n`, so a schedule where they disagree was added, and one mutant failed to BUILD under `-Werror` on an unused parameter, which is a passing mutant proving nothing and was rewritten to keep the parameter used and wrong. **NOT REACHED from a production entry point** and O25 carries the disclosure: `grep` over `src/`, `include/` and `examples/` for the five new symbols and the two headers returns nothing outside the four files of this change, so there is no production call site to delete and `.agents/reachability.md`'s mutation is already answered. W5b-2 owns the wiring. **No token, no load and no speed number is claimed**, and none was observed: no oracle for this model runs on any device this project reaches (the reference needs 305.78 GiB FP8 or 598.5 GiB BF16 against ~119.63 GiB), so what is gated is the NUMERICS of one block against a tiny-shape reference and nothing about the MODEL. GPU gate `PENDING`, reason recorded above, no result invented |

.agents/claims/CLAIM-GLM53-FLASH-W5B2.md

@@ -0,0 +1,5 @@
+# CLAIM-GLM53-FLASH-W5B2
+
+| Claim | Row IDs | Agent | Worktree / remote dir | Branch | Owned scope | State | Last update |
+|---|---|---|---|---|---|---|---|
+| `CLAIM-GLM53-FLASH-W5B2` | `MODEL-MM-glm5-next-glm5-next-for-conditional-generation` (`ACTIVE`) | Claude Code (opus-5), helper role — a fresh implementer working from the committed spec | local linked worktree `/home/mudler/_git/vllm.cpp-glmw5b2`, base SHA `10b5cabb01eac7e24c5aa781538577b5f47c2217`. CPU only: no `rc` lease was taken, no GPU was used, no `ssh` to a fleet device was attempted and no file mutex was needed, because nothing in this wave has a device arm to measure. NO checkpoint download and NO materialising load: `/mnt/nas_share/rc/ckpt/GLM-5.3-Flash-UD-Q2_K_XL/` was opened only through the committed 1412-tensor header manifest and the committed `config.json` fixture, never as bytes — W5c measured that a materialising load on this box stops at 8.09 GiB RSS in uninterruptible CIFS I/O (O22). The `transformers` `v5.16.1` oracle ran from a `git worktree` at that tag plus a throwaway `--system-site-packages` venv holding only `tokenizers>=0.23.1` and `safetensors>=0.8.0`, so the resident torch 2.11.0 was reused rather than re-downloaded; its `modeling_glm5_next.py` was verified to hash `2092bbb4efa2a8087b74f4a4da37635c503fe1df9ae73f1e6e8342af8b4b8e8b` — the value W3, W5b-1 and W5c all recorded — READ OFF THE MODULE THE GENERATOR IMPORTED, and re-asserted a second time by `git cat-file -p v5.16.1:...` in the source clone | `row/MODEL-MM-GLM53-FLASH-W5B2`, issue [#2241](https://github.com/mudler/vllm.cpp/issues/2241) | Owns W5b-2**a** of [glm5-next-flash.md](../specs/glm5-next-flash.md) `## Work breakdown`: the decoder layer, the mHC stream threading, `Glm5NextTextModel::Forward` and the binding of the DSA block to `MakeGlm5NextKVCache`'s groups 0 and 2. That is the new `src/vllm/model_executor/models/glm5_next_layer.{h,cpp}`, an additive `DsaCache*` parameter on `glm5_next::Attention` and a `SelectIndexerTopkFromPacked` lifted out of `SelectIndexerTopk`'s body (both null-default and byte-identical on the uncached path W5b-1 gated, proved by rerunning W5b-1's two suites unchanged), `tests/vllm/models/test_glm5_next_layer.cpp`, `tests/vllm/models/fixtures/gen_glm5_next_layer_goldens.py` and its emitted `.inc`, two CMake registrations, this claim, one appended `.agents/issue-index.md` row, and the spec's `### W5b-2` split into `### W5b-2a` / `### W5b-2b` plus `## Owed` O26 and `## Now`. **EXCLUDES making `ForwardGlm5NextForConditionalGeneration` stop refusing by name, and says so rather than narrowing silently**: that is W5b-2b's, still tracked by [#2241](https://github.com/mudler/vllm.cpp/issues/2241), which this pull request references and does NOT close. The split is not a size decision — it is that reachability needs a weight bridge for the FOUR arms `BridgeDsaLayer` does not cover, and the MoE half of that is a residency design problem (~1,150 GiB of f32 routed experts across the 42 sparse layers against a ~119.63 GiB box, with `kBridgeTensorF32ByteCeiling` correctly refusing a 9.0 GiB bank today), not four more `BridgeDsaLayer`s. EXCLUDES the KDA arm's numerics (W2), the DSA indexer's selection (W3) — `SelectIndexerTopk`'s BODY is lifted into a second entry point and not otherwise touched, and its 1934 assertions are unchanged — the mHC bricks (W4), the MoE block (W5) — `MoeForward` is CALLED and not modified — the weight tower (W5c), the vision tower (W6) and the converter (W7). EXCLUDES any routing of the experts through `layers::MlpGateUpMethodBase` / `vt::MergedGemmGroup`: O19 / [#2260](https://github.com/mudler/vllm.cpp/issues/2260) records that doing so makes `MoeGateUpSwiGLUGroupedCuda` throw for this 101.14 GiB-resident model, and `glm5_next_layer.cpp` reaches only `vt::MoeRouterTopK`, `vt::MoeCombine` and `vt::KdaGatedDeltaRule` through the blocks W2 and W5 already landed. EXCLUDES any parity-pin advance and the `.agents/model-matrix.md` row: the row's lifecycle state does not move | `ACTIVE` | 2026-08-30 — landed all four deliverables. **RED captured FIRST**, on the first run of the new suite: 4 of 10 cases and 7 of 1647 assertions failed, layer 0 (KDA + dense) green and every DSA layer red by 2.7 to 8.1. Bisected against oracle intermediates rather than guessed — the mHC pre plus `input_layernorm` agreed to 4.8e-07 and the attention output did not — and the cause was in the ORACLE CONFIGURATION, not the port: `Glm5NextPreTrainedModel` sets `_supports_sdpa = True`, so a default `Glm5NextTextConfig` resolves `_attn_implementation` to `sdpa`, whose `build_attention_mask_from_topk` returns a BOOLEAN mask (`:1249-1250`) instead of the additive `finfo.min` one the eager arm builds (`:1252-1256`). The two backends DISAGREE on a left-padded query row where every key is masked — torch's SDPA emits 0.0 and eager's uniform softmax emits the mean of the values, measured 0.0 against our 0.509 — so the generator now pins `cfg._attn_implementation = "eager"`, which is the arm W5b-1 gated and the one `:1227-1228` names as the only interface a 3-D per-(query, key) mask can reach. Recorded in O26 as a fact about `glm5_next`, not about this fixture. Green is 10/10 cases and 1656/1656 assertions, exit 0, with the eight sibling glm5 suites unchanged and green (attn 160, dsa 1934, bridge 96, moe 1614, mhc 98, scaffold 2660, gguf_load 8731, kda 342). **Fourteen negative mutations, each sha256-proved applied, built and restored byte-for-byte; ALL FOURTEEN kill their gate — but only after a fifteenth finding, which is the one worth reading.** The mutation that truncates the attention's key range to the current window under a filled cache SURVIVED at 1647 of 1647 on the first pass. Its output is all-NaN; `NaN - want` is NaN and `NaN > x` is FALSE for every x, so the running maximum in the test's own `MaxGap` helper never left its initial zero and an ALL-NaN FORWARD READ AS A PERFECT MATCH on every gap assertion in the file. `MinStreamSeparation` and the cached-tail loop were blind the same way (`std::max(m, NaN)` returns `m`). All three now treat a non-finite value as an INFINITE gap and report the count separately, so a failure distinguishes "wrong number" from "not a number"; the mutation then reds 3 assertions and the suite grew from 1647 to 1656. One earlier mutant also failed to BUILD under `-Werror` on an unused variable, which is a passing mutant proving nothing, and was rewritten to keep the variable used and the arithmetic wrong. **NOT REACHED from a production entry point, and this wave's own scope said it would be** — O26 carries the disclosure in the strong form: `ForwardGlm5NextForConditionalGeneration` still refuses by name, so the only call site of `glm5_next_layer.{h,cpp}` is its own gate's, and O15, O16, O17, O23 and O25 are NOT discharged, because `.agents/reachability.md` is explicit that "an intermediate hop that is itself unreached does not carry". What changed is that the five primitives had five separate dead ends and now have ONE assembly point, gated against the reference. **No token, no load and no speed number is claimed**, and none was observed: no oracle for this model runs on any device this project reaches, so what is gated is the CONTROL FLOW and the NUMERICS of a five-layer stack at tiny shapes and nothing about the 321.32B model. GPU gate `PENDING`, reason recorded above, no result invented |

.agents/claims/CLAIM-GLM53-FLASH-W5B2B.md

@@ -0,0 +1,5 @@
+# CLAIM-GLM53-FLASH-W5B2B
+
+| Claim | Row IDs | Agent | Worktree / remote dir | Branch | Owned scope | State | Last update |
+|---|---|---|---|---|---|---|---|
+| `CLAIM-GLM53-FLASH-W5B2B` | `MODEL-MM-glm5-next-glm5-next-for-conditional-generation` (`ACTIVE`) | Claude Code (opus-5), helper role — a fresh implementer working from the committed spec | local linked worktree `/home/mudler/_git/vllm.cpp-glmw5b2b`, base SHA `c0fa299b1a03225e88854a605a07a3d726ae2062`. CPU only for every gate: the whole suite is host f32 and the forward REFUSES a non-CPU queue by name, so no `rc` lease was needed to gate it and none was taken for that purpose. A `dgx:gpu0` lease WAS taken, for one thing only — the decisive check, an attempted real generation at the staged artifact — through `rc run -d dgx:gpu0`, never `ssh`. NO checkpoint download and NO materialising load on this box: `/mnt/nas_share/rc/ckpt/GLM-5.3-Flash-UD-Q2_K_XL/` was read only for its shard SIZES, because W5c measured that a materialising load here stops at 8.09 GiB RSS in uninterruptible CIFS I/O. No `transformers` run and no golden regeneration: this wave adds no numerics, so its oracle is the RESIDENT tower assembled from the same loader tensors, and the goldens W5b-2a captured at `v5.16.1` are consumed unchanged by `test_glm5_next_layer` (10 cases, 1656 assertions, unmoved) | `row/MODEL-MM-GLM53-FLASH-W5B2B`, issue [#2337](https://github.com/mudler/vllm.cpp/issues/2337), split out of [#2241](https://github.com/mudler/vllm.cpp/issues/2241) whose append-only index row is spent on W5b-2a | Owns W5b-2**b** of [glm5-next-flash.md](../specs/glm5-next-flash.md) `## Work breakdown`: the weight bridge for the four arms `BridgeDsaLayer` does not cover, and the engine binding that makes `ForwardGlm5NextForConditionalGeneration` stop refusing by name. That is `DecodeOwnedTensorRowsToF32` / `HostF32RowBytes` / `BridgeKdaLayer` / `BridgeMlp` / `BridgeMhcSite` / `BridgeMoeLayer` / `GgufExpertSource` in `glm5_next_bridge.{h,cpp}`, an `ExpertSource` seam and a grouped per-expert visit in `glm5_next_moe.{h,cpp}`, a `LayerWeightSource` overload of `TextModelForward` in `glm5_next_layer.{h,cpp}`, the new `glm5_next_forward.{h,cpp}`, the registry hook, the new `tests/vllm/models/test_glm5_next_forward.cpp`, four appended cases in `test_glm5_next_bridge.cpp`, the MOVED refusal pin in `test_glm5_next_scaffold.cpp`, two CMake registrations, one `scripts/runner-routing-allowlist.txt` entry, this claim, one appended `.agents/issue-index.md` row, and the spec's §W5b-2b outcome plus `## Owed` O27 and `## Now`. **EXCLUDES the numerics of every primitive it now reaches** — W2's KDA, W3's DSA indexer, W4's mHC, W5's MoE and W5b-1's attention are CALLED and not modified, and their seven suites are rerun unchanged. EXCLUDES the vision tower (W6), the converter (W7), the safetensors arm, the MTP head, any parity-pin advance and the `.agents/model-matrix.md` row: the row's lifecycle state does not move, because O1 does not. EXCLUDES routing the experts through `layers::MlpGateUpMethodBase` / `vt::MergedGemmGroup`: O19 / [#2260](https://github.com/mudler/vllm.cpp/issues/2260) records that doing so makes `MoeGateUpSwiGLUGroupedCuda` throw for this model, and the per-expert source keeps that unreachable BY CONSTRUCTION because it hands the block host floats. EXCLUDES ragged batching and the device arm, both REFUSED BY NAME rather than approximated, and both recorded under O27 | `ACTIVE` | 2026-08-30 — **`ModelRegistry::Forward` reaches this model, and the reachability mutation is the deliverable.** Deleting the `Glm5NextHostForward` call in the registry hook and returning an empty `ForwardLogits{}` REDS `test_glm5_next_forward` at 11 of 118 assertions; O26 said there was no production call site to delete, and there is one now. **RED CAPTURED FIRST from an EXISTING gate**: `test_glm5_next_scaffold`'s case pinning "the forward REFUSES BY NAME" went red at 8 assertions in 1 case the moment the hook ran, and the pin MOVED with the change the way W3's `MlaBlockDims` pin moved rather than being deleted by it. **The residency decision, with its arithmetic**: one sparse layer's three expert banks are 27.0 GiB in f32 and the 42 sparse layers together are 1,134 GiB against ~119.63 GiB usable — 9.5x over — so the banks are NEVER bridged; `num_experts_per_tok` is 8 of 288, one expert is 100,663,296 bytes (0.09375 GiB) and one bank ROW is 33,554,432 (32x UNDER the unchanged 1 GiB ceiling, where the bank is 9x over), and `MoeForward` visits each HIT expert ONCE. The forward's f32 peak is one layer plus one expert plus one 64 MiB `lm_head` chunk, under 0.75 GiB, against a 426.72 GiB resident tower. **Thirteen negative mutations on tree `91354df62`, each proved applied by a diff hash, each BUILD rc=0, each restored byte-for-byte; ALL THIRTEEN now kill their gate — but TWO SURVIVED the first suite and both repairs are in this branch.** M12, swapping the two mHC sites inside the layer source, left every logit BIT-IDENTICAL at 86 of 86: the fixture's mHC `fn` payloads are ramps in the hundreds and thousands, so the sigmoid gates saturate and the Sinkhorn projection converges to the same matrix from either site, and a gate that could only see the swap through the logits is a mute switch at that geometry — the mapping is now asserted STRUCTURALLY with the two tensors asserted to DIFFER, and M12 reds 24. M13, removing the per-expert grouping, left 32030 of 32030 green because the case ran ONE token, where every selected expert is hit once whatever the code does; a six-token case now fills 12 slots from 2 distinct experts and M13 reds 3. M5 kills by SIGSEGV (rc=139) rather than by an assertion, which corrected the refusal's own message: without it the loop dereferences a null source, it does not read zeros. **Green:** forward 9 cases / 118 assertions, bridge 19 / 32228, scaffold 38 / 2652, with the six sibling suites unchanged and green (layer 10/1656, moe 8/1614, gguf_load 16/8731, attn 14/160, kda 28/342, dsa 10/1934, mhc 5/98). **No token, no load and no speed number is claimed for the 321.32B model**, and none was gated: what runs in CI is a synthetic 4-layer `glm5next` miniature at `hidden_size` 32, and O1 stands unchanged. The `dgx:gpu0` attempt at the staged 101.25 GiB artifact is reported as exactly what it printed, in the pull request body |