Merge origin/main into row/ENG-RESIDENCY-CONFIG-W2

The base moved while W2 was under review and repair, and #1200 landed edits to
`.agents/specs/expert-streaming.md` two rows above this row's own entry.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

# Conflicts:
#	.agents/specs/expert-streaming.md

Author: mudler

.agents/issue-index.md

@@ -78,6 +78,7 @@ These change how the engine runs and have no CLI flag (or complement one).
 | `VT_VULKAN_DEVICE` | first suitable device | Forces the Vulkan physical device index. Required on a multi-GPU host to pin the intended device |
 | `VT_KV_CACHE_F32` | off (native KV dtype) | Forces the KV cache to fp32. A precision/diagnostic lever, at the cost of double the KV memory |
 | `VLLM_CPP_VOCODER_DEVICE` | `cpu` | Which device the shared 1-D BigVGAN vocoder core (`vllm::vocoder1d` — MiniMax-Music3, MiniMax-H3's audio VAE, LTX-2.5's audio VAE, IndexTTS-2.5) runs its convolutions on. It takes any device name `vt` knows (`vt::DeviceTypeName` — `cpu`, `cuda`, `metal`, `vulkan`, `xpu`, `rocm`, `tenstorrent`) and resolves it through `vt::DeviceTypeFromName`, so a provider registered for a new backend becomes reachable here with no edit. A name `vt` does not know, or one whose device has no registered `vt::Conv1d` / `vt::ConvTranspose1d` provider in this build, is REFUSED by name — never silently downgraded to the host, because a silent fallback means an operator who asked for a device never learns they did not get one. The transposed convolution is 88.5 % of MiniMax-Music3's acoustic-half profile and on scalar host loops a 45 s clip is a multi-hour decode, so this is the knob that decides whether that stage runs on the GPU. The two providers are BYTE-IDENTICAL — one f64 accumulator per output element walked in the same order, the host pinned `-ffp-contract=off` and the device kernel pinned with `__dmul_rn`/`__dadd_rn` — and `tests/vt/test_ops_conv1d_general.cpp` gates that with `memcmp`, not a tolerance. It still defaults to `cpu`: flipping four shipped audio models onto a device arm is not a default the row that ADDED the arm is entitled to set, and the flip is owed to the wiring row named in [.agents/specs/minimax-music3.md](../.agents/specs/minimax-music3.md) §11.4. Requires a CUDA build — asking for `cuda` without one throws rather than falling back silently ([#672](https://github.com/mudler/vllm.cpp/issues/672)) |
+| `VLLM_CPP_MUSIC3_PROFILE` | unset (off) | Prints a per-STAGE wall-clock and RSS split for a MiniMax-Music3 request to **stderr** when it finishes. `examples/minimax_music3_gen/main.cpp` times only the checkpoint load and the whole of `vllm_synthesize`, and at a real duration those two numbers cannot say which of the six stages spent the time — four of the six are still host reference loops ([.agents/specs/minimax-music3.md](../.agents/specs/minimax-music3.md) §11.1, §14.2), so the split is the question. Rows are LEAVES (they partition the request and are summed) or SPANS (they enclose leaves and are printed but never added), plus pure counters; `unattributed` is the glue between the leaves and is printed rather than spread silently over the measured stages. `calls` on `denoise.dit_device` counts denoise STEPS, so the DiT forward count is twice it. Accepts `1`, `true`, `on`, `yes` (case-insensitive) — **any other value, including a near miss like `y`, leaves it off**, because an operator who mistypes should get a run with no table rather than a run whose meaning quietly changed. With it off no clock is read and no `/proc` file is opened. It is an attribution instrument and takes no GPU clock window ([.agents/benchmarking.md](../.agents/benchmarking.md)), so its rows are a within-run split and must not be quoted as per-kernel or cross-box figures ([#672](https://github.com/mudler/vllm.cpp/issues/672)) |
 | `VT_ENABLE_JUMP_FORWARD` | off | Opt-in to jump-forward constrained decoding (SGLang parity SW3): when a grammar/structured-output request reaches a state with exactly one valid next token, that token is emitted without a model step. Currently drives only the standalone driver (`DrainForcedTokens`); output-identical by construction (it fires only where the constrained sampler already has a single valid token), so it changes speed, never tokens. Off by default until the production scheduler splice (jumped-token KV recompute) lands. Set `1`/`true`/`on` to enable |
 | `VT_SERVER_MAX_PROMPT_CHARS` | `200000` | Rejects larger `/v1/chat/completions` prompts before scheduling. `0` disables the guard. This is a character count after chat-template rendering, not a token limit |
 | `VT_SERVER_MAX_NEW_TOKENS` | `4096` | Caps the request's `max_tokens` value for `/v1/chat/completions`. `0` disables the cap |

.agents/kernel-matrix.md

@@ -79,7 +79,7 @@ are our reading of their documented behavior, not measurements.
 | GPTQ | ◐ CPU dequant | ✅ | ✅ | ☐ |
 | MXFP4 compressed-tensors | ◐ W4A16 Marlin, mem 2.63x less. gate_up FUSION + decode-graph default-ON; #44 3/3, 32B 6/6. **`VT_MARLIN_DENSE` DEFAULT-ON** (`KERNEL-MARLIN-DENSE-EXEC`): dense marlin 48-CTA, byte-faithful, beats MoE (c8 0.969) | ✅ | ✅ | ☐ |
 | fp8 weights, per-tensor scale | ✅ | ✅ | ✅ | ☐ |
-| Block-wise (fine-grained 128x128) FP8, the `weight_scale_inv` layout | ☐ REFUSED BY NAME at load (#1166): `Qwen/Qwen3.8-27B-FP8` declares `weight_block_size` [128, 128] and this build is per-tensor FP8 only ([spec](../.agents/specs/fp8-blockwise-refusal.md)) | ✅ | ✅ | ☐ |
+| Block-wise (fine-grained 128x128) FP8, the `weight_scale_inv` layout | ◐ LOADS, cannot run (#1189 M3): weight + `cdiv` scale rung + config/tensor cross-check; BF16 scale widened to f32. Linear method is M4, so `Prepare` refuses by name ([spec](../.agents/specs/model-fp8-block-weight.md)) | ✅ | ✅ | ☐ |
 | Per-tensor FP8 W8A8 linear is a shared seam any model can bind | ✅ `models/dense_fp8_gemm.h` + `layers::Fp8W8A8LinearMethod` (#940), bound via `layers::MakeLinearMethod`. One definition, CUDA only ([spec](../.agents/specs/vt-fp8-shared-seam.md)) | ✅ `Fp8LinearMethod` | ✅ | ☐ |
 | FP8 W8A8 works on a CUDA arch without `cutlass-fp8` | ✅ `vt::QuantFp8Static` registers from an unconditional TU (#960); sm_110 measured ([spec](../.agents/specs/vt-fp8-quant-arch-gate.md)) | ✅ | ✅ | ☐ |
 | fp8-tower GDN `in_proj` emits bf16, unlocking packed GDN decode | ◐ `VT_GDN_FP8_IN_BF16` + `VT_GDN_PACKED_DECODE_FP8_TOWER` (inert alone), both default **OFF**, ungated (#339) ([spec](../.agents/specs/perf-fp8-alpha-fold.md)) | ✅ bf16 `out_dtype` | ☐ | ☐ |

.agents/specs/expert-streaming.md

@@ -596,32 +596,41 @@ quantizes the activation once; a checkpoint whose scales differ keeps the two
 separate GEMMs automatically. `VT_GDN_MERGED_QKVZ_FP8=0` restores the two GEMMs
 in the same binary.
 
-### Block-wise FP8 is refused at load
+### Block-wise FP8 loads and does not run yet
 
-This build reads per-tensor FP8, where one scale covers a whole weight. It does
-not read block-wise FP8, also called fine-grained FP8, where one scale covers
-each 128x128 block of the weight. A block-wise checkpoint declares
-`quantization_config.weight_block_size` in its `config.json`, and it stores its
-scales under `weight_scale_inv` rather than under `weight_scale`.
+Block-wise FP8, also called fine-grained FP8, keeps one scale for each 128x128
+block of a weight rather than one scale for the whole weight. A block-wise
+checkpoint declares `quantization_config.weight_block_size` in its
+`config.json` and stores its scales under `weight_scale_inv` rather than under
+`weight_scale`.
 
 `Qwen/Qwen3.8-27B-FP8` is such a checkpoint. At revision
 `017b9c7af6b5689d5dd426a76e0bc077eb5ca20a` it declares `weight_block_size`
 `[128, 128]` with `activation_scheme` `dynamic`, and it stores
 `self_attn.q_proj.weight` as `F8_E4M3` `[12288, 5120]` beside
 `self_attn.q_proj.weight_scale_inv` as `BF16` `[96, 40]`.
 
-Loading it stops with a message that names the key:
+That checkpoint now LOADS. The weights are read into a block-wise FP8 weight,
+the `BF16` scale is widened to `F32` by value the way vLLM widens it, and the
+config is cross-checked against the tensors so a disagreement is named rather
+than guessed at. Nothing can execute the weight yet, so the model refuses to
+finish preparing:
 
 ```text
-quantization_config.weight_block_size [128, 128] selects block-wise
-(fine-grained) FP8, which is not implemented. This build implements per-tensor
-FP8 only.
+block-wise (fine-grained) 128x128 FP8 weights LOADED for
+model.layers.0.self_attn.q_proj and nothing in this build can execute them
 ```
 
-The refusal is deliberate. Nothing is wrong with that checkpoint, and the
-missing arm is in this project. To run the same model here, use a per-tensor
-FP8, BF16, NVFP4, or GGUF checkpoint of it. Issue
-[#1166](https://github.com/mudler/vllm.cpp/issues/1166) tracks the port.
+Two block-wise configurations are refused earlier, at load, because no build
+here implements them: an `activation_scheme` other than `dynamic`, and a
+`weight_block_size` other than `[128, 128]`. Both messages name the key and the
+value your `config.json` declares.
+
+Nothing is wrong with those checkpoints; the missing arm is in this project. To
+run the same model today, use a per-tensor FP8, BF16, NVFP4, or GGUF checkpoint
+of it. Issue [#1189](https://github.com/mudler/vllm.cpp/issues/1189) tracks the
+remaining milestones, and
+[#1166](https://github.com/mudler/vllm.cpp/issues/1166) is the original report.
 
 ### A per-tensor scale has to be one F32 number
 
@@ -2192,6 +2201,44 @@ arm — `VLLM_CPP_MUSIC3_DEVICE=1` selects the device one, unset is the CPU one
 at the same bounds, with the same negative control. Numbers for both are in
 [BENCHMARKS](BENCHMARKS.md).
 
+#### Where the time actually goes: `VLLM_CPP_MUSIC3_PROFILE`
+
+The table above says which stage runs where. It does not say what each one
+*costs*, and at a real duration that is the only question anyone asks. Set
+
+```sh
+VLLM_CPP_MUSIC3_PROFILE=1 minimax-music3-gen --model ... --duration 20 --steps 30 --device 1
+```
+
+and the engine prints a `MUSIC3_PROFILE` table to **stderr** when the request
+finishes: one row per stage with seconds, a call count, and its share of the
+request, then the resident-set size at each stage boundary.
+
+Read it as follows.
+
+* `leaf` rows partition the request and are the ones that add up. `span` rows
+  enclose leaves — `ar.TOTAL_loop`, `denoise.TOTAL` — and are printed for
+  context but never summed, so the table cannot claim more work than the run
+  contained.
+* `cnt` rows carry no time at all. They are the counts a split has to state to
+  be readable: frames, windows, requested steps.
+* `unattributed` is the glue between the leaves — chunk slicing, the overlap
+  blend, the Euler step, the WAV assembly. It is printed rather than spread
+  silently over the measured stages, so a bracket in the wrong place shows up as
+  a number instead of as a plausible share somewhere else.
+* the `calls` column on `denoise.dit_device` counts *steps*, not forwards: one
+  bracket covers both classifier-free-guidance branches, so the forward count is
+  twice it.
+
+It is **off unless the variable is set to `1`, `true`, `on` or `yes`**. Any
+other value, including a near miss like `y`, leaves it off — an operator who
+mistypes gets a run with no table rather than a run whose meaning quietly
+changed. With it off, no clock is read and no `/proc` file is opened.
+
+This is an attribution instrument, not a benchmark harness: it takes no GPU
+clock window, so its rows are a within-run **split** and must not be quoted as
+per-kernel or cross-box figures.
+
 **Measured, so expectations are calibrated rather than hoped for.** On a Jetson
 Thor (sm_110, 14 cores) the device arm was *slower* on a two-frame request
 (846.6 s vs 835.1 s) and 5.4 % faster on a ten-frame one (1430.4 s vs 1512.1 s).

.agents/specs/gdn-replayssm.md

@@ -0,0 +1,85 @@
+ARCH=sm_121a
+ARCH_HITS_IN_RES_USAGE=2
+COMPILE_RC=0
+CUOBJDUMP_RC=0
+LINEINFO_COMPILE_RC=0
+OBJECT_BYTES=435184
+NVCC_VERSION=Cuda compilation tools, release 13.0, V13.0.88 Build cuda_13.0.r13.0/compiler.36424714_0 
+PTXAS_VERSION=Cuda compilation tools, release 13.0, V13.0.88 Build cuda_13.0.r13.0/compiler.36424714_0 
+COMPILE_CMD=nvcc -std=c++20 -O3 -arch=sm_121a -Xptxas -v -c /tmp/replayssm-w0b/probe.cu -o /tmp/replayssm-w0b/probe.o
+PROBE_SHA256=f7d323651cf1a5720f0ce712b802a10248278566e1abb9ecdbae8e62fcf0f4b3
+SCRIPT_SHA256=c5218d087949e94647e28d6799b2f870e34643bc754dfaec084b562fd979ef55
+--- ptxas -v ---
+ptxas info    : 0 bytes gmem
+ptxas info    : Compiling entry function '_Z25ProbeReplaySsmFusedKernelI13__nv_bfloat16fLi128ELi16ELi32ELi64EEvPT_PKS1_S4_S4_PKvN45_GLOBAL__N__f6ac47b2_8_probe_cu_24629ce9_31385DTypeES6_S8_PT0_SA_SA_PfPKiSD_lllllllllf' for 'sm_121a'
+ptxas info    : Function properties for _Z25ProbeReplaySsmFusedKernelI13__nv_bfloat16fLi128ELi16ELi32ELi64EEvPT_PKS1_S4_S4_PKvN45_GLOBAL__N__f6ac47b2_8_probe_cu_24629ce9_31385DTypeES6_S8_PT0_SA_SA_PfPKiSD_lllllllllf
+    0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
+ptxas info    : Used 95 registers, used 0 barriers, 1224 bytes smem
+ptxas info    : Compile time = 203.592 ms
+ptxas info    : Compiling entry function '_Z28ProbeReplaySsmFlushSeqKernelI13__nv_bfloat16fLi128ELi16ELi64EEvPT_PKS1_S4_S4_PKvN45_GLOBAL__N__f6ac47b2_8_probe_cu_24629ce9_31385DTypeES6_S8_PT0_SA_SA_PfPKiSD_lllllllllf' for 'sm_121a'
+ptxas info    : Function properties for _Z28ProbeReplaySsmFlushSeqKernelI13__nv_bfloat16fLi128ELi16ELi64EEvPT_PKS1_S4_S4_PKvN45_GLOBAL__N__f6ac47b2_8_probe_cu_24629ce9_31385DTypeES6_S8_PT0_SA_SA_PfPKiSD_lllllllllf
+    0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
+ptxas info    : Used 94 registers, used 0 barriers, 1156 bytes smem
+ptxas info    : Compile time = 178.604 ms
+ptxas info    : Compiling entry function '_Z25ProbeReplaySsmFlushKernelI13__nv_bfloat16fLi128ELi16ELi64EEvPT_PKS1_S4_S4_PKvN45_GLOBAL__N__f6ac47b2_8_probe_cu_24629ce9_31385DTypeES6_S8_PT0_SA_SA_PfPKiSD_lllllllllf' for 'sm_121a'
+ptxas info    : Function properties for _Z25ProbeReplaySsmFlushKernelI13__nv_bfloat16fLi128ELi16ELi64EEvPT_PKS1_S4_S4_PKvN45_GLOBAL__N__f6ac47b2_8_probe_cu_24629ce9_31385DTypeES6_S8_PT0_SA_SA_PfPKiSD_lllllllllf
+    0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
+ptxas info    : Used 94 registers, used 0 barriers, 1156 bytes smem
+ptxas info    : Compile time = 178.771 ms
+ptxas info    : Compiling entry function '_Z28ProbeReplaySsmNonFlushKernelI13__nv_bfloat16fLi128ELi16EEvPT_PKS1_S4_S4_PKvN45_GLOBAL__N__f6ac47b2_8_probe_cu_24629ce9_31385DTypeES6_S8_PT0_SA_SA_PfPKiSD_lllllllllf' for 'sm_121a'
+ptxas info    : Function properties for _Z28ProbeReplaySsmNonFlushKernelI13__nv_bfloat16fLi128ELi16EEvPT_PKS1_S4_S4_PKvN45_GLOBAL__N__f6ac47b2_8_probe_cu_24629ce9_31385DTypeES6_S8_PT0_SA_SA_PfPKiSD_lllllllllf
+    0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
+ptxas info    : Used 42 registers, used 0 barriers, 1224 bytes smem
+ptxas info    : Compile time = 123.199 ms
+ptxas info    : Compiling entry function '_Z27ProbeReplaySsmRegTileKernelI13__nv_bfloat16fLi128ELi16EEvPT_PKS1_S4_S4_PKvN45_GLOBAL__N__f6ac47b2_8_probe_cu_24629ce9_31385DTypeES6_S8_PT0_SA_SA_PfPKiSD_lllllllllf' for 'sm_121a'
+ptxas info    : Function properties for _Z27ProbeReplaySsmRegTileKernelI13__nv_bfloat16fLi128ELi16EEvPT_PKS1_S4_S4_PKvN45_GLOBAL__N__f6ac47b2_8_probe_cu_24629ce9_31385DTypeES6_S8_PT0_SA_SA_PfPKiSD_lllllllllf
+    96 bytes stack frame, 96 bytes spill stores, 96 bytes spill loads
+ptxas info    : Used 255 registers, used 0 barriers, 96 bytes cumulative stack size, 1224 bytes smem
+ptxas info    : Compile time = 269.512 ms
+ptxas info    : Compiling entry function '_Z25ProbeControlRegTileKernelI13__nv_bfloat16fLi128EEvPT_PKS1_S4_S4_PKvN45_GLOBAL__N__f6ac47b2_8_probe_cu_24629ce9_31385DTypeES6_S8_PT0_PKilllllllllf' for 'sm_121a'
+ptxas info    : Function properties for _Z25ProbeControlRegTileKernelI13__nv_bfloat16fLi128EEvPT_PKS1_S4_S4_PKvN45_GLOBAL__N__f6ac47b2_8_probe_cu_24629ce9_31385DTypeES6_S8_PT0_PKilllllllllf
+    56 bytes stack frame, 56 bytes spill stores, 56 bytes spill loads
+ptxas info    : Used 255 registers, used 0 barriers, 56 bytes cumulative stack size, 1024 bytes smem
+ptxas info    : Compile time = 187.856 ms
+--- cuobjdump -res-usage ---
+
+Fatbin ptx code:
+================
+arch = sm_121
+code version = [9,0]
+host = linux
+compile_size = 64bit
+compressed
+ptxasOptions = -v  
+
+Fatbin elf code:
+================
+arch = sm_121a
+code version = [1,8]
+host = linux
+compile_size = 64bit
+
+Resource usage:
+ Common:
+  GLOBAL:0
+ Function _Z25ProbeReplaySsmFusedKernelI13__nv_bfloat16fLi128ELi16ELi32ELi64EEvPT_PKS1_S4_S4_PKvN45_GLOBAL__N__f6ac47b2_8_probe_cu_24629ce9_31385DTypeES6_S8_PT0_SA_SA_PfPKiSD_lllllllllf:
+  REG:95 STACK:0 SHARED:2248 LOCAL:0 CONSTANT[0]:1084 TEXTURE:0 SURFACE:0 SAMPLER:0
+ Function _Z28ProbeReplaySsmFlushSeqKernelI13__nv_bfloat16fLi128ELi16ELi64EEvPT_PKS1_S4_S4_PKvN45_GLOBAL__N__f6ac47b2_8_probe_cu_24629ce9_31385DTypeES6_S8_PT0_SA_SA_PfPKiSD_lllllllllf:
+  REG:94 STACK:0 SHARED:2180 LOCAL:0 CONSTANT[0]:1084 TEXTURE:0 SURFACE:0 SAMPLER:0
+ Function _Z25ProbeReplaySsmFlushKernelI13__nv_bfloat16fLi128ELi16ELi64EEvPT_PKS1_S4_S4_PKvN45_GLOBAL__N__f6ac47b2_8_probe_cu_24629ce9_31385DTypeES6_S8_PT0_SA_SA_PfPKiSD_lllllllllf:
+  REG:94 STACK:0 SHARED:2180 LOCAL:0 CONSTANT[0]:1084 TEXTURE:0 SURFACE:0 SAMPLER:0
+ Function _Z28ProbeReplaySsmNonFlushKernelI13__nv_bfloat16fLi128ELi16EEvPT_PKS1_S4_S4_PKvN45_GLOBAL__N__f6ac47b2_8_probe_cu_24629ce9_31385DTypeES6_S8_PT0_SA_SA_PfPKiSD_lllllllllf:
+  REG:42 STACK:0 SHARED:2248 LOCAL:0 CONSTANT[0]:1084 TEXTURE:0 SURFACE:0 SAMPLER:0
+ Function _Z27ProbeReplaySsmRegTileKernelI13__nv_bfloat16fLi128ELi16EEvPT_PKS1_S4_S4_PKvN45_GLOBAL__N__f6ac47b2_8_probe_cu_24629ce9_31385DTypeES6_S8_PT0_SA_SA_PfPKiSD_lllllllllf:
+  REG:255 STACK:96 SHARED:2248 LOCAL:0 CONSTANT[0]:1084 TEXTURE:0 SURFACE:0 SAMPLER:0
+ Function _Z25ProbeControlRegTileKernelI13__nv_bfloat16fLi128EEvPT_PKS1_S4_S4_PKvN45_GLOBAL__N__f6ac47b2_8_probe_cu_24629ce9_31385DTypeES6_S8_PT0_PKilllllllllf:
+  REG:255 STACK:56 SHARED:2048 LOCAL:0 CONSTANT[0]:1052 TEXTURE:0 SURFACE:0 SAMPLER:0
+
+Fatbin ptx code:
+================
+arch = sm_121a
+code version = [9,0]
+host = linux
+compile_size = 64bit
+compressed
+ptxasOptions = -v