docs: pin and reproduce Qwen3.8-2.4T-A95B UD-Q1_0 on one GB10 (#1194)

.agents/issue-index.md

@@ -383,6 +383,7 @@ rather than merged. `scripts/check-agent-record.py` gates both.
 | [#1189](https://github.com/mudler/vllm.cpp/issues/1189) | `VT-QUANT-FP8-GROUP` | Block-wise (128x128) FP8 so `Qwen/Qwen3.8-27B-FP8` runs instead of being refused. `weight_block_size` appears nowhere in `src/` or `include/`, so `469f38395` refuses the arm by name (#1166). Six independently landable milestones; M1 lands here. M1 is `vt::QuantFp8Group`, the dynamic per-token per-group activation quant, CPU and CUDA. The numerics mirror the kernel that ACTUALLY EXECUTES on a CUDA-alike platform with a contiguous input, which is the C++ custom op at `csrc/libtorch_stable/quantization/w8a8/fp8/per_token_group_quant.cu:42-96` and NOT the Triton kernel at `fp8_utils.py:95-150`: `fp8_utils.py:635-650` calls the former and returns before the latter. The two arms differ, and the difference is measurable rather than cosmetic. The CUDA kernel divides twice, `local_absmax / max_8bit` at `:68` and `static_cast<float>(src) / y_s` at `:85`; the Triton kernel multiplies by `(1.0 / fp8_max)` at `fp8_utils.py:145` under a comment that names the 1-ULP gap. Upstream's own test tolerates the gap with `rtol=0.15` (`test_block_fp8.py:112-114`), so a value comparison cannot tell the two apart and only a byte comparison against a spelled-out reference can. `eps` is the reduction's INITIAL value (`:47`), not a post-clamp, which is what keeps an all-zero group from dividing by zero. Scope refused here and owed to later milestones: the block-scaled GEMM (M2), `Fp8BlockWeight` and the loader (M3), `Fp8BlockLinearMethod` and the Qwen3.5 wiring (M4), the mainloop-scaled CUTLASS kernel and the column-major/TMA-aligned scale layouts (M5), merged `gate_up`/QKV (M6). M1 lands UNREACHED: no production entry point dispatches `vt::QuantFp8Group` at its merge commit, M4 owns the wiring, and `.agents/specs/vt-quant-fp8-group.md` lists it under `## Owed`. The CUDA arm compiles and its on-hardware leg is owed too, because the row took no GPU lease by design: the CPU arm is the gateable one | feature |
 | [#1190](https://github.com/mudler/vllm.cpp/issues/1190) | `ENV-AGNOSTIC-CAMPAIGN` | One operator's hosts, share paths and addresses are written into 227 tracked files where a `.env` placeholder belongs, so a second developer who follows the protocol documents is told to reach a box on one home network. Re-derived at `fd64c76ee` with `git grep -cIE 'dgx\.casa\|nas_share\|192\.168\.\|thor:gpu0'`: `dgx.casa` 203 files, `nas_share` 32, `192.168.` 24, `thor:gpu0` 11. The mechanism was already there and almost unused: `.env.example` is tracked, `.env` and `.agents/developer-preferences.md` are ignored, `scripts/agent-onboard.py --env-set` already records one answered value and refuses an undeclared key, and `${VLLM_SOURCE}` and `${GPU_LOCK}` already resolve in 59 and 50 files while `${DEVICE_ARCH}`, `${DEVICE_TOOLKIT_ROOT}` and `${DEVICE_COMPILER}` resolve in none. This SCOPING row lands the rule and the mechanism, not the sweep. The rule is a substitution test: replace the literal with a second developer's value and if the sentence stays true it is CONFIGURATION and becomes `${KEY}`, and if it becomes false it is PROVENANCE and stays literal. That reverses the issue's own ranking, because all 34 hits in the densest guide file `.agents/environment.md` are provenance or named-profile definition and none is configuration, so density does not predict the defect and a blind `sed` would falsify records. Landed: three new keys `GATE_CHECKOUT`, `SHARED_STORAGE_ROOT` and `GATE_DEVICE` derived from the literals that recur and map to no existing key; the create-on-first-use route in `scripts/agent-start.py`, which printed `environment: missing` as a status label and then listed next actions that never mentioned it, so the fallback in practice was a host name copied from a document; the matching obligation in `AGENTS.md`; and one worked example. The worked example `scripts/dgx-bringup.sh` found a live defect rather than a cosmetic one: it defaulted `CUTLASS_DIR` to `$HOME/cutlass_probe` while `.agents/environment.md:389` records `$HOME/cutlass-4.5.0` as mandatory on the same box, and a configure that misses CUTLASS silently drops the sm120a NVFP4 GEMM and FlashAttention-2, which that file measures as moving the SACRED `test_qwen27_paged_engine` from 235/235 to 234/235 with the source untouched, so a stale hard-coded default is a false green. Its new test also caught that `set -a; . ./.env; set +a` over an `.env.example` copy blanks a value the caller exported, so the process-environment contract is now executable. Waves `ENV-AGNOSTIC-W1-TOOLING` through `ENV-AGNOSTIC-W5-LEDGERS` own the sweep, partition all 227 files, and open their own issues. Spec [`env-agnostic.md`](specs/env-agnostic.md) | bug |
 | [#1193](https://github.com/mudler/vllm.cpp/issues/1193) | `SPEC-DSPARK-QWEN3-ROUTING` | A Qwen3 DSpark draft declaring `architectures=["DSparkDraftModel"]` with `model_type` `qwen3` has no route. The pin forces every DSpark draft that is not `Qwen3DSparkModel` or `Gemma4DSparkModel` onto `model_type` `deepseek_v4` (`vllm/config/speculative.py:934-944` @ `555967922`), and vLLM PR 52197 (merged 2026-08-17 at `7075ddac`) replaced that with a leading branch normalizing the pair to `Qwen3DSparkModel`. We diverge from BOTH: the forced rewrite was never ported, so nothing in `src/vllm/entrypoints/model_loader.cpp` reads a draft config's `architectures` key at all, and `SpeculativeConfig::IsDsparkDraft` (`include/vllm/config/speculative.h:120-136`) has no production caller — every reference outside its header is in `tests/vllm/config/test_speculative_dspark.cpp:132-140`, and `ResolveSpecConfig` branches on `cli.method` alone. The checkpoint is real and gateable here: `RadixArk/Qwen3.8-27B-DSpark` at revision `85ef153be924f17ce4bf62726954eeaa4a73e854` carries exactly that config shape in one 2718576122-byte shard, drafting five layers for a 64-layer Qwen3.8-27B target | bug |
+| [#1194](https://github.com/mudler/vllm.cpp/issues/1194) | `ENG-EXPERT-STREAM` | `Qwen3.8-2.4T-A95B UD-Q1_0` serves on one 119.631 GiB GB10 and no user-facing document said how. AGENTS.md binds the checkpoint pin to [`docs/USAGE.md`](../docs/USAGE.md): file name, size, repo AND revision, sha256 for a quantized artifact. Revision `567d3e6ac26c5474b18311e619c04350fb9a5556` appeared only in this index (row #1123) and in [`expert-streaming.md`](specs/expert-streaming.md), and `USAGE.md` named the checkpoint three times and pinned it zero times. Nor was there an end-to-end recipe: the `Streaming routed experts from disk (capacity mode)` section is a MECHANISM reference (schema, precedence, #1135 reach limits, statistics line, the per-device limits) and never says which file `--model` takes, what the load costs, what decode costs, or where the ceiling is, while its four sibling per-model recipes in the same file (Muse Glimmer 30B, Nemotron-3.5-Lightning-30B, MiniMax-H3, MiniMax-Music3) all do. FIXED as a fifth sibling recipe that LINKS the mechanism section rather than restating it, so each fact keeps one home; `docs/examples/` was NOT created, because `examples/` in this tree is runnable C++ CMake targets and `docs/` is one topic per file listed in the README index, so a new markdown genre would have had one member and duplicated facts that already have owners. The pin is settled against the ARTIFACT rather than against a document or the HuggingFace tree API, whose `lfs.oid` is fabricated for a gated repo: **ten** shards `UD-Q1_0/Qwen3.8-2.4T-A95B-UD-Q1_0-000{01..10}-of-00010.gguf` summing to exactly **397,256,393,248 B (369.97 GiB)**, shard 1 declaring `split.count = 10` and `split.tensors.count = 1702` in its own metadata and carrying zero tensors, and shard 1's sha256 recomputed from the mirrored copy as `b7770552...0334d` (10,943,264 B), equal to the download manifest that also gives shard 2 `2765517f...3dcd` (48,759,636,544 B). That settles [#1420](https://github.com/mudler/vllm.cpp/issues/1420), whose two copy-paste commands named `-00001-of-00008.gguf`, a file that does not exist at that revision, and both now name `-00001-of-00010.gguf`. The section publishes no new measurement and carries only figures already recorded: the CPU arm's **11.05 s/token at 4000 expert-stream slots** (`ENG-EXPERT-STREAM-DEVICE` W0e), the 8000-slot 39.98 to 45.40 s/token memory result, and the 66.7 s/token streaming-off run, with NO ratio between them, because they were taken on different source trees. No `--device cuda` speed number appears, because that arm's token gate FAILS and `docs/BENCHMARKS.md` carries G0-SPEED as VOID. The unit arithmetic was re-derived rather than transcribed: 2790 slices x 2,490,368 B is **6.95 GB (6.47 GiB)** per token, so a ~5 GB/s NVMe floor is **1.39 s/token (~0.72 tok/s)**, not the 1.29 s / 0.77 tok/s a GiB-for-GB substitution gives | doc |
 | [#1202](https://github.com/mudler/vllm.cpp/issues/1202) | — | `Ltx2FuseLoraIntoTensor` (`src/vllm/model_executor/models/ltx2_lora.cpp:321-334`) computes the `(B * strength) @ A` LoRA product with a scalar single-threaded triple loop: one thread, no blocking, no SIMD, a non-inlined `vt::BF16ToF32` per multiply, and an inner operand `pair->a[k * cols + i]` striding by `cols` so every load in the innermost loop is its own cache line. Measured on `dgx` (GB10, 20 cores) loading the full/dev transformer (21,004,025,600 params) with the shipped 8.9 GB distilled adapter: three `gdb` stacks all reading `vt::BF16ToF32` <- `Ltx2FuseLoraIntoTensor` <- `Ltx2LoadDitFromSafetensors` <- `Ltx2VideoEngine::Load`, one thread at 99.9% of one core with 19 idle, and an f32 working set growing 9.432 -> 10.235 GiB over 300-629 s = **2.3% of one pass in 10.4 minutes**, cross-checked against the sum of `out*in*rank` over the 1660 targeted modules = 8.53e12 MAC, consistent with ~0.53 GFLOP/s. The operation is a rank-`r` GEMM and belongs on the `vt::` GEMM seam like every other projection in the tree; the arithmetic is already a correct mirror of `fuse_loras.py:103-116` (`B * strength` rounds to bf16 BEFORE the product, f32 accumulation, bf16 store) and only the execution strategy is wrong, so a replacement has a bit-exact oracle rather than a tolerance. Blocks every LoRA-bearing pipeline kind on the full model; `one_stage` is unaffected because upstream marks it `Full` with no adapter, which is why it is the only full-model arm that currently reaches generation. Owed by [`ltx25-decode-speed.md`](specs/ltx25-decode-speed.md) `## Owed`, whose §5 already frames "why the decode is single-threaded and on the host" | perf |
 | [#1208](https://github.com/mudler/vllm.cpp/issues/1208) | — | `Linear` in the LTX-2.5 text tower (`src/vllm/model_executor/models/ltx2_text_encoder.cpp:60-71`) is a scalar single-threaded triple loop with a `double` accumulator and both operands widened per multiply, so it cannot reach the f32 FMA path even if the compiler vectorised it. On the full model's caption projection (in_features on the order of 1.9e5) cost is `rows * out_features * in_features` and it dominates the pre-generation wall: measured on `dgx`, resident memory went flat at t≈150 s and stayed **byte-identical for 1073 s** with stacks in the text feature extractor. Because `ltx2-gen` prints nothing between load and completion ([#1010](https://github.com/mudler/vllm.cpp/issues/1010) owns that), this presents to a user as a hang rather than as slow arithmetic, and it is why an earlier probe's "reached Generate" claim was retracted — the trace does not support it. TWO separate defects: the execution strategy, which belongs on the `vt::` GEMM seam; and the `double` accumulator, which is NOT a mirror — `torch.nn.functional.linear` on f32 inputs accumulates in f32, and the comment directly above this function cites `F.linear` as its reference, so the widening diverges from the oracle it names, cannot be bit-compared against upstream, and hides reduction-order differences an f32 accumulator would expose. Per the dtype-polarity rule an f64 accumulator kept deliberately needs a one-line reason beside it. Every LTX-2.5 pipeline kind goes through the text tower, so this is on the critical path of all of them including `one_stage`. Owed by [`ltx25-decode-speed.md`](specs/ltx25-decode-speed.md) `## Owed` | perf |
 | [#1210](https://github.com/mudler/vllm.cpp/issues/1210) | — | The two-stage LoRA rebind cost that `src/vllm/multimodal/ltx2_video.cpp:2843-2851` records as "UNMEASURED on real weights", with a later perf row owning the number, is now measured. A two-stage recipe loads FUSED, phase 0 asks `Ltx2PhaseLoraScope::kNoAdapters` and `Ltx2RebindDitLoras` un-fuses, phase 1 asks `kAllAdapters` and re-fuses — so the load-time fusion is **provably wasted**, undone before any denoise step runs, and the DiT is left fused so the next render pays the same two again. At [#1202](https://github.com/mudler/vllm.cpp/issues/1202)'s measured ~0.53 GFLOP/s each pass is hours, and a two-stage full-model render pays three of them before the first step. TWO independent fixes, not one change: making the fusion fast (#1202) shrinks the constant but leaves the wasted round trip; separately, `Ltx2PipelineRecipe::phases` is available before `Load` runs, so the load could honour phase 0's scope and skip the fuse/un-fuse entirely. The terminal fused state is chosen rather than forced and is worth revisiting in the same change. Affects `a2vid_two_stage` and `ti2vid_two_stage`; `one_stage` pays nothing. NOT CLAIMED: the wall-clock figures are a rate measured over a 10.4-minute window and extrapolated, not a completed pass — no two-stage full-model render has completed, so the end-to-end number stays open; the rate, thread count and stack attribution are measured. Owed by [`ltx25-decode-speed.md`](specs/ltx25-decode-speed.md) `## Owed` | perf |

.agents/specs/expert-streaming.md

@@ -1586,6 +1586,39 @@ into other suites.
 | a fitting GGUF still loads | the SAME call with a generous budget: the throw is a LATER, different one (the synthetic file has no tokenizer), which is what proves the check let it through rather than that it never ran |
 | the CPU arm is untouched | the same file with `device=cpu` never refuses, whatever the budget |
 
+## The user-facing recipe and the checkpoint pin ([#1194](https://github.com/mudler/vllm.cpp/issues/1194))
+
+`docs/USAGE.md` named `Qwen3.8-2.4T-A95B UD-Q1_0` three times and pinned it zero
+times. AGENTS.md binds the pin to that file: file name, size, repo AND revision,
+and a sha256 for a quantized artifact. The streaming section there is a MECHANISM
+reference. It never said which file `--model` takes, what the load costs, what
+decode costs, or where the ceiling is, while the four sibling per-model recipes
+in the same file all do.
+
+Landed as a fifth sibling recipe, `Qwen3.8-2.4T-A95B UD-Q1_0: 370 GiB served from
+a 119 GiB box`, which LINKS the mechanism section rather than restating it, so
+each fact keeps one home. Three facts are repeated on purpose and the section
+says which ones and why: which device to use, the expert bytes a token reads, and
+the two streaming decode figures. Both places quote `ENG-EXPERT-STREAM-DEVICE`
+W0e, so the re-measure that row owes has to change both.
+
+The pin is settled against the ARTIFACT rather than against a document or the
+HuggingFace tree API, whose `lfs.oid` is fabricated for a gated repo: TEN shards
+`UD-Q1_0/Qwen3.8-2.4T-A95B-UD-Q1_0-000{01..10}-of-00010.gguf` summing to exactly
+397,256,393,248 B (369.97 GiB), with shard 1 declaring `split.count = 10` and
+`split.tensors.count = 1702` in its own metadata and carrying no tensors. That
+also answers [#1420](https://github.com/mudler/vllm.cpp/issues/1420), whose two
+copy-paste commands named `-of-00008`, a file that does not exist at that
+revision.
+
+The section publishes no new measurement. Every figure in it was already
+recorded, it carries no ratio between runs taken on different source trees, and
+it carries no `--device cuda` speed number, because that arm's token gate fails.
+The figures come from the W0e harness `benchmarks/expert_stream_device_w0e.cpp`
+and not from the `vllm-server` command the section publishes, and the section
+discloses that difference beside the prompt and environment differences it
+already disclosed.
+
 ## Owed
 
 Carried debt for this row. Each item names why it is not closed here.