fix(ENG-EXPERT-STREAM): seal the IQ1 decode against the oracles, not against itself

.agents/issue-index.md

@@ -270,3 +270,4 @@ rather than merged. `scripts/check-agent-record.py` gates both.
 | [#995](https://github.com/mudler/vllm.cpp/issues/995) | `ENG-EXPERT-STREAM` | `check-env-doc` and `test_check_env_doc` RED in `scripts/agent-preflight.sh` on **every** branch based on `3005447f8`: `VT_MOE_EXPERT_STREAM`, `VT_MOE_EXPERT_STREAM_SLOTS` and `VT_MOE_EXPERT_STREAM_SLOT_BYTES` are read from `src/vllm/model_executor/models/qwen3_5.cpp` (`:5145`, `:5164` @ `3005447f8`) and appear in neither `docs/ENVIRONMENT.md` (`grep -c` returns **0**) nor `scripts/env-doc-allowlist.txt`. Arrived with [#993](https://github.com/mudler/vllm.cpp/pull/993). Found while gating [#986](https://github.com/mudler/vllm.cpp/issues/986) and proved pre-existing with a matched-arm check rather than asserted: the three sites are in a file that branch does not touch, and `git diff origin/main...HEAD | grep '^+.*VT_MOE_EXPERT'` returns nothing. NOT fixed in flow: documenting a knob means stating its default and when to touch it, and the expert-streamer's slot accounting belongs to the row that added it, so a plausible-sounding entry written by a passer-by is how `docs/ENVIRONMENT.md` stops being trustworthy. It is a PRE-FLIGHT gate, so it fails before every edit and presents to each author as a red their own diff caused, which is the shape [#965](https://github.com/mudler/vllm.cpp/issues/965) and [#968](https://github.com/mudler/vllm.cpp/issues/968) both took | bug |
 | [#987](https://github.com/mudler/vllm.cpp/issues/987) | `LTX25-RETAKE` | Two `ltx-2.5` refusal messages state reasons that are no longer true. (a) `src/vllm/multimodal/ltx2_video.cpp:1608 @ 0e1bee42f` says "nothing reads `ref_video_dir` at all", and MiniMax-H3 has always consumed the directory in full (`ReadReferenceClipChw`, `src/vllm/multimodal/minimax_h3_video.cpp:135 @ 0e1bee42f`, called at `:650`); [#975](https://github.com/mudler/vllm.cpp/issues/975) inherited the wider claim from this message. The claim that holds is narrower: the LTX-2.5 engine never reads the directory's CONTENTS. (b) `ltx2_video.cpp:1636-1638 @ 0e1bee42f` says "there is no AUDIO_VAE_ENCODER key filter", and `c2019b0e3` landed `Ltx2AudioVaeEncoderKeyRules()` (`include/vllm/model_executor/models/ltx2_audio_input.h:73 @ 0e1bee42f`) with a live call through `Ltx2EncodeAudioToLatent`. Both rewritten in the `WHAT IS *NOT* THE REASON` shape in the same flow, with one assertion tied to the LOCAL fact that the LTX side now reads the directory | bug |
 | [#995](https://github.com/mudler/vllm.cpp/issues/995) | `ENG-EXPERT-STREAM` | `check-env-doc` and `test_check_env_doc` are RED on `origin/main`, so every branch cut from it inherits a preflight failure its own diff did not cause: `VT_MOE_EXPERT_STREAM`, `VT_MOE_EXPERT_STREAM_SLOTS` and `VT_MOE_EXPERT_STREAM_SLOT_BYTES` are read from `src/vllm/model_executor/models/qwen3_5.cpp` (`:5145`, `:5195`, `:5189` @ `4496ef196`) and appear in neither `docs/ENVIRONMENT.md` nor `scripts/env-doc-allowlist.txt`. They arrived with the `ENG-EXPERT-STREAM` W4 wiring commit `3005447f8` ([#993](https://github.com/mudler/vllm.cpp/pull/993)). Same shape as [#965](https://github.com/mudler/vllm.cpp/issues/965)/[#968](https://github.com/mudler/vllm.cpp/issues/968): a pre-existing red presenting to each author in turn as their own. Fixed by DOCUMENTING all three in `docs/ENVIRONMENT.md`, not by allowlisting them — a knob that changes the host memory a MoE decode reserves and disables the default-on grouped-MoE path is a deployment surface, not a kernel-internal micro-tuning switch | bug
+| [#1023](https://github.com/mudler/vllm.cpp/issues/1023) | `ENG-EXPERT-STREAM` | The IQ1_S (ggml 19) / IQ1_XXXS (ggml 66) decode landed in [#946](https://github.com/mudler/vllm.cpp/pull/946) with every parameter EXCEPT the grid pinned only by self-consistency. The grid seal (FNV-1a digest + lane census) works and stops one table short: `ReferenceDotF64` (`tests/vt/test_ops_quant_dot.cpp:231-236`) and the G3 NMSE reference (`:915`) both decode the weight with `vt::cpu::BlockToFloat`, the function under test, so they are independent only in the SUMMATION. Three injected defects, each applied and compiled, left the suite green with an UNCHANGED assertion count: `kIq1sDelta` `0.125F`->`0.25F` (`cpu_quant_iq_tables.h:422`, affects BOTH encodings), the IQ1_S delta sign inverted in dequant AND vec_dot, and the IQ1_S scale read from `qh` bits 13-15 instead of 12-14 in both paths. Second defect, same PR: `gguf_dequant.cpp:107-116` lists no `case 19` and no `case 66`, so the expansion path throws `unsupported ggml type` for the two encodings the target checkpoints are 96.92 % made of, and `RouteGgufTensor` (`gguf_keep_quant.cpp:122`) sends a tensor there whenever `VT_CPU_REF` is on, keep-quant is off, K is ragged, or the role is not verbatim — a refusal to load on the reference lane. **ggml 18 (IQ3_XXS) carried the same omission**, pre-existing since the DeepSeek-V4 UD-IQ2_XXS port, and all three are one shared `switch` branch. Repaired by golden vectors whose EXPECTED values come from the ORACLES themselves (`ggml_get_type_traits(type)->to_float` built from `ggml-org/llama.cpp @ 237ad9b96` for 18/19 and `unslothai/llama.cpp @ 36fe8e1cc` for 66) over REAL checkpoint bytes, which is the first reference for this encoding family that is not this tree's own decoder. Also: the `2e-3` NMSE ceiling passed the doubled-delta defect that `6e-4` fails (measured 5.240e-4 unmutated, 6.967e-4 mutated on `iq1_s`), and on `iq1_xxxs` that defect moves the statistic the WRONG WAY (3.109e-4 -> 1.420e-4), so no ceiling can catch it and only the goldens can; and both new census cases claimed TOTAL coverage of 1702 records while summing to 864, F32's 838 tensors omitted. Spec [`expert-streaming.md`](specs/expert-streaming.md) | bug |

.agents/specs/expert-streaming.md

@@ -557,6 +557,70 @@ oracle gateable: both sides are transcriptions of the same source, so a defect i
 the FORK would be reproduced identically by both. Only building and running the
 fork closes that, which is what #933 owes.
 
+### The decode is now sealed against the ORACLES, not against itself (#1023)
+
+An independent review of #946 found that this seal covered the CODEBOOKS and
+nothing else. `kIq1sGrid` and `kIq1xxxsGrid` each carry an FNV-1a digest and a
+lane census, and corrupting a grid entry does fail. Every other decode parameter
+was pinned only by self-consistency, because `ReferenceDotF64` and the G3
+`MatmulBTQuant` NMSE reference both decode the weight with
+`vt::cpu::BlockToFloat`, the function under test. They are independent in the
+SUMMATION and nowhere else. Three defects were injected, each applied and
+compiled, and the whole gate stayed green with an unchanged assertion count:
+
+| Mutation | Before #1023 | After |
+|---|---|---|
+| `kIq1sDelta` `0.125F` to `0.25F`, which hits BOTH encodings | UNCAUGHT | 3 cases, 2049 assertions fail |
+| IQ1_S delta sign inverted in dequant AND vec_dot | UNCAUGHT | 1 case, 1024 assertions fail |
+| IQ1_S scale from `qh` bits 13-15 instead of 12-14, both paths | UNCAUGHT | 1 case, 960 assertions fail |
+
+What closes it is a committed golden-vector fixture,
+`tests/vt/iq1_golden_vectors.h`, whose EXPECTED values are produced by the
+oracles themselves rather than by this tree:
+`ggml_get_type_traits(type)->to_float`, called in a build of
+`ggml-org/llama.cpp @ 237ad9b96` for IQ1_S and IQ3_XXS, and of
+`unslothai/llama.cpp @ 36fe8e1cc7f2b3b8c92fdda0ab07600141921786` for IQ1_XXXS.
+The IQ1 inputs are real `blk.0.ffn_gate_exps.weight` bytes, four 256-element
+blocks from shard 2 of each split, so this commits a reproducible slice of the
+1179648-weight run above. Agreement is BIT-EXACT on all 1024 values per
+encoding, and `kIq1sDelta` is additionally sealed by value against upstream
+`IQ1S_DELTA` (`ggml-common.h:1121`).
+
+This is stronger than the 15 August run, and it is worth saying how. That run
+compared our C++ against a hand transcription of the fork, so BOTH sides were
+transcriptions and a defect in the FORK would have been reproduced identically
+by each. These vectors are decoded by the fork's own compiled code. That still
+does not make the fork gateable in the sense #933 owes, which is running the
+MODEL, but the fork's decoder is no longer transcribed at all.
+
+The same review found the NMSE ceiling widened past the point where it
+discriminates. It was `2e-3`, described as about 4x the residual. Re-measured 16
+August 2026 over all 12 shapes per type, with the ceiling forced to `1e-12` so
+doctest prints every captured value:
+
+| Type | Unmutated max | With `kIq1sDelta = 0.25` |
+|---|---|---|
+| `iq1_s` | 5.240e-4 (m=4, n=1) | 6.967e-4 |
+| `iq1_xxxs` | 3.109e-4 (m=1, n=1) | 1.420e-4 |
+
+So `2e-3` passed that defect and `6e-4`, the value now set, fails it. The
+`iq1_xxxs` column is the more useful half: the defect moves that statistic the
+WRONG WAY, so NO ceiling catches it there. An NMSE against a dequant-f32
+reference cannot seal a decode parameter at all when both sides decode through
+the same function. The ceiling bounds quantization error, which is its own job;
+the goldens carry the decode.
+
+Two further findings, repaired in the same flow. `DequantGgufRowToF32` listed no
+`case 19` and no `case 66`, so the expansion path threw `unsupported ggml type`
+for the two encodings the target checkpoints are 96.92 % made of, and
+`RouteGgufTensor` routes a tensor there whenever `VT_CPU_REF` is set, keep-quant
+is off, K is ragged, or the role is not verbatim: a refusal to load on the
+reference lane. `case 18` (IQ3_XXS, the DeepSeek-V4 UD-IQ2_XXS `ffn_down`
+encoding) carried the same omission and is fixed with it, gated on its own
+oracle-produced golden. And both checkpoint-census cases claimed TOTAL coverage
+of 1702 tensor records while enumerating six of the seven encodings and summing
+to 864; they now carry F32's 838 tensors and assert that the buckets sum.
+
 Why this is in this spec rather than its own row: the encoding is the load
 path's half of the same capability. A streamer that can address an expert slice
 it cannot decode moves bytes for nothing, so the row's own gate cannot be met

src/vllm/model_executor/model_loader/gguf_dequant.cpp

@@ -113,11 +113,22 @@ std::vector<float> DequantGgufRowToF32(uint32_t ggml_type, const uint8_t* data,
     case 14:   // Q6_K
     case 22:   // IQ2_S  (~2.5-bit codebook; UD-IQ2_M ffn_gate/up experts)
     case 39:   // MXFP4  (OCP micro-scaling fp4; UD-IQ2_M ffn_down experts)
+    case 18:   // IQ3_XXS (~3-bit codebook; UD-IQ2_XXS ffn_down)
+    case 19:   // IQ1_S    (1.5625 bpw; Qwen3.8-2.4T-A95B UD-IQ1_S experts)
+    case 66:   // IQ1_XXXS (1.1875 bpw; UD-Q1_0 experts, fork-anchored)
     case 16: {  // IQ2_XXS (~2-bit codebook; UD-IQ2_XXS DeepSeek-V4 vehicle)
       // The block decoders moved to vt (src/vt/cpu/cpu_quant_dequant.cpp) so
       // the loader oracle and the compute-in-quant GEMM's generic fallback
       // share ONE implementation. The code is byte-identical to what lived
       // here, so numerics are unchanged; this test suite gates that.
+      // This list is the loader's EXPANSION path, and it has drifted behind
+      // `BlockDTypeFromGgmlTypeId` twice: 18, 19 and 66 were all decodable in
+      // vt while this switch still threw "unsupported ggml type" for them. The
+      // drift is not a corner case, because `RouteGgufTensor` routes a tensor
+      // here whenever the VT_CPU_REF oracle switch is on, keep-quant is off, K
+      // is ragged, or the role is not a verbatim one. A missing case is
+      // therefore a refusal to load the checkpoint on the reference lane.
+      // Keep this list in step with vt/dtype.cpp `kBlockDTypes`.
       // NOTE: MXFP4 (39) decodes here through the vt kMXFP4 block dtype — the
       // GGML micro-scaling form (kvalues_mxfp4 == 2*e2m1, scale 2^(byte-128)) —
       // NOT through the compressed-tensors E8M0ToF32 path used for NVFP4 (40)

src/vt/cpu/cpu_quant_dot.cpp

@@ -588,11 +588,14 @@ void VecDotIQ3_XXSQ8_K(int n, float* s, size_t bs, const void* vx, size_t bx,
 // bits from `qh[ib]`; kIq1sGrid entries are packed TERNARY (-1/0/+1) bytes, so
 // there is no sign array to apply.
 //
-// The delta term is why this kernel reads `bsums` and the others do not. IQ1_S
-// reconstructs a weight as dl*(grid[j] + delta), so the dot splits into
-// sum(dl*grid*q8) plus delta*dl*sum(q8), and that second sum over each group of
-// 16 is exactly what Q8_K already caches in `bsums`. Recomputing it from `qs`
-// would be arithmetically identical but would read the activation twice.
+// The delta term is why this kernel reads `bsums`. IQ1_S reconstructs a weight
+// as dl*(grid[j] + delta), so the dot splits into sum(dl*grid*q8) plus
+// delta*dl*sum(q8), and that second sum over each group of 16 is exactly what
+// Q8_K already caches in `bsums`. Recomputing it from `qs` would be
+// arithmetically identical but would read the activation twice.
+//
+// The reason is specific to the delta; the FIELD is not. Q2_K, Q4_K and Q5_K
+// read `bsums` too, for their own per-sub-block minimum term.
 void VecDotIQ1_SQ8_K(int n, float* s, size_t bs, const void* vx, size_t bx,
                      const void* vy, size_t by, int nrc) {
   VT_CHECK(n % kQK_K == 0, "vec_dot_iq1_s_q8_K: n must be a multiple of 256");

src/vt/cpu/cpu_quant_iq_tables.h

@@ -1,8 +1,16 @@
-// I-quant codebook + sign tables — a 1:1 mirror of llama.cpp @ 237ad9b96
-// `ggml/src/ggml-common.h`:
+// I-quant codebook + sign tables. Every table here but one mirrors llama.cpp
+// @ 237ad9b96 `ggml/src/ggml-common.h` 1:1:
 //   :499 kmask_iq2xs (8)    :503 ksigns_iq2xs (128)
 //   :550 iq2xxs_grid (256)  :1007 iq3xxs_grid (256)
 //   :748 iq2s_grid (1024)   :1116 kvalues_mxfp4 (16)
+//   :1121 IQ1S_DELTA        :1124 iq1s_grid (2048)
+//
+// The exception is `kIq1xxxsGrid` (256), which comes from a DIFFERENT tree:
+// the pinned fork oracle unslothai/llama.cpp @ 36fe8e1cc,
+// `ggml-common.h:2095 iq1_xxxs_grid`, because ggml type 66 is declared by no
+// upstream llama.cpp. See .agents/oracles/llama-cpp-unsloth.md and the note
+// above that table. IQ1_XXXS reuses upstream's own IQ1S_DELTA unchanged, so the
+// one delta constant serves both IQ1 encodings.
 //
 // These are the shared decode tables for the IQ2_XXS / IQ3_XXS / IQ2_S / MXFP4
 // block encodings. They were originally private to cpu_quant_dequant.cpp (the

tests/vllm/test_gguf_dequant.cpp

@@ -2,6 +2,7 @@
 
 #include <cstdint>
 #include <cstring>
+#include <iterator>
 #include <vector>
 
 #include "vllm/model_executor/model_loader/gguf_dequant.h"
@@ -448,3 +449,57 @@ TEST_CASE("DequantGgufRowToF32 rejects unsupported i-quant type") {
   std::vector<uint8_t> b2(136, 0);
   CHECK_THROWS_AS(DequantGgufRowToF32(23, b2.data(), 256), std::runtime_error);
 }
+
+// --- IQ1_S (19) / IQ1_XXXS (66): the two encodings the Qwen3.8-2.4T-A95B
+// checkpoints are 96.92 % made of. `vt::cpu::BlockToFloat` has decoded both
+// since #946, but this expand-to-bf16 path did not list them, so it threw
+// "unsupported ggml type". That is not a corner: `RouteGgufTensor` sends a
+// tensor here whenever the VT_CPU_REF oracle switch is on, keep-quant is off,
+// K is ragged, or the role is not a verbatim one, so the reference lane could
+// not load the target checkpoint at all.
+//
+// Gated on the ORACLE-produced goldens rather than on "does not throw": the
+// inputs are real checkpoint bytes and the expected values come from the pinned
+// upstream and fork ggml themselves. Provenance in
+// tests/vt/iq1_golden_vectors.h.
+#include "../vt/iq1_golden_vectors.h"
+
+namespace {
+
+void CheckGgufDequantAgainstOracle(uint32_t ggml_type, const uint8_t* blocks,
+                                   const uint32_t* golden_bits, size_t n) {
+  const std::vector<float> out =
+      DequantGgufRowToF32(ggml_type, blocks, static_cast<int64_t>(n));
+  REQUIRE(out.size() == n);
+  for (size_t i = 0; i < n; ++i) {
+    CAPTURE(i);
+    uint32_t bits = 0;
+    std::memcpy(&bits, &out[i], sizeof(bits));
+    CHECK(bits == golden_bits[i]);
+  }
+}
+
+}  // namespace
+
+TEST_CASE("DequantGgufRowToF32 IQ1_S row matches the pinned oracle") {
+  CheckGgufDequantAgainstOracle(19, vllm_test::kIq1sGoldenBlocks,
+                                vllm_test::kIq1sGoldenBits,
+                                std::size(vllm_test::kIq1sGoldenBits));
+}
+
+TEST_CASE("DequantGgufRowToF32 IQ1_XXXS row matches the pinned FORK oracle") {
+  CheckGgufDequantAgainstOracle(66, vllm_test::kIq1xxxsGoldenBlocks,
+                                vllm_test::kIq1xxxsGoldenBits,
+                                std::size(vllm_test::kIq1xxxsGoldenBits));
+}
+
+TEST_CASE("DequantGgufRowToF32 IQ3_XXS row matches the pinned oracle") {
+  // Same omission, found by the same review, in the same shared branch, so it
+  // is fixed in the same flow rather than filed and deferred (issue #1023).
+  // IQ3_XXS is the `ffn_down` encoding of the DeepSeek-V4 UD-IQ2_XXS vehicle,
+  // and `vt::cpu::BlockToFloat` has decoded it since that port; only this
+  // expansion path never listed it.
+  CheckGgufDequantAgainstOracle(18, vllm_test::kIq3xxsGoldenBlocks,
+                                vllm_test::kIq3xxsGoldenBits,
+                                std::size(vllm_test::kIq3xxsGoldenBits));
+}