diff --git a/.agents/issue-index.md b/.agents/issue-index.md index 1897927cf..3b529d2b3 100644 --- a/.agents/issue-index.md +++ b/.agents/issue-index.md @@ -267,7 +267,6 @@ rather than merged. `scripts/check-agent-record.py` gates both. | [#940](https://github.com/mudler/vllm.cpp/issues/940) | `MODEL-TEXT-nemotron-h-nemotron-hfor-causal-lm` | The FP8 W8A8 linear path is not a shared seam: `ResidentFp8`, `MatmulFp8CutlassD` and `MatmulFp8CutlassPreQuantD` lived in the anonymous namespace of `src/vllm/model_executor/models/qwen3_5.cpp` (`:1458`, `:1495`, `:1517` @ `c7cb59fbb`), so a second model could reach them only by copying them — the hand-rolled parallel path AGENTS.md §"Shared seams" forbids. NVFP4 already had `dense_nvfp4_gemm.h` + `compressed_tensors/schemes/nvfp4.h`; FP8 had neither half. Forced by `MODEL-NEMOTRON-H` ([#517](https://github.com/mudler/vllm.cpp/issues/517)), whose 46 FP8 W8A8 mamba `in_proj`/`out_proj` projections are 36.6% of decode bytes and 27.6% of GEMM FLOPs, and whose `in_proj` produces the fused `zxbcdt` the conv and the SSD scan consume — so that block cannot be split and has no device path at all without the seam. Extracted to `dense_fp8_gemm.h` + `quantization/fp8.h` with Qwen3.5 byte-identity as the gate; spec [`vt-fp8-shared-seam.md`](specs/vt-fp8-shared-seam.md) | bug | | [#974](https://github.com/mudler/vllm.cpp/issues/974) | — | The FP8 W8A8 resident helpers move weight bytes host->device without `vllm::load_stats::AddDeviceUpload` and without the post-upload `AdoptDeviceBytesAsHost`, while every other resident-weight helper in the same file performs both: `ResidentWeight` (`src/vllm/model_executor/models/qwen3_5.cpp:1009,1016 @ c7cb59fbb`) and `ResidentNvfp4` (`:1106,1111,1116,1121`), whose own comment states the obligation against [#150](https://github.com/mudler/vllm.cpp/issues/150). Affects `ResidentFp8` (`:1458`, now `dense_fp8_gemm.h`), `ResidentFp8Qkv` (`:1555`) and `ResidentFp8Qkvz` (`:3440`). Two unmeasured consequences: load accounting is short by the whole fp8 tower, and its device pages are never re-tagged, which is the shape of the GB10 weight-residency penalty on the 27B decode gap's largest attributed bucket. Found while extracting those entry points in [#940](https://github.com/mudler/vllm.cpp/issues/940) and deliberately NOT fixed there: a byte-identity gate cannot see a device -behaviour change hidden in a move. Listed under `## Owed` in [`vt-fp8-shared-seam.md`](specs/vt-fp8-shared-seam.md) | bug | | [#986](https://github.com/mudler/vllm.cpp/issues/986) | `LTX25-DFR-PIPELINE` | `DFRPipeline` (`dfr_pipeline.py` + `dfr_layout.py` @ `fd4ded7f`) has no representation in this tree — `git grep -i dfr` over `src include tests examples docs` returns ZERO product hits, against 87 `ltx2` hits in `ltx2.cpp` alone as the positive control ([#604](https://github.com/mudler/vllm.cpp/issues/604)). It matters beyond its own feature because it is the **ONLY** upstream consumer of the temporal x2 latent upsampler this project already ships: [`ltx25-temporal-upsampler.md`](specs/ltx25-temporal-upsampler.md) section 7 records the operator as ported, loader-parsed and gated but *"not reachable from any shipped pipeline"*, and `docs/FEATURES.md` carries that as `Temporal x2 ups gated, UNDRIVEN`. No issue tracked that state — `undriven` returned zero hits across open and closed issues. DFR also needs the generated-keyframe-slot READBACK that [#920](https://github.com/mudler/vllm.cpp/issues/920) refused by name and left owed after it CLOSED, so the debt had a spec bullet and no open issue; it needs the LAYOUT and the EXTRACTION but **not** the standalone single-frame decode, because DFR never decodes its slots — it hands them to the spatial upsampler (`dfr_pipeline.py:348`) and feeds them back as `initial_keyframes` (`:364`). Spec [`ltx25-dfr-pipeline.md`](specs/ltx25-dfr-pipeline.md). Campaign [#644](https://github.com/mudler/vllm.cpp/issues/644) | feature | -| [#995](https://github.com/mudler/vllm.cpp/issues/995) | `ENG-EXPERT-STREAM` | `check-env-doc` and `test_check_env_doc` RED in `scripts/agent-preflight.sh` on **every** branch based on `3005447f8`: `VT_MOE_EXPERT_STREAM`, `VT_MOE_EXPERT_STREAM_SLOTS` and `VT_MOE_EXPERT_STREAM_SLOT_BYTES` are read from `src/vllm/model_executor/models/qwen3_5.cpp` (`:5145`, `:5164` @ `3005447f8`) and appear in neither `docs/ENVIRONMENT.md` (`grep -c` returns **0**) nor `scripts/env-doc-allowlist.txt`. Arrived with [#993](https://github.com/mudler/vllm.cpp/pull/993). Found while gating [#986](https://github.com/mudler/vllm.cpp/issues/986) and proved pre-existing with a matched-arm check rather than asserted: the three sites are in a file that branch does not touch, and `git diff origin/main...HEAD | grep '^+.*VT_MOE_EXPERT'` returns nothing. NOT fixed in flow: documenting a knob means stating its default and when to touch it, and the expert-streamer's slot accounting belongs to the row that added it, so a plausible-sounding entry written by a passer-by is how `docs/ENVIRONMENT.md` stops being trustworthy. It is a PRE-FLIGHT gate, so it fails before every edit and presents to each author as a red their own diff caused, which is the shape [#965](https://github.com/mudler/vllm.cpp/issues/965) and [#968](https://github.com/mudler/vllm.cpp/issues/968) both took | bug | +| [#995](https://github.com/mudler/vllm.cpp/issues/995) | `ENG-EXPERT-STREAM` | `check-env-doc` and `test_check_env_doc` were RED on `origin/main`, so every branch cut from it inherited a preflight failure its own diff did not cause: `VT_MOE_EXPERT_STREAM`, `VT_MOE_EXPERT_STREAM_SLOTS` and `VT_MOE_EXPERT_STREAM_SLOT_BYTES` are read from `src/vllm/model_executor/models/qwen3_5.cpp` (`:5145`, `:5195`, `:5189` @ `4496ef196`) and appeared in neither `docs/ENVIRONMENT.md` (`grep -c` returned **0**) nor `scripts/env-doc-allowlist.txt`. They arrived with the `ENG-EXPERT-STREAM` W4 wiring commit `3005447f8` ([#993](https://github.com/mudler/vllm.cpp/pull/993)). Found while gating [#986](https://github.com/mudler/vllm.cpp/issues/986) and proved pre-existing with a matched-arm check rather than asserted: the three sites are in a file that branch does not touch, and `git diff origin/main...HEAD \| grep '^+.*VT_MOE_EXPERT'` returns nothing. It is a PRE-FLIGHT gate, so it failed before every edit and presented to each author in turn as a red their own diff caused -- the shape [#965](https://github.com/mudler/vllm.cpp/issues/965) and [#968](https://github.com/mudler/vllm.cpp/issues/968) both took. The discoverer deliberately did NOT fix it in flow: documenting a knob means stating its default and when to touch it, and the expert-streamer's slot accounting belongs to the row that added it, so a plausible-sounding entry written by a passer-by is how `docs/ENVIRONMENT.md` stops being trustworthy. FIXED in [#997](https://github.com/mudler/vllm.cpp/pull/997) by DOCUMENTING all three in `docs/ENVIRONMENT.md`, not by allowlisting them: a knob that changes the host memory a MoE decode reserves and disables the default-on grouped-MoE path is a deployment surface, not a kernel-internal micro-tuning switch | bug | | [#987](https://github.com/mudler/vllm.cpp/issues/987) | `LTX25-RETAKE` | Two `ltx-2.5` refusal messages state reasons that are no longer true. (a) `src/vllm/multimodal/ltx2_video.cpp:1608 @ 0e1bee42f` says "nothing reads `ref_video_dir` at all", and MiniMax-H3 has always consumed the directory in full (`ReadReferenceClipChw`, `src/vllm/multimodal/minimax_h3_video.cpp:135 @ 0e1bee42f`, called at `:650`); [#975](https://github.com/mudler/vllm.cpp/issues/975) inherited the wider claim from this message. The claim that holds is narrower: the LTX-2.5 engine never reads the directory's CONTENTS. (b) `ltx2_video.cpp:1636-1638 @ 0e1bee42f` says "there is no AUDIO_VAE_ENCODER key filter", and `c2019b0e3` landed `Ltx2AudioVaeEncoderKeyRules()` (`include/vllm/model_executor/models/ltx2_audio_input.h:73 @ 0e1bee42f`) with a live call through `Ltx2EncodeAudioToLatent`. Both rewritten in the `WHAT IS *NOT* THE REASON` shape in the same flow, with one assertion tied to the LOCAL fact that the LTX side now reads the directory | bug | -| [#995](https://github.com/mudler/vllm.cpp/issues/995) | `ENG-EXPERT-STREAM` | `check-env-doc` and `test_check_env_doc` are RED on `origin/main`, so every branch cut from it inherits a preflight failure its own diff did not cause: `VT_MOE_EXPERT_STREAM`, `VT_MOE_EXPERT_STREAM_SLOTS` and `VT_MOE_EXPERT_STREAM_SLOT_BYTES` are read from `src/vllm/model_executor/models/qwen3_5.cpp` (`:5145`, `:5195`, `:5189` @ `4496ef196`) and appear in neither `docs/ENVIRONMENT.md` nor `scripts/env-doc-allowlist.txt`. They arrived with the `ENG-EXPERT-STREAM` W4 wiring commit `3005447f8` ([#993](https://github.com/mudler/vllm.cpp/pull/993)). Same shape as [#965](https://github.com/mudler/vllm.cpp/issues/965)/[#968](https://github.com/mudler/vllm.cpp/issues/968): a pre-existing red presenting to each author in turn as their own. Fixed by DOCUMENTING all three in `docs/ENVIRONMENT.md`, not by allowlisting them — a knob that changes the host memory a MoE decode reserves and disables the default-on grouped-MoE path is a deployment surface, not a kernel-internal micro-tuning switch | bug | [#1023](https://github.com/mudler/vllm.cpp/issues/1023) | `ENG-EXPERT-STREAM` | The IQ1_S (ggml 19) / IQ1_XXXS (ggml 66) decode landed in [#946](https://github.com/mudler/vllm.cpp/pull/946) with every parameter EXCEPT the grid pinned only by self-consistency. The grid seal (FNV-1a digest + lane census) works and stops one table short: `ReferenceDotF64` (`tests/vt/test_ops_quant_dot.cpp:231-236`) and the G3 NMSE reference (`:915`) both decode the weight with `vt::cpu::BlockToFloat`, the function under test, so they are independent only in the SUMMATION. Three injected defects, each applied and compiled, left the suite green with an UNCHANGED assertion count: `kIq1sDelta` `0.125F`->`0.25F` (`cpu_quant_iq_tables.h:422`, affects BOTH encodings), the IQ1_S delta sign inverted in dequant AND vec_dot, and the IQ1_S scale read from `qh` bits 13-15 instead of 12-14 in both paths. Second defect, same PR: `gguf_dequant.cpp:107-116` lists no `case 19` and no `case 66`, so the expansion path throws `unsupported ggml type` for the two encodings the target checkpoints are 96.92 % made of, and `RouteGgufTensor` (`gguf_keep_quant.cpp:122`) sends a tensor there whenever `VT_CPU_REF` is on, keep-quant is off, K is ragged, or the role is not verbatim — a refusal to load on the reference lane. **ggml 18 (IQ3_XXS) carried the same omission**, pre-existing since the DeepSeek-V4 UD-IQ2_XXS port, and all three are one shared `switch` branch. Repaired by golden vectors whose EXPECTED values come from the ORACLES themselves (`ggml_get_type_traits(type)->to_float` built from `ggml-org/llama.cpp @ 237ad9b96` for 18/19 and `unslothai/llama.cpp @ 36fe8e1cc` for 66) over REAL checkpoint bytes, which is the first reference for this encoding family that is not this tree's own decoder. Also: the `2e-3` NMSE ceiling passed the doubled-delta defect that `6e-4` fails (measured 5.240e-4 unmutated, 6.967e-4 mutated on `iq1_s`), and on `iq1_xxxs` that defect moves the statistic the WRONG WAY (3.109e-4 -> 1.420e-4), so no ceiling can catch it and only the goldens can; and both new census cases claimed TOTAL coverage of 1702 records while summing to 864, F32's 838 tensors omitted. Spec [`expert-streaming.md`](specs/expert-streaming.md) | bug |