feat(SPEC-DFLASH2): W6 -- DFlash2 drafts token-exact against a beyond-pin vLLM oracle, and the speed axis is owed (#1314) (#1637)

DFlash2 (`DFlash2DraftModel`) now drafts on this engine, token-exact
against a
beyond-pin vLLM oracle. W6 is the final wave of the six-wave port and
the one
that reads the oracle: W1-W5 landed the config surface, the grouped
dynamic
convolution, the candidate selector, the pair-emitting top-k, the path
walk and
the GGUF arm.

WHAT THE GATE MEASURED. G2 compares our draft blocks against a vLLM
capture on
the published `z-lab/Qwen3.8-27B-DFlash2` checkpoint: 4 of 4 prompts
token-exact,
45 of 47 draft blocks byte-identical. G3 reads acceptance per prompt --
49/54/54/52 accepted over 15/10/10/12 blocks -- identical on both
engines. The
two divergent blocks reject the divergent slot on BOTH sides, which is
why `len`
stays in sync and why G3 is recorded as a COROLLARY of G2 rather than an
independent instrument: on this capture it is arithmetically entailed.

WHAT THE ORACLE DID. Running vLLM against itself across its own
attention
backends, the outputs diverge on 1 of 4 prompts and acceptance moves
0.597 vs
0.657. That is upstream disagreeing with itself, recorded because it
bounds what
any single-backend comparison can claim.

THE PIN IS A DATED EXCEPTION. vllm-project/vllm#52816 MERGED 2026-08-21
at
05:27:22Z (merge commit `b389ac29`, head `3406ec1d`). The gate stays
pinned at
`66e5414c` because that is the wheel that executed; re-labelling a
capture to a
head it never ran would be a false pin. #1561 owns moving it.

The `#1538` index row was SUPERSEDED, not corrected:
`.agents/issue-index.md` is
append-only and a landed row is never edited, so the first repair wave
appended
#1561 rather than editing #1538. The spec's `## Upstream chain` table is
the
authority on what that row's superseded text says.

WHAT IS OWED, NAMED RATHER THAN LEFT TO BE FOUND.

- The SPEED axis is NOT TAKEN. No idle-host A/B was reached, our draft
is off
the paged CUDA-graph fast path, and wall clocks were
CIFS-load-dominated. This
  is an open gap, not a ceiling.
- O24: the `at_end` fixture is committed but its `<` to `<=` mutation
was never
run in-suite. An out-of-suite compilation of `ReconstructAcceptance`
shows the
case DOES discriminate (`verified` 1 vs 2), which removes the risk that
the
  fixture proves nothing -- it does NOT discharge the owed run.
- O25 / #1607: the DFlash2 startup notice prints TWICE per load.
`CheckDflash2DraftArm`
is reached from `FromModelDir` and again through the engine
constructor's
speculative-config resolution, and carries no once-flag. No weights are
loaded
  twice; the classification and its paragraph re-run.
- #1562: the capture harness exists only as prose, so the FLASH_ATTN
golden's
backend label is a post-hoc relabel that cannot be re-derived from this
tree.

INHERITED RED. `test_runner`'s "refuses a non-multiple-of-16 block size"
case
fails on this tree and on pristine `origin/main` for x86 CPU builds. It
is #1608,
introduced by #1273 and owned by `BACKEND-ROCM`: `build-test-cpu` was
green at
`483cd319` and red at `e2a9e035`, while `build-test-cpu-arm64` passes at
both.
Not caused here and not repaired here. `test_ltx2_video` (#1559) is
load-sensitive
and passed on the sibling branch at the same pin.

Full ctest on the merged tree: 586 tests, 581 passed, 3 skipped, 2
failed, both
named above.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: ClaudeCode:claude-opus-5 [Claude Code]

---------

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>

Author: localai-bot

.agents/benchmark-record.md

@@ -26067,3 +26067,246 @@ never construct `Qwen3NextAttention`. The `Qwen3.8-27B` rows ran through
 `tools/bench/run_serve_low.py`, which has passed `--language-model-only` since it
 was written. The Qwen3.5-4B GDN prefill kernel row is conv and post-conv timing
 on the LINEAR-attention path, which the full-attention preamble does not touch.
+
+## SPEC-DFLASH2 W6 — the gates, taken against a beyond-pin oracle that had to be made to answer (2026-08-21, `row/SPEC-DFLASH2-W6`, `dgx:gpu0`, #1314 / #1456 / #1538)
+
+**The first time either engine has been asked what the OTHER's DFlash2 draft
+proposed.** Everything before this wave measured mechanisms; this measures the
+two claims the mechanisms were built to make.
+
+### The two engines, pinned
+
+| | ours | oracle |
+|---|---|---|
+| revision | tree `0ac277b3a66b5deabe4871959f0f03566c08deda` (RECONSTRUCTED, see below), base `origin/main` `5702d8f83` | vLLM `0.1.dev1+g66e5414c6` ([vllm#52816](https://github.com/vllm-project/vllm/pull/52816) head `66e5414c6`, **MERGED** `2026-08-21T05:27:22Z` at `3406ec1d`, merge commit `b389ac29`) |
+| artifact | built in-lease, `nvcc` 13.0, `sm_121a`, `CUDA_OBJECTS_BUILT=34` | wheel sha256 `fbc247ab1bda93a81ff7a68658cdda65b697e263ad2c43a2bc62c2591d207439` |
+| attention | our own kernels | `TRITON_ATTN`, read back off the built engine and asserted |
+| graphs | DFlash2 draft runs OFF the paged CUDA-graph fast path | `Capturing dflash2 CUDA graphs (FULL)`, 77 s |
+| concurrency | 1 | `max_num_seqs=1`, `enforce_eager=False` |
+
+**THE OURS-SIDE PIN WAS WRONG AND IS CORRECTED HERE (2026-08-21).** This entry
+originally pinned tree `81b530cff097db493e44e4de9a1c727530ed4467`. That tree is
+the FAILING run's tree: it lacks `Reconstructed::verified`, the per-prompt
+`CHECK(our_recon_here == their_acc)` and three instrument assertions — exactly
++8 executed assertions on this workload — and this entry's own text records the
+pre-fix run as 134 assertions with one failure against the spec's 142/0.
+`134 + 8 = 142`. The two recorded trees differ in exactly ONE compiled file
+(`tests/parity/test_qwen38_dflash2_spec_decode.cpp`; the goldens are data), so
+the passing binary's compiled sources are `81b530cff`'s with blob
+`47c53d17a58584987599028519148747b3f018e9` in that slot, which is tree
+`0ac277b3a66b5deabe4871959f0f03566c08deda`. It is a RECONSTRUCTION: no
+`git write-tree` was taken after the fix, so no recorded object names the passing
+run. An earlier revision of this entry added that "the dispatched mutation counts
+(5 and 37)" were taken on `81b530cff` by the same arithmetic; that clause is
+DELETED, because this entry's own `### Mutations` heading says W6 recorded none,
+`git grep "5 and 37" bb416e0ae` returns nothing, and W6's commit body carries no
+mutation prose — the two counts are unfindable, and the standing statement is
+that **W6 recorded no mutation count anywhere**. `81b530cff` also carries no
+`dflash2_27b` goldens at all, which is CONSISTENT with the run reading one
+through `VLLM_DFLASH2_GOLDEN` off the lease — an inference, not a reading, since
+an untracked golden in the run's worktree fits the same evidence and no log
+survives to separate them. Either way the golden's sha256 is what pins the DATA
+and it matches the committed file byte for byte.
+
+**AND THE ORACLE'S PULL REQUEST HAD ALREADY MERGED WHEN THIS WAS WRITTEN.**
+vllm#52816 merged at `2026-08-21T05:27:22Z`, 46 minutes before the wave's work
+commit. The capture is legitimately pinned to `66e5414c` because that is the
+wheel that ran, and it predates the merge — but the row's gate head is now one
+merge behind vLLM's `main`, which
+[#1561](https://github.com/mudler/vllm.cpp/issues/1561) owns.
+
+Target `Qwen/Qwen3.8-27B` @ `1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0`, 18
+shards, 51.75 GiB, verified against the pinned checkout by `config.json` and
+`model.safetensors.index.json` sha256 rather than by directory name. Draft
+`z-lab/Qwen3.8-27B-DFlash2` @ `50307d4c4cde6860d4eee73e2547cd786fe8e8a4`,
+sha256 `67fc76d68dc5a9415511a4f394ef744d67510cd20e93b37cc2cc7d28e4bab65c`
+recomputed from the staged copy. k=7, greedy, `max_tokens` 64, the four
+`SPEC-DFLASH` D5 prompts.
+
+### G2 — 4/4 token-exact, 45/47 draft blocks byte-identical
+
+| prompt | tokens | shared | draft blocks identical |
+|---|---|---:|---:|
+| `The capital of France is` | exact | 64/64 | 14/15 |
+| `def fibonacci(n):` | exact | 64/64 | 10/10 |
+| `Q: What is 17 * 23?\nA:` | exact | 64/64 | 10/10 |
+| `The three laws of robotics are` | exact | 64/64 | 11/12 |
+
+The two blocks that differ each differ by ONE token at slot 2 and then emit the
+same target tokens: `[14227 369 14227 13 198 760 6511]` against
+`[14227 369 24844 ...]`, and `[39262 279 9861 2574 314 539 279]` against
+`[39262 279 10895 ...]`.
+
+**The attribution to "the selector's rank contraction" is WITHDRAWN**
+([#1564](https://github.com/mudler/vllm.cpp/issues/1564)). Nothing here measured
+which op moved: the golden records `{call, req_row, anchor, drafts}` and no
+values, no logits and no top-2 gap. The shape argues against the op that was
+named — in BOTH blocks only slot 2 changes while slots 3-6 are byte-identical,
+and `src/vt/cpu/cpu_ops.cpp:3219` has the walk read the predecessor row the
+previous step chose, so a flipped child index would move four later slots and
+did not. A rank swap at the same winning slot in `ComputeCandidates`' top-k
+explains it with no coincidences. Neither is measured; the next capture records
+the top-2 candidate margin on both sides, which both engines already compute.
+
+### G3 — acceptance IDENTICAL, same-trajectory — and a COROLLARY of G2, not a second measurement
+
+All four prompts produced the same token stream on both engines, so there is no
+trajectory to confound. "By construction" means by ADMISSION and not by teacher
+forcing: both engines run FREE and a prompt is admitted only when the two
+independently emitted the same stream.
+
+**The per-prompt equality below is very nearly ENTAILED by G2's result.**
+Established 2026-08-21 by recomputing the oracle side from the committed golden:
+outputs identical on all four prompts, 45 of 47 blocks byte-identical (so the
+reconstruction runs on identical inputs for 45 of them), and BOTH divergent
+blocks have their divergent slot rejected on both sides — record 0 block 13's
+output slot 2 is `31785`, neither draft's candidate, so both accept 2; record 3
+block 6 has `279` at slot 1 in both drafts against an output of `9861`, so both
+accept 1. Per-prompt equality follows arithmetically, and so do 216-vs-216 and
+the 7-token truncation deficit. The result is a real consistency check on the
+reconstruction and it is NOT withdrawn — had either flipped slot been accepted on
+one side the counts would have parted — but it must not be quoted as an
+independent second gate reading.
+
+| prompt | ours | oracle |
+|---|---:|---:|
+| 0 | 49 | 49 |
+| 1 | 54 | 54 |
+| 2 | 54 | 54 |
+| 3 | 52 | 52 |
+| total | **209** | **209** |
+
+Counted the same way on both sides, from drafts and output. On the other
+instrument they also agree: our runner's counter reads 216 and
+`vllm:spec_decode_num_accepted_tokens` reads 216; our reconstruction verifies 47
+blocks and `vllm:spec_decode_num_drafts` reads 47.
+
+**Both of those were PRINTED and neither was ASSERTED in the run this entry
+records.** `our_acc_sum` was a `MESSAGE` only, our verified block count was not
+computed at all, and the "47" the gate printed was `draft_blocks_compared` —
+bounded by the shared-prefix cut, so a count of blocks the gate could PAIR rather
+than blocks this engine verified. What the run DID assert is the oracle's
+reconstruction against vLLM's two counters and the per-prompt
+reconstruction-vs-reconstruction equality. Both are asserted now, guarded on
+`same_traj == total` because vLLM's counters are pooled, and NEITHER new
+assertion has yet executed on a device. The 209/216 gap is
+`max_tokens` truncating the last block of three of four requests, it appears on
+BOTH engines, and mixing the two instruments is the D8 shape in miniature.
+
+### G4 / O17 — a published artifact LOADED, and O13 measured at runtime
+
+`examples/vllm-cli` was pointed at `Qwen3.8-27B-DFlash2-Q4_K_M.gguf`
+(1 143 006 752 B, sha256 `18a380ef...`) over the real 27B target. It loaded,
+proposed 7 speculative blocks and generated 24 tokens.
+
+| arm | on disk | peak RSS |
+|---|---:|---:|
+| Q4_K_M GGUF | 1.06 GiB | 47 028 616 KB (44.85 GiB) |
+| safetensors | 3.58 GiB | 46 701 608 KB (44.54 GiB) |
+
+0.70% apart on files 2.52 GiB apart, and the QUANTIZED arm is the LARGER by
+319 MiB. The two arms propose DIFFERENT drafts and emit the SAME tokens.
+
+### The oracle's own backend A/B, which reprices what G2 may demand
+
+Same wheel, same host, same workload, FULL decode graphs on both; only the
+attention backend differs.
+
+| | FLASH_ATTN | TRITON_ATTN |
+|---|---:|---:|
+| `spec_decode_num_drafts` | 50 | 47 |
+| `spec_decode_num_draft_tokens` | 350 | 329 |
+| `spec_decode_num_accepted_tokens` | 209 | 216 |
+| accepted / drafted | 0.597 | 0.657 |
+
+**Identical output on 3 of 4 prompts**; `def fibonacci(n):` diverges at
+generated index 4. vLLM does not reproduce its own greedy continuation across
+two of its own backends on this model, so a 4-of-4 strict bar would be one vLLM
+fails against itself.
+
+**AND THAT FLASH_ATTN ARM'S BACKEND LABEL IS A POST-HOC RELABEL**
+([#1562](https://github.com/mudler/vllm.cpp/issues/1562)). Its golden carries
+`attention_backend_source: "corrected from the run log by w6-relabel.py"`, not a
+value read back off the built engine, which is the rule `## Owed` O22 itself lays
+down. Neither the log nor the script is committed, so the label is unauditable.
+The TRITON_ATTN arm's label WAS read back and is unaffected. That golden also
+carries NO per-block drafts on any record, so it can answer output identity and
+cannot answer draft identity or acceptance; the gate now reports that as VOID
+rather than as a structural finding about our drafts.
+
+### FLASH_ATTN is USABLE here, which #1456 concluded it was not
+
+The first capture exported `VLLM_ATTENTION_BACKEND=TRITON_ATTN`, vLLM ignored it
+because that variable does not exist at this revision, auto-selection took
+`FLASH_ATTN`, and the run worked end to end -- 54.87 GiB loaded, graphs captured,
+256 coherent tokens, speculation live. #1456's SASS measurement stands;
+`sm_80` PTX JITs forward and `cudaErrorUnsupportedPtxVersion` is the opposite
+failure. Posted to #1456.
+
+### SPEED — NOT TAKEN
+
+No ratio is claimed and none should be inferred. No idle-host same-binary A/B was
+run. Our DFlash2 draft is off the paged CUDA-graph fast path while the oracle
+graphs its draft step. And every wall-clock here was dominated by reading a
+51.75 GiB checkpoint over CIFS: 11:17 and 12:15 for 24 tokens, `tok_s` 0.115 and
+0.090, which measure loading and are recorded so nobody quotes them as decode.
+
+### The device arms, on hardware, zero skips
+
+`test_ops_dflash2_grouped_conv` 9936 assertions, `test_ops_dflash2_selector_edges`
+3859, `test_ops_topk_values_indices` 560, `test_ops_dflash2_path_walk` 83,
+`test_qwen3_dflash2_draft` 277, `test_dflash2_runner_reach` 86,
+`test_dflash2_argmax_guard` 30. All `Status: SUCCESS!`, all
+`CUDA_SKIP_LINES=0`, `DEVICE_SUITES_FAILED=0`.
+
+### Four instrument defects, three in the oracle hook and one in our own gate
+
+Recorded in the spec's `## Owed` O23. The pattern is the finding: each presents
+as a verdict about the CODE. The engine core is a separate process by default;
+`capture_model()` calls the hooked method inside a CUDA graph capture; and
+`_generate_draft` is the wrong seam entirely because `propose` replays the
+captured graph. The third asserted `InprocClient`, `HOOK_ON_CLASS=traced` and
+the resolved backend, and was still blind. Only the abort-on-zero caught them.
+The fourth was ours: the gate compared 55 raw propose calls against vLLM's 47
+drafts, which counts only blocks starting inside the output. One assertion of
+134 failed on it.
+
+### Mutations
+
+W6 recorded none. Taken 2026-08-21 by the W6 repair wave on the CPU dev box at
+the merged tree, each with its match count, `git diff --stat` and compile rc
+printed, each restored sha256-verified and rebuilt. Unmutated the suite reads
+**4 cases / 65 assertions / 0 failed / `Status: SUCCESS!` / rc 0** there (the
+e2e case SKIPs without a checkpoint); before this wave it read 3 cases / 41, and
+after the SECOND repair wave it reads 4 cases / 70.
+
+| mutation | result |
+|---|---|
+| `ListField` drops the last id of every list | 1 case / 7 of 65 red, rc 1 |
+| `len += 1 + acc` becomes `len += acc` | 1 case / 7 of 65 red, rc 1 |
+| the `len + j >= out.size()` truncation guard deleted | **SIGSEGV, rc 139** |
+
+**SECOND repair wave (2026-08-21).** Unmutated `test_dflash2_runner_reach` reads
+3 cases / 90 assertions and `test_qwen38_dflash2_spec_decode` 4 cases / 70, both
+`SUCCESS!` / rc 0.
+
+| mutation | result |
+|---|---|
+| **M1** — the DFlash2 startup notice reverted to its pre-repair `"is OPEN upstream at head 66e5414c"` text (match 1, `+3/-5`, compile rc 0) | 1 case / **4 of 90** red, `Status: FAILURE!`, rc 1 — the four new merged-state assertions and nothing else |
+| **M2** — `len < out.size()` -> `len <= out.size()` on `Reconstructed::verified` | **NOT TAKEN.** Attempted and VOIDED by a harness race: two instances ran concurrently and the second took its baseline after the first had mutated, so it read `match count: 0` — its "before" hash `770bee0a` is the MUTATED file, against the clean `843d610b`. Tree verified undamaged (both files byte-identical to `HEAD`); harness now takes a `flock`. A clean retake was unaffordable because every cycle rebuilds the whole 464-object library and the box ran at loadavg 145 / 12 objects per 10 min. The `at_end` boundary fixture IS committed and is UNPROVEN; taking M2 on a quiet box is owed |
+| liveness by TOTAL block count instead of per record | 1 case / 1 red |
+| every golden declared live | 1 case / 2 red |
+| `with_blocks` counts drafts-less records too | 1 case / 4 red |
+| the `hook_stats` residual claimed 0 instead of 3 | 1 case / 1 red |
+
+**The third one is why a doctest assertion line is not a verdict.** It prints
+`assertions: 64 | 64 passed | 0 failed` — one FEWER than green and all passing —
+while the case failed and the process died on a signal. `Status:` and the exit
+code bind; the assertions line reads like a pass.
+
+NOT mutated, and named rather than assumed: the e2e case's own `gd.live` guards
+and its two new ours-vs-theirs assertions. That case is dgx-only and SKIPs here,
+so no mutation of it — and no production-call-site deletion — can be executed on
+this box. The always-on case exists so the liveness RULE is gated where it runs;
+the WIRING is owed a device run.
+

.agents/claims/CLAIM-SPEC-DFLASH2-W6.md

@@ -0,0 +1,5 @@
+# CLAIM-SPEC-DFLASH2-W6
+
+| Claim | Row IDs | Agent | Worktree / remote dir | Branch | Owned scope | State | Last update |
+|---|---|---|---|---|---|---|---|
+| `CLAIM-SPEC-DFLASH2-W6` | `SPEC-DFLASH2` (`ACTIVE`) | Claude Code (opus-5), helper role — W6 took the gates; a FRESH IMPLEMENTER, who wrote none of W6, repaired the fresh review's findings | W6: `dgx:gpu0` through an `rc` lease, wheel built in-lease at `66e5414c`, `nvcc` 13.0, `sm_121a`. Repair wave: isolated worktree, CPU only, no lease, no GPU, no oracle run, no checkpoint | `row/SPEC-DFLASH2-W6`, issue [#1314](https://github.com/mudler/vllm.cpp/issues/1314) | Owns ONLY wave W6 of the spec's `## Work breakdown`: reading `## Gates` G2 (draft-token identity) and G3 (acceptance, same-trajectory) against the beyond-pin oracle on a leased GPU, and the records that result invalidates. That is `tests/parity/test_qwen38_dflash2_spec_decode.cpp` and the two committed goldens under `tests/parity/goldens/dflash2_27b/`; the `SPEC-DFLASH2` engine-matrix row; `docs/STATUS.md`, `docs/BENCHMARKS.md`, `docs/FEATURES.md` and `docs/SPECULATIVE-DECODING.md`; the spec's `## Upstream chain`, `## Gates`, `## Oracle`, `## Dependencies`, `## Work breakdown`, `## Owed` and `## Now`; and one `.agents/benchmark-record.md` entry. EXCLUDES any throughput ratio, which `## Gates` defers and which no wave has taken. EXCLUDES reconciling the port onto merged upstream ([#1561](https://github.com/mudler/vllm.cpp/issues/1561)), which the repair wave opened and did NOT do. EXCLUDES the probabilistic draft-sample arm (`## Owed` O12) and the CUDA top-k NaN ordering ([#1489](https://github.com/mudler/vllm.cpp/issues/1489)). EXCLUDES any parity-pin advance: the oracle is BEYOND-PIN and `.agents/upstream-sync.md`'s pin block is untouched | `ACTIVE` | 2026-08-21 — W6 read G2 (4/4 prompts token-exact, 45/47 draft blocks byte-identical) and G3 (acceptance identical per prompt, 209 both engines) on `dgx:gpu0`, and G4/O17 end to end on a published GGUF artifact. A fresh review returned FAIL on one HIGH and six MEDIUM findings; a fresh implementer repaired them on the same branch. HIGH: vllm#52816 had MERGED 46 minutes before W6's work commit and five statements plus the gate-head rule said otherwise — corrected, the head kept as a DATED exception because the capture predates the merge, and [#1561](https://github.com/mudler/vllm.cpp/issues/1561) opened for the reconciliation. MEDIUM: a golden with zero drafts passed the liveness precondition and produced a structural verdict about OUR engine — repaired with `InspectGoldenDrafts`, per record rather than by total, VOID rather than failed, and gated on every box by a new checkpoint-free case; G3 was declared teacher-forced and is an admission condition; G3's headline is a COROLLARY of G2 on this capture, verified by recomputing both divergent blocks; the near-tie attribution named an op the block shape argues against and is WITHDRAWN ([#1564](https://github.com/mudler/vllm.cpp/issues/1564)); the pinned tree was the FAILING run's and the passing tree is reconstructed as `0ac277b3`; and W6 recorded no mutations anywhere — seven are recorded now, each with match count, compile rc and sha256-verified restore. The oracle harness is prose only and is owed ([#1562](https://github.com/mudler/vllm.cpp/issues/1562)) |