ChunkedSmallT re-streams full KV per query chunk (DFlash K>=8 verify at long context)

[giveen]

ChunkedSmallT re-streams full KV once per query chunk (DFlash K≥8 verify at long context)

Finding

Above the prompt route limit, ChunkedSmallT partitions the query axis (launch_chunked_small_t, src/ops/softmax_attention/dense/causal_cache/causal_softmax_attention.cpp) while every chunk sweeps the full visible key range. Key-splitting (causal_attention_split_capacity + reduce kernel) already happens inside each chunk, so total KV sweeps = ceil(W / chunk):

W (q24/bf16)	chunk	KV sweeps
≤8	— (SmallT)	1
9–10	5	2
11–12	6	2
13–16	8	2

(W=16 → 3 sweeps only on q16 models.) Plain decode (W=1) and MTP (W≤6 → SmallT) are unaffected; on q24 this bites DFlash verify with K≥8 at long context, where KV bytes dominate. Current published configs (DFlash/DFlash2 K=7 → W=8) never trigger it.

Measurement (ninfer_causal_softmax_attention_bench, q24, batch 1, cold cache)

context	W=8	W=16	ratio
16K bf16	77µs @ 868 GB/s	135µs @ 497 GB/s	1.75× (L2 absorbs part)
64K bf16	201µs @ 1338 GB/s	393µs @ 683 GB/s	1.96×
16K bf16 fragmented	78µs	137µs	1.76×
16K nvfp4	47µs	88µs	1.87×

At 64K that's ~190µs wasted per full-attention layer per verify step (16 such layers on qwen3_6_27b). Could not go end-to-end: no local artifact carries a DFlash draft component, so acceptance-vs-K at long context is unmeasured.

Why no PR yet

• The "return width" stopgap is invalid: causal_attention_split_capacity throws for tokens > 8 (q24) / > 6 (q16) — no TokenTile-12/16 kernel exists. A real single-sweep fix needs new kernel instantiations + op-dev oracle qualification.
• L2 persistence for KV has no in-tree infra (greenfield) and only pays off at ≤32K contexts.
• An adaptive K-cap needs drafter-owner acceptance data at 32–64K.

Proposal

1. Note the W≥9 cliff wherever the DFlash draft window is configured (guardrail so a future K bump doesn't silently double attention traffic).
2. Decide with the speculation owner: adaptive K-cap at long context vs TokenTile-12/16 kernel work, backed by an acceptance-vs-K measurement on a DFlash-capable artifact.