Track upstream spec-decode and sparse-MLA performance stack

[randomvariable]

Objective

Track and consume the active upstream spec-decode and sparse-MLA performance stack relevant to the fork.

References

• Kernel warmup: [Spec Decode][V1] Warm Eagle and DFlash/DSpark spec-decode Triton kernels at startup vllm-project/vllm#48804
• fp8_ds_mla sparse mixed-batch + MTP: [Attention][MLA] FlashMLA sparse: DCP on the fp8_ds_mla mixed-batch path + MTP (stacked on #46076) vllm-project/vllm#46514
• Fused AR speculator CUDA graph: [MRV2][Spec] Fuse AR speculator multi-step decodes back into one CUDA graph vllm-project/vllm#46849
• Dense short-prefill sparse-indexer skip (merged): [Attention] Skip sparse indexer scoring for dense short prefills vllm-project/vllm#48407

Proposed work

• Track dependency/rebase state and port only stable, separable changes.
• Verify merged [Attention] Skip sparse indexer scoring for dense short prefills vllm-project/vllm#48407 is present after upstream convergence.
• Measure cold-start first-request latency before/after kernel warmup.
• Measure draft-step launch overhead and graph replay for fused AR speculation.
• Validate fp8_ds_mla mixed-batch/MTP numerics and decode throughput on current sparse-MLA ABI.

Acceptance criteria

• Each imported change is tied to a focused regression or benchmark.
• Cold and warm measurements are reported separately.
• No runtime JIT fallback remains where AOT/warmup is expected.
• For each imported change, define model/context/batch matrix, numerical tolerance, graph-replay assertion, named reference backend, and allowed regression before default enablement.

Target hardware

• CUDA sm_121a (GB10 / DGX Spark)
• ROCm gfx1151 (Strix Halo) — sparse-MLA indexer and spec-decode must work on both targets; gfx1151 uses AITER sparse attention path

[randomvariable]

Current-model applicability and capability split

Track each upstream capability independently:

1. speculative-kernel warmup for the exact EAGLE, DFlash, DSpark, or native-MTP implementation;
2. fused autoregressive-speculator graph replay;
3. fp8_ds_mla sparse mixed-batch and MTP support;
4. dense short-prefill sparse-indexer bypass.

A model is eligible only for tracks matching its drafter type, attention layout, quantization, and selected backend. Success in one track does not establish support for another.

DeepSeek-V4-Flash and GLM-5.2 are sparse/compressed-MLA candidates. Qwen3.6, GLM-5.2, Nemotron 3 Ultra, Hy3, and Step-3.7-Flash are native-MTP architecture candidates. These sets overlap only where the actual vLLM implementation exposes both required paths.

Status correction: upstream PR vllm-project#48407 merged on 2026-07-28, so treat it as a convergence/regression check rather than pending port work. PRs vllm-project#46514, vllm-project#46849, and vllm-project#48804 remain open.

[randomvariable]

Additional capability rows

Add Step-3.7-Flash to native-MTP warmup/graph experiments, but keep it separate from sparse-MLA tracks. Add Mistral Small 4 only to MLA tracks after proving the implementation accepts its low-rank attention representation. Neither architecture match connects all four capabilities into one supported stack.

[randomvariable]

Status: partial.

a9b1ee711 adds EAGLE/DFlash speculative-decode kernel warmup; 2a6c161ff repairs the integration conflict. Unit coverage in tests/model_executor/test_jit_warmup.py verifies warmup key enumeration and input preparation. #20 still tracks cold-versus-warm latency measurements, AR graph replay, fp8_ds_mla mixed-batch/MTP numerics and throughput, and confirmation of the remaining upstream stack. Next gate: report cold/warm first-request latency and graph-replay evidence on a named model/context/batch matrix.