feat: add DeepSeek-V4 Flash DSpark node perf recipe + required mods

[vedcsolution]

Adds:

• fix-dspark-dsv4-d2t: guards speculator.py:86 against DeepSeek-V4 (no draft_id_to_target_id attribute — full-vocab Markov drafting, no reduced-vocab scatter path). Upstream: [Bugfix][Spec Decode] Add missing draft_id_to_target_id to DSparkDeepseekV4ForCausalLM vllm-project/vllm#47429.
• add-dsv4-topk256: sparse-MLA SM120 TOPK=256 decode instantiations for DSpark. Upstream: fix(sparse-mla-sm120): add TOPK=256 decode-dsv4 instantiation for DeepSeek-V4 DSpark flashinfer-ai/flashinfer#3817.
• add-dsv4-topk256-prefill: sparse-MLA SM120 TOPK=256 BF16 prefill instantiation. Upstream: fix(sparse-mla-sm120): add TOPK=256 DSV4 prefill instantiation flashinfer-ai/flashinfer#3834 (supersedes closed #3896).

SM12x DeepSeek-V4 DSpark requires both FlashInfer TOPK=256 halves. Carry and remove these two mods together, and remove them only after the FlashInfer version pinned by the image contains both #3817 and #3834.

• deepseek-v4-flash-dspark-node-perf.yaml: perf pass 1 recipe for dual DGX Spark TP=2, vllm-node image. Safe generic flags, memory-conservative for GB10 121GB unified memory.

See recipe YAML for full flag rationale.

[ormandj]

Dependency correction: SM12x DeepSeek-V4 DSpark needs both FlashInfer TOPK=256 halves—decode #3817 and prefill #3834. Carrying only #3817 leaves the prefill instantiation missing. #3896 was superseded by #3834, so the carry and removal note should name #3817 + #3834 and remove both only after a pinned FlashInfer release contains them.

[vedcsolution]

Thanks @ormandj — corrected in a2424eb.

The carry is now split and paired explicitly:

• decode: FlashInfer #3817 (mods/add-dsv4-topk256)
• prefill: FlashInfer #3834 (mods/add-dsv4-topk256-prefill, BF16 TOPK=256)

The recipe and PR removal note now require the FlashInfer version pinned by the image to contain both changes before either mod is removed, and note that #3896 was superseded by #3834.

I also validated the pair end-to-end on 2x DGX Spark GB10 (SM121), .121 head + .195 worker, using the pinned vllm-node image: both mods apply idempotently on both nodes, the live sources contain the full decode grid and BF16 prefill branch, the service reached /health 200, a 564-token prefill + 96-token decode and four concurrent 272+48 token requests completed, and both node logs remained clear of missing-instantiation / num_tokens > 64 failures. The 1 GiB RAM+swap watchdogs did not trigger.

[sqrlmstr5000]

Confirmed working on 2x DGX Sparks!

Green is with the current deepseek recipe, Yellow is with this recipe. I'm using the model through VSCode. Results are pretty impressive!

[sqrlmstr5000]

@vedcsolution think we'll be able to use this as a base for the 0731 release?

EDIT: The answer is yes, you can :)

I think this may have helped.

version 0.25.2.dev0+g752a3a504.d20260730
model deepseek-ai/DeepSeek-V4-Flash-0731

./build-and-copy.sh --vllm-ref releases/v0.25.1 --rebuild-vllm

Changed these based on the 0731 model card suggestions.

      --speculative-config '{{"method":"dspark","num_speculative_tokens":{num_speculative_tokens},"draft_sample_method":"greedy"}}' \
      --override-generation-config '{{"temperature":1.0,"top_p":0.95}}' \

[vedcsolution]

@sqrlmstr5000 Yes — we put the official deepseek-ai/DeepSeek-V4-Flash-0731 through a full first-light + sweep on 2x GB10 yesterday (our base is a newer vLLM line, v0.26.1rc1.dev191, carrying the same paired TOPK=256 mods from this PR). tp2 (TP=2 + EP) numbers, isolated port, warm canonical pass:

Metric	tp2 (2x GB10)
Single stream, 512 tok e2e (incl. TTFT)	~35 t/s
4 concurrent, aggregate	65–70 t/s
DSpark acceptance	40.1% (γ=5, probabilistic+block)
KV pool @ util 0.85, max-len 262k	18.15 GiB ≈ 345k tokens (~55 KiB/token effective)
Cold NFS load → READY (safetensors + lazy)	~5.5 min

Two sweep findings directly relevant to the config you posted:

1. γ: 5 beats 7 on GB10. vLLM validates num_speculative_tokens >= dspark_block_size (5), so 5 and 7 are the only valid points — and the model card's γ=7 loses on this bandwidth-bound hardware: 30.1 t/s @ 24.7% acceptance vs 33.1 @ 35.2% for γ=5 (it drafts ~40% more tokens to accept fewer of them).
2. "draft_sample_method":"probabilistic" + "rejection_sample_method":"block" beat greedy at temp 1.0: +6.6% single-stream (35.3 vs 33.1 t/s) and +5 pts acceptance (40.1% vs 35.2%). One caveat: we carry vLLM PRs #47386 + #47524 (Gumbel draft/target decoupling, 64-bit uniforms) as mods, which is what keeps probabilistic sampling target-faithful — on a build without those, greedy is the safer default, so your choice is reasonable as-is.

Also measured: --kv-cache-dtype fp8_ds_mla bought nothing over fp8_e4m3 (identical pool, neutral perf). And if your build line grows the newer flags: use_fp4_indexer_cache hard-asserts outside the SM100 family (never pass it on GB10 — the FP8 fallback is automatic), and deep_gemm_mega_moe is SM100-only kernels, so --moe-backend auto (which resolves to DEEPGEMM_MXFP4 here) is the right call.

FWIW, the checkpoint also scales nicely past 2 nodes: on 4x Spark TP=4+EP we get ~47 t/s single-stream / ~101 at 4-concurrent, and with --max-num-seqs 32 it saturates around ~337 t/s aggregate at 32-concurrent. Native 1M context fits too — but pin the KV pool (--kv-cache-memory-bytes, we use 48 GiB/node) because at 1M the profiler leaves only ~2 GiB free under load, which is inside unified-memory freeze territory.