Skip to content

feat: DeepSeek V4 Flash DSpark C12 NVFP4 recipe + mod - #311

Open
mtdphn wants to merge 1 commit into
eugr:mainfrom
mtdphn:feat/dsv4-dspark-nvfp4-recipe
Open

feat: DeepSeek V4 Flash DSpark C12 NVFP4 recipe + mod#311
mtdphn wants to merge 1 commit into
eugr:mainfrom
mtdphn:feat/dsv4-dspark-nvfp4-recipe

Conversation

@mtdphn

@mtdphn mtdphn commented Jul 7, 2026

Copy link
Copy Markdown

spark-vllm-docker v1 style recipe for DeepSeek-V4-Flash-DSpark

Eugr-format recipe for serving DeepSeek-V4-Flash-DSpark on 2x DGX Spark (GB10) with vLLM TP=2, DSpark speculative decoding, NVFP4 KV cache (nvfp4_ds_mla), B12X MoE backend, and the 2026-07-03 garble fix.

Includes

  • Recipe: recipes/deepseek-v4-flash-dspark.yaml — v1 format, cluster-only (2-node)
  • Mod: mods/deepseek-v4-flash-dspark/ — 18 overlay files + 3 NVFP4 patch stages

Key Parameters

Param Value
container vllm-dspark-runtime:dspark-nvfp4-stage-c
gpu_memory_utilization 0.77
max_model_len 350000
kv_cache_dtype nvfp4_ds_mla
max_num_seqs 12
num_speculative_tokens 5
distributed-executor-backend mp

Attribution

  • tonyd2wild — C12 NVFP4 default config (canonical 350K verified profile)
  • MiaAI-Lab — DSpark concurrency patch (Keys C12)
  • 0rand, renek — DSpark cold-start garble root cause analysis
  • Aiden Le — production-3.2 image reference, fp8 fallback guidance

Verified

  • Deployed on 2x DGX Spark (back-to-back, 100G fabric)
  • Tool-eval-bench: 87/100 (15 scenarios, 10 trials)
  • 13 passed, 1 partial, 1 failed
  • Median turn time: 2.4s

Notes

  • Tested on Gigabyte Atom heads at gpu_memory_utilization=0.77 (~94 GiB free). Clean Spark units can use 0.80.
  • The mod's run.sh verification script previously emitted bare print(...) calls (missing f-string prefix) causing benign SyntaxWarning during import verification — fixed in this submission.

spark-vllm-docker v1 style recipe for serving DeepSeek-V4-Flash-DSpark on 2x DGX Spark (GB10)
with vLLM TP=2, DSpark speculative decoding, NVFP4 KV cache (nvfp4_ds_mla),
B12X MoE backend, and the 2026-07-03 garble fix.

Recipe: recipes/deepseek-v4-flash-dspark.yaml
Mod: mods/deepseek-v4-flash-dspark/ (18 overlay files + 3 NVFP4 patches)

Based on tonyd2wild C12 NVFP4 default config (2026-07-04).
Includes Keys DSpark concurrency patch via overlay files.
NVFP4 KV cache implementation via 3-stage patch pipeline.

Default: gpu_memory_utilization=0.77, max_model_len=350000, nvfp4_ds_mla.
Distributed executor backend: mp (multiprocessing).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant