[Discussion]: DeepSeek-V4-Flash-NVFP4

[1zilc]

NVIDIA has released https://huggingface.co/nvidia/DeepSeek-V4-Flash-NVFP4. Has anyone successfully run it on sm12x, and how does its performance compare to the original version?👀

[jasl]

Yeah, I plan to support it.
Eugr says he is working on the FlashInfer side, so I would wait a little bit.

[jasl]

Sorry, I forgot to say, I've already added the support

[1zilc]

Thank you very much for your outstanding work. I track your latest commits daily and build/test them. So far, both performance and stability have reached an ideal state. I hope the vLLM official team can merge your contributions as soon as possible. The SM120 platform can only achieve its intended performance thanks to contributors like you. By the way, this NVFP4 model is actually larger in size than the original version 😅.