[Roadmap] Blackwell MXFP8 and NVFP4 RL training

[zianglih]

@HumansAnd

Miles

• MXFP8
  • Implement Blackwell MXFP8 recipe #512
  • Add --first-last-layers-bf16 #614
• NVFP4
  • Implement nvfp4 #546
  • Bump Megatron version to inlcude --te-precision-config-file #567

SGLang

• MXFP8
  • Add mxfp8 support for online quantization, Triton dense linear, and CUTLASS MoE sgl-project/sglang#17449
  • [RL] Support per-layer mixed FP8/BF16 serving for FP8 checkpoints sgl-project/sglang#18742
  • Add support for FlashInfer mxfp8 sgl-project/sglang#18945
  • Expand deep_gemm entrypoint to support more FP8 recipes. sgl-project/sglang#17294
• NVFP4
  • [RL] Add an nvfp4 online input scale mode sgl-project/sglang#18012
  • Fix nvfp4 weight update sgl-project/sglang#18085

TransformerEngine

• MXFP8 & NVFP4
  • Add NVTE_BACKWARD_MODE=default|unquant|dequant NVIDIA/TransformerEngine#2644
    • For high-precision wgrad & dgrad
  • Add selection for dequant or original high-precision input

FlashInfer

• MXFP8
  • Implement cutlass_fused_moe mxfp8 flashinfer-ai/flashinfer#2581