Skip to content

[Perf] Enable SonicMoE FP8 storage release#2

Open
A-nnonymous wants to merge 1 commit into
pkuzyc:sonicmoe_fp8_dispatchfrom
A-nnonymous:panzhaowu/sonicmoe-fp8-storage-release
Open

[Perf] Enable SonicMoE FP8 storage release#2
A-nnonymous wants to merge 1 commit into
pkuzyc:sonicmoe_fp8_dispatchfrom
A-nnonymous:panzhaowu/sonicmoe-fp8-storage-release

Conversation

@A-nnonymous

Copy link
Copy Markdown

Summary

  • install native 1x32 SonicMoE FP8 weight caches from PaddleFleet expert quantization
  • clear SonicMoE FP8 weight caches once per step before reinstalling to avoid stale tensor-version cache growth
  • keep imports under paddlefleet_ops.sonicmoe and harden unavailable cudnn frontend loading
  • update sonic-moe submodule to [Perf] Improve FP8 dispatch storage release PFCCLab/supersonic-moe#43

Validation

  • python -m py_compile src/paddlefleet/transformer/moe/moe_expert.py packages/paddlefleet_ops/src/paddlefleet_ops/__init__.py
  • import smoke: paddlefleet_ops.is_sonic_moe_available() == True, bare sonicmoe evicted from sys.modules
  • FP8 config smoke: enabled=True, quack=True, fused_gated=True, fuse_y1_quant=True, fp8_wgrad=True, iso32_weight=False, recompute_z=False, save_z_fp8=False
  • 20-step PaddleFormers SonicMoE FP8 smoke completed on 8 GPUs with no Traceback/OOM
  • 300-step validation completed before final cleanup: FP8 global peak 79.953 GiB vs BF16 85.731 GiB; current memory stable around 16.986 GiB

Dependency

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant