[Performance] Integrate SonicMoE hotpath optimizations

[A-nnonymous]

PR Category

Performance Optimization

PR Types

Performance

Description

Summary:

• Replace the SonicMoE hotpath env toggles in Fleet with SonicMoEConfig() defaults.
• Keep imports under the strict paddlefleet_ops.sonicmoe.* namespace.
• Add capability detection for the SonicMoE metadata router-score scatter backward symbol, with fallback to the existing differentiable router-score path when older paddlefleet_ops builds are used.
• Update the packages/paddlefleet_ops/third_party/sonic-moe submodule pointer to PFCCLab/supersonic-moe commit 08062c431f59571b582645e19b400943c745e556 (Add SonicMoE weight layout bit-exact tests, Add bit-exact tests for SonicMoE weight layout kernels PFCCLab/supersonic-moe#62), based on merged SonicMoE layout kernel optimization [MoE] Add Base MoE Layer #61 and metadata/layout hotpath update Fix attention dim order #60.
• Preserve persistent SonicMoE weight/grad buffer identity during layout conversion by writing converted values back with paddle.assign(..., output=tensor); this intentionally avoids _share_buffer_to for parameters, main_grad, and param.grad because optimizers and gradient buffers may hold those storages.

Dependency:

• Optimize SonicMoE metadata and layout hotpaths PFCCLab/supersonic-moe#60 has been merged; it provides the metadata and layout hotpath base currently used by Fleet.
• Optimize SonicMoE weight layout kernels PFCCLab/supersonic-moe#61 has been merged; it carries the tiled W1/W2 layout kernel optimization now referenced by this Fleet submodule pointer.
• Add bit-exact tests for SonicMoE weight layout kernels PFCCLab/supersonic-moe#62 adds strict bit-exact layout-kernel tests, including opt-in >4GB tensor coverage and compute-sanitizer validation.

Validation:

• pre-commit run --all-files
• python -m py_compile src/paddlefleet/transformer/moe/fusion_layer_utils.py src/paddlefleet/transformer/moe/moe_expert.py
• _SonicRouterScoresFromMetadata smoke: [TK_padded] metadata scores with [TK] score indices keep out.stop_gradient=False when topk_scores.stop_gradient=False.
• MAX_JOBS=192 uv pip install -e packages/paddlefleet_ops -vv --no-build-isolation completed successfully after the Sonic config changes.
• No-env resolver smoke with SONIC_MOE_FUSE_Y1_QUANT, SONIC_MOE_FUSE_Y1_BF16_TRUNC, and SONIC_MOE_FUSED_WEIGHT_LAYOUT unset: defaults and _FP8Config snapshot are True True True.
• GPU fused-layout parity smoke: W1 grouped->sonic, sonic->grouped roundtrip, fallback roundtrip, and W2 transpose all reported max diff 0.0.
• SonicMoE [MoE] Add Base MoE Layer #61 file-level validation: python -m py_compile sonicmoe/ernie_compat/weight_layout_fusion.py, git diff --check, and GPU bit-exact smoke for small W1/W2 plus A1B-shape W1 [32,1024,2048] roundtrip and W2 [32,1024,1024] transpose.
• SonicMoE Add estimate_mfu #62 test validation:
  • python -m py_compile sonicmoe/ernie_compat/weight_layout_fusion.py tests/ops/test_weight_layout_fusion.py
  • CUDA_VISIBLE_DEVICES=0 python -m pytest -q tests/ops/test_weight_layout_fusion.py: 54 passed, 3 skipped
  • CUDA_VISIBLE_DEVICES=5 SONIC_MOE_RUN_LARGE_LAYOUT_TESTS=1 python -m pytest -q tests/ops/test_weight_layout_fusion.py -k large_tensor_crosses_int32_boundary: 3 passed
  • compute-sanitizer --tool memcheck on W2 transpose >4GB case: 1 passed, ERROR SUMMARY: 0 errors
  • compute-sanitizer --tool memcheck on both W1 >4GB cases: 2 passed, ERROR SUMMARY: 0 errors
• SonicMoE layout write-back pointer smoke after review fix: _assign_tensor() keeps target data_ptr() stable for same-shape and reshape write-back while copying values correctly.
• NCU/event timing from the same optimized SonicMoE implementation used in the runtime install path:
  • W1 grouped -> Sonic: 3995.6 us -> 590.5 us, 6.77x
  • W1 Sonic -> grouped: 3867.0 us -> 604.8 us, 6.39x
  • W2 transpose: 1948.2 us -> 299.0 us, 6.52x
• Nsight Systems smoke: output/nsys_profiles/sonic_cfg_default_10_13_20260705_224656_rank0_3639289.nsys-rep captured steps 10-13 with no legacy env flags. Timeline contains GemmGatedSm100ZeroMatQuantPostActQuant plus fused layout kernels.
• Cache/warmup sanity: after clearing SonicMoE DeepEP topk metadata cache, script/warmup_sonicmoe_deepep_topk_metadata.py rebuilt the artifact successfully, and a 4-step A1B training smoke completed all ranks with normal loss.
• A1B barrier-on 950-step run completed with ret=0 and last_step=950 using moe_dispatch_bwd_ep_barrier: true; final loss=1.0540837, global_grad_norm=0.205339, tokens_per_sec_per_card_total_average=15879.4.
• A1B 950-step loss/norm comparison against /root/paddlejob/share-storage/gpfs/system-public/zhangyichen/erniebot/eb5v2_logs/fp8_sonic_950steps/workerlog.0: common steps 1..950; loss max abs diff 2.8595e-02, mean abs diff 1.8329e-03, final loss diff 1.0437e-03; global grad norm max abs diff 1.49989, mean abs diff 0.10330, final norm diff 0.010845.

Note:

• This update intentionally does not change EP barrier behavior. The earlier local moe_dispatch_bwd_ep_barrier experiment was removed from this PR scope because barrier-off has correctness risk.
• The worktree-local uv.lock resolver churn is not included in this PR.
• Direct pytest invocation of the referenced single-card SonicMoE precision test now reaches the FP8 forward path but fails because weight.fp8 is missing before quant_weight(), which is unrelated to the buffer identity fix. The local pointer-stability smoke for _assign_tensor() passes.

是否引起精度变化

否

[risemeup1111]

已完成首轮 review，未发现需要阻塞合入的代码正确性问题。会导致 Codestyle Check 失败的格式问题已标在 inline comments 里。

优先级：P2
非行级：PR 描述不属于 diff 行。当前 PR body 没有使用仓库模板里的 ### PR Category / ### PR Types / ### Description 字段，Check PR Description 已因此失败。请按 .github/pull_request_template.md 补齐，例如：PR Category: Performance Optimization、PR Types: Performance，并把现有 Summary/Dependency/Validation 内容放到 ### Description 下。

^{Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.}

[Copilot]

Pull request overview

该 PR 旨在将 Fleet 侧 SonicMoE 热路径的若干优化开关从“环境变量驱动”迁移到 SonicMoEConfig() 的默认配置/运行时配置上下文，并在权重布局转换与路由分数梯度路径上接入（可选的）fused/metadata 快路径，以匹配更新后的 paddlefleet_ops.sonicmoe.* 生态与子模块版本。

Changes:

• 在 SonicMoEExpert 中引入 SonicMoEConfig() 默认配置，并在 W1 grouped↔sonic、W2 transpose 布局变换中优先尝试 fused kernel，失败时回退到原有 Python 实现（带日志）。
• 在 run_sonic_moe() 中增加 SonicMoEConfig().activate() 运行时上下文，并在 deepep metadata 提供 _score_src_idx 时走新的 _SonicRouterScoresFromMetadata 梯度通路。
• 调整 FP8 weight carrier 的构造方式，使其更明确地作为“仅承载 shape/dtype/FP8 attrs 的载体”。

流程/规范提醒：

• PR 标题目前不符合约定的 [CLASS]Title 格式；建议改为类似 [Perf] Integrate SonicMoE hotpath optimizations / [Refactor] ... 等。

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
src/paddlefleet/transformer/moe/moe_expert.py	引入 SonicMoEConfig 默认配置并接入 fused 权重布局转换（带异常回退与日志）。
src/paddlefleet/transformer/moe/fusion_layer_utils.py	增加 SonicMoEConfig 运行时激活上下文；新增从 metadata 路由分数构建梯度的 PyLayer 快路径；调整 FP8 weight carrier。

[risemeup1111]

已复查最新提交。SonicMoE config 默认路径和 submodule 指针更新未发现新的阻塞性代码问题；之前的格式问题仍由已有 inline threads 覆盖，其中导入顺序线程已按当前代码补充了更新后的修复形态。

优先级：P2
非行级：PR 描述仍未按仓库模板提供 ### PR Category / ### PR Types / ### Description，当前 Check PR Description 仍在按现有正文解析并失败。建议改成模板结构，例如：### PR Category 下填 Performance Optimization，### PR Types 下填 Performance，现有 Summary/Dependency/Validation 放到 ### Description 下。

^{Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.}

[risemeup1111]

已复查当前提交。此前 Nyanpasu 提出的格式和 PR 描述问题已解决；这次新增的 router score metadata guard 有一个会阻断合法 FP8 多专家 padding 场景的问题，详情见 inline comment，需要修复后再合入。

^{Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.}

[risemeup1111]

已复查最新提交；上一轮指出的 router score padding guard 已移除，先前 P1 问题已解决。当前未发现新的需要阻塞合入的代码问题。

^{Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.}

[Paddle-CI-Bot]

PaddleFleet Log Analysis

Run #29088355020 · Attempt 1

日志分析报告

流水线名称	问题标签	修复建议	日志片段
Coverage Upload And Check	增量覆盖率不达标（exit 9）	为本次新增的 SonicMoE 相关代码补充单测，覆盖率需达到 90%	报错代码

失败的测试case:

无单测失败，Coverage 检查本身失败（exit code 9）
diff-cover 报告：
  src/paddlefleet/transformer/moe/fusion_layer_utils.py  覆盖率 17.0%
    Missing lines: 53-56, 58-59, 62-63, 83, 102-104, 110-111, 114-115,
                   119-120, 123-124, 129-130, 133-140, 144-146, 152, 155,
                   2388-2389, 2393, 2414
  src/paddlefleet/transformer/moe/moe_expert.py          覆盖率 44.7%
    Missing lines: 466-468, 472-477, 488-493, 506-511

  Total changed lines: 85  |  Missing: 60  |  Overall: 29%（阈值 90%）

根本原因分析:
PR 新增了 SonicMoE 集成相关代码（fusion_layer_utils.py 中的 _SonicRouterScoresFromMetadata、_sonic_moe_runtime_config_context，以及 moe_expert.py 中 SonicMoEExpert._use_fused_weight_layout 和 fused weight layout 路径），但对应的单测覆盖率仅 29%，远低于 90% 阈值，触发 exit 9。

修复建议:

1. 在 tests/ 下为以下新增代码补充单测：
   • fusion_layer_utils.py L53-63（SonicMoEConfig/_scatter_router_scores_i32 导入分支）、L102-155（_SonicRouterScoresFromMetadata forward/backward 路径）、L2388-2414（run_sonic_moe 中新增的 metadata router scores 分支）
   • moe_expert.py L466-468（_use_fused_weight_layout）、L472-493（fused_grouped_w1_to_sonic 调用路径）、L506-511（fused_transpose_w2_layout 调用路径）
2. 对于依赖 paddlefleet_ops.sonicmoe 的路径，可用 unittest.mock.patch mock 掉算子，使分支在 CPU 环境下也能被覆盖。
3. 补完后本地运行 diff-cover 确认增量覆盖率 ≥ 90% 再推送。

---

🔍 准确性记录：请点击评论底部 😊 图标，选择 👍（准确）或 👎（有误），将自动记录到 CI 监控系统

_{🔄 每次 Re-run 后自动更新}

[risemeup1111]

已复查最新提交；_assign_tensor() 已恢复为 paddle.assign(..., output=tensor) 的写入方式，之前的 buffer identity 风险看起来已解决。SonicMoE 子模块更新新增了上游 layout bit-exact 测试，未发现新的需要阻塞合入的代码问题。

当前仍有 Build Fleet whl / Check approval CI 失败；从日志看 Build Fleet whl 失败发生在拉取 Docker 镜像时的 registry/auth 超时，未进入代码构建阶段，建议重跑或等待基础设施恢复。

^{Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.}

[PaddlePaddle-bot]

🤖 Paddle-CI-Agent | pr_review | 2026-07-07 17:51:18

📋 Review 摘要

PR 概述：集成 SonicMoE hotpath 配置、metadata router-score scatter backward 和 fused weight layout 路径。
变更范围：src/paddlefleet/transformer/moe/fusion_layer_utils.py、src/paddlefleet/transformer/moe/moe_expert.py、packages/paddlefleet_ops/third_party/sonic-moe
影响面 Tag：MoE OP Third-Party

问题

未发现阻塞性问题。PR 规范检查见下节。

历史 Findings 修复情况

Finding	问题	状态
F1	_share_buffer_to 不能用于更新持久化参数/梯度 buffer	✅ 已修复

📝 PR 规范检查

符合规范：标题已使用 [Performance] Tag，描述包含 ### PR Category / ### PR Types / ### Description 结构且内容非空。

总体评价

本轮按风险优先审查了 SonicMoE metadata router-score fallback、SonicMoEConfig 默认配置接入、fused weight layout fallback 和 submodule 指针说明。未发现需要阻塞合入的新问题；历史 buffer identity finding 已通过 paddle.assign(..., output=tensor) 写回路径修复。

[codecov-commenter]

Codecov Report

❌ Patch coverage is 29.41176% with 60 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (develop@7fb30f5). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
.../paddlefleet/transformer/moe/fusion_layer_utils.py	17.02%	39 Missing ⚠️
src/paddlefleet/transformer/moe/moe_expert.py	44.73%	21 Missing ⚠️

❌ Your patch status has failed because the patch coverage (29.41%) is below the target coverage (90.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##             develop    #1400   +/-   ##
==========================================
  Coverage           ?   29.41%           
==========================================
  Files              ?        2           
  Lines              ?       85           
  Branches           ?       15           
==========================================
  Hits               ?       25           
  Misses             ?       60           
  Partials           ?        0           

Flag	Coverage Δ
coverage_combine	29.41% <29.41%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
src/paddlefleet/transformer/moe/moe_expert.py	44.73% <44.73%> (ø)
.../paddlefleet/transformer/moe/fusion_layer_utils.py	17.02% <17.02%> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.