Conversation
新增 B200 Dynamo+SGLang AgentX 配方，并依赖固定 SGLang 镜像的默认优化设置。
|
Thanks for the contribution!
中文感谢你的贡献！
|
补充 DSV4 B200 AgentX 配方的上游 PR 链接。
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| - "Rely on the pinned SGLang image's defaults for JIT norm, Top-K v2, SWA leaf splitting, unified-cache out-of-window reclamation, and custom all-reduce v2 instead of overriding them in the recipes." | ||
| - "新增 B200 Dynamo+SGLang AgentX TP8 聚合式 HiCache 配方，并发为 1、4、8；使用 lmsysorg/sglang:nightly-dev-20260916-c9a8fba9、检查点内置 DSpark K=6、整节点 CPU/DRAM 分配，以及按数据点设置的请求数/CUDA graph 上限。" | ||
| - "依赖固定 SGLang 镜像对 JIT norm、Top-K v2、SWA leaf splitting、unified-cache 窗口外槽位回收和 custom all-reduce v2 的默认行为，不在配方中覆盖这些设置。" | ||
| pr-link: TBD |
There was a problem hiding this comment.
🔴 Both new changelog entries use pr-link: TBD (lines 8600 and 8611) instead of the required XXX placeholder, so the changelog gate CI will reject this PR. validate_added_pr_link in infx/workflows/validate_perf_changelog.py:134-145 only accepts PR_LINK_PLACEHOLDERS = {"XXX", ".../pull/XXX"} or the exact https://github.com/SemiAnalysisAI/InferenceX/pull/<PR#> URL; "TBD" matches neither, so validate_added_pr_link raises ChangelogValidationError("new PR entry must use ... or an XXX placeholder; found 'TBD'"). …
Why this was flagged
…Fix: replace both pr-link: TBD occurrences with pr-link: XXX so infx.workflows.validate_perf_changelog (run in .github/workflows/run-sweep.yml:117-125) and the later PR-number substitution in infx/workflows/prepare_perf_changelog_merge.py pass, covering both new entries at perf-changelog.yaml:8600 and perf-changelog.yaml:8611.
The trigger is CI running infx.workflows.validate_perf_changelog (invoked from .github/workflows/run-sweep.yml:117-125, 'Validate perf-changelog matrix') on this PR's diff of perf-changelog.yaml. validate_added_pr_link (infx/workflows/validate_perf_changelog.py:134-145) is called for each appended entry with its pr-link string. It checks CANONICAL_PR_LINK regex or membership in PR_LINK_PLACEHOLDERS = {"XXX", "https://github.com/SemiAnalysisAI/InferenceX/pull/XXX"} (lines 20-24). 'TBD' is neither, so it raises ChangelogValidationError with message "new PR entry must use 'https://github.com/SemiAnalysisAI/InferenceX/pull/' or an XXX placeholder; found 'TBD'", failing the workflow that main() (line 336) turns into exit code 1. On base main, entries follow…
Verification: normal (mechanism corrected) — Both new appended entries end with pr-link: TBD (the added block at perf-changelog.yaml:8587+, agg entry ~line 8600 and disagg entry ~line 8611). The repo convention/tooling requires the literal XXX placeholder until the PR exists (CONTRIBUTING.md:180; .github/workflows/claude.yml:235). "TBD" is not accepted: PR_LINK_PLACEHOLDERS = {"XXX",… | Severity:…
| - "Use automatic golden acceptance-length selection and rely on the pinned SGLang image's defaults for SWA leaf splitting, unified-cache out-of-window reclamation, and custom all-reduce v2." | ||
| - "新增 B200 Dynamo+SGLang AgentX DEP8/DEP8 HiCache 配方：1P1D 并发 64、128，以及 2P1D 并发 256；decode max-running-requests 分别限制为 128、256、512；1P1D 的 decode swa-full-tokens-ratio 为 0.01，2P1D c256 为 0.005 且 memory fraction 为 0.91。" | ||
| - "使用自动黄金接受长度选择，并依赖固定 SGLang 镜像对 SWA leaf splitting、unified-cache 窗口外槽位回收和 custom all-reduce v2 的默认行为。" | ||
| pr-link: TBD |
There was a problem hiding this comment.
🟡 (optional) The new perf-changelog entry for dsv4-fp4-b200-dynamo-sglang-agentic-disagg misstates the tuned decode value: it says "decode swa-full-tokens-ratio 0.01 for 1P1D" but the actual decode arg in both 1P1D recipes is 0.02 (0.01 is the prefill value). Fix: correct the description to state decode swa-full-tokens-ratio 0.02 for 1P1D (prefill uses 0.01), so the changelog matches disagg-b200-1p1d-dep8-dep8-c64-mtp-kvoffload.yaml and disagg-b200-1p1d-dep8-dep8-c128-mtp-kvoffload.yaml.
Why this was flagged
perf-changelog.yaml is the record maintainers rely on to know what tuning a recipe uses without opening the YAML. The new entry (perf-changelog.yaml line ~8611) claims decode swa-full-tokens-ratio 0.01 for the 1P1D recipes, but disagg-b200-1p1d-dep8-dep8-c64-mtp-kvoffload.yaml:194 and disagg-b200-1p1d-dep8-dep8-c128-mtp-kvoffload.yaml:194 both set decode swa-full-tokens-ratio: 0.02; 0.01 is actually the prefill value at line 122 in each file. A maintainer trusting the changelog to diff or reproduce the decode tuning gets the wrong number, since nothing cross-checks changelog prose against recipe args.
Verification: nit. The changelog description is factually wrong but nothing functional breaks. In disagg-b200-1p1d-dep8-dep8-c64-mtp-kvoffload.yaml the prefill worker (disaggregation-mode: prefill at line 117) sets swa-full-tokens-ratio: 0.01 at line 122, while the decode worker (disaggregation-mode: decode at line 190) sets swa-full-tokens-ratio: 0.02 at line 194.… | nit. The changelog entry at…
There was a problem hiding this comment.
[by Codex] Fixed in 96dadb0. The changelog now states the actual 1P1D values: prefill swa-full-tokens-ratio: 0.01 and decode swa-full-tokens-ratio: 0.02; the 2P1D c256 decode value remains 0.005.\n\n
中文
\n\n已在 96dadb0 中修复。changelog 现准确记录 1P1D：prefill 为swa-full-tokens-ratio: 0.01，decode 为 swa-full-tokens-ratio: 0.02；2P1D c256 的 decode 值仍为 0.005。\n\n更正 DSV4 解耦配方变更记录中的 prefill 与 decode SWA 比例。
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=35687016078 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=35687016078 |
|
As a PR reviewer and CODEOWNER, I have reviewed this and have:
Additional detail section:
Signed: |
❌❌❌ REJECTED ❌❌❌@kedarpotdar-nv — two blocking issues at ❌ Check 4 (Reuse command): FAIL — No authorized reuse command has been posted on this PR. The only comments are two bot notices and the sign-off; an ❌ Check 13 (Draft runs as shipped): FAIL —
Passed and not applicable checks✅ Check 0 (CODEOWNER): PASS — ✅ Check 1 (Passing sweep on in-PR commit): PASS — head ✅ Check 2 (Evals pass): PASS — ➖ Check 3 (Recipe linked/merged/complete): N/A — disaggregated/multi-node submission; both entries are ✅ Check 5 (Latest checklist template): PASS — all 14 items plus the merged-upstream sub-item of the current ✅ Check 6 (Upstream image / engine-first): PASS — image is upstream ✅ Check 7 (No deprecated models/scenarios): PASS — ✅ Check 8 (No architecture hacks): PASS — no ✅ Check 9 (Spec-decode via chat template): PASS — recipes run ✅ Check 10 (No engine patches): PASS — no ✅ Check 11 (Agentic spec-decode golden AL): PASS — ➖ Check 12 (Append-only): N/A — neither new Assessed commit: |
This branch has not been deployed
2 participants
Summary
SGLANG_OPT_USE_JIT_NORM,SGLANG_OPT_USE_TOPK_V2,SGLANG_OPT_SWA_SPLIT_LEAF_ON_INSERT,SGLANG_OPT_UNIFIED_CACHE_FREE_OUT_OF_WINDOW_SLOTS, andSGLANG_OPT_USE_CUSTOM_ALL_REDUCE_V2.DeepSeek-V4-Pro-0813checkpoint alias on the B200 Nscale native srt-slurm path.This supersedes #3257 with a clean branch rebased on current
main.Validation
bash -n runners/launch_b200-nscale-slurm.shsrtctl migrate --verifyover all six recipes: 6 identical, 0 mismatched, 0 skipped, 0 unreadableAI model disclosure
OpenAI GPT-5 (Codex) prepared the branch, validation, and PR description. No delegated agents were used.
中文
摘要
SGLANG_OPT_USE_JIT_NORM、SGLANG_OPT_USE_TOPK_V2、SGLANG_OPT_SWA_SPLIT_LEAF_ON_INSERT、SGLANG_OPT_UNIFIED_CACHE_FREE_OUT_OF_WINDOW_SLOTS和SGLANG_OPT_USE_CUSTOM_ALL_REDUCE_V2。DeepSeek-V4-Pro-0813检查点别名。此 PR 使用基于最新
main的干净分支取代 #3257。验证
bash -n runners/launch_b200-nscale-slurm.shsrtctl migrate --verify：6 个一致，0 个不匹配，0 个跳过，0 个不可读AI 模型披露
OpenAI GPT-5（Codex）用于准备分支、验证及 PR 描述。未使用委派代理。