[AgentX] vLLM DeepSeek-V4 B200 aggregate / vLLM DeepSeek-V4 B200 聚合配置#2224
Conversation
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
1 similar comment
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
cquil11
added
NVIDIA
full-sweep-enabled
agentx
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29432192113 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| @@ -1,5 +1,5 @@ | |||
| #!/usr/bin/env bash | |||
| set -euo pipefail | |||
| set -eo pipefail | |||
There was a problem hiding this comment.
🟡 Line 2 changed set -euo pipefail to set -eo pipefail, dropping nounset — this is the only script among all 24 in benchmarks/single_node/agentic/ that doesn't use -u. The rewrite replaced every ${VAR:-default}-guarded reference ($DCP_SIZE, $SLURM_JOB_ID, $SLURMD_NODENAME, $MODEL_PATH, $KV_OFFLOAD_BACKEND) with bare $VAR forms, which is why -u had to go; the near-identical dsv4_fp4_b300_vllm.sh shows the same optional-var pattern works fine with -u kept and ${VAR:-} guards. Recommend restoring set -euo pipefail and guarding the new bare references, to keep this script's safety net consistent with its siblings.
Extended reasoning...
The bug: Line 2 of dsv4_fp4_b200_vllm.sh was changed from set -euo pipefail to set -eo pipefail, silently dropping the nounset (-u) flag. Every other script in benchmarks/single_node/agentic/ — including the near-identical dsv4_fp4_b300_vllm.sh, plus dsv4_fp4_mi355x_vllm.sh, dsv4_fp4_b200_sglang.sh, dsv4_fp8_h200.sh, and all the kimik2.5_*/qwen3.5_*/minimaxm3_* variants — still uses set -euo pipefail. This script is now the sole outlier in the directory.\n\nWhy it happened: The diff rewrote several previously-guarded optional-variable references into bare, unguarded forms: DCP_SIZE=${DCP_SIZE:-1} became an if [ -z "$DCP_SIZE" ]; then DCP_SIZE=1; fi block (which still reads the bare $DCP_SIZE first), ${SLURM_JOB_ID:-}/${SLURMD_NODENAME:-unknown} became bare $SLURM_JOB_ID/$SLURMD_NODENAME, ${MODEL_PATH:-} became bare $MODEL_PATH, and the new case "$KV_OFFLOAD_BACKEND" in "") branch reads $KV_OFFLOAD_BACKEND unguarded (note it isn't even part of check_env_vars, so it can legitimately be unset). None of these would survive set -u if the variable is genuinely unset, so -u had to be dropped to make the rewritten code work as written.\n\nWhy this is avoidable: The sibling dsv4_fp4_b300_vllm.sh proves -u and this exact optional-variable pattern coexist fine: it keeps set -euo pipefail while using DCP_SIZE="${DCP_SIZE:-1}", declare -p SLURM_JOB_ID >/dev/null 2>&1 && [ -n "$SLURM_JOB_ID" ], and similar guards for MODEL_PATH. The new case statement here could equally have matched on "${KV_OFFLOAD_BACKEND:-}" instead of the bare variable. So dropping -u wasn't required by the case-statement refactor — it was a side effect of un-guarding several other references at the same time.\n\nImpact: With -u disabled, a future typo in a variable name anywhere in this script (e.g. RESULT_DIR misspelled as RESUT_DIR) will silently expand to an empty string and continue executing rather than aborting immediately with an "unbound variable" error — exactly the failure mode -u exists to catch. As written today the script runs correctly (all currently-referenced optional vars are handled, and required vars are separately validated by check_env_vars), so there's no concrete runtime failure right now. This is a latent robustness/consistency regression rather than an active bug.\n\nProof by example: Suppose a future edit adds echo "Writing to $RESUT_DIR" (a typo of $RESULT_DIR) somewhere in this script. With set -eo pipefail (current state), $RESUT_DIR expands to "", the echo prints "Writing to ", and the script continues — the typo goes unnoticed until output is inspected downstream. With set -euo pipefail (every sibling script's setting), the same line would immediately abort with RESUT_DIR: unbound variable, catching the bug at the point of introduction. This is precisely why the convention exists across all 23 other scripts in the directory.\n\nFix: Restore set -euo pipefail on line 2 and guard the newly-unguarded references the same way dsv4_fp4_b300_vllm.sh does: DCP_SIZE="${DCP_SIZE:-1}"/PCP_SIZE="${PCP_SIZE:-1}", [[ -n "${SLURM_JOB_ID:-}" ]] / ${SLURMD_NODENAME:-unknown}, [[ -n "${MODEL_PATH:-}" ]], and case "${KV_OFFLOAD_BACKEND:-}" in.
53cf054 to
455eabc
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29432414334 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29445892486 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29445892486 |
|
vLLM recipe PR: vllm-project/recipes#645 |
|
/reuse-sweep-run 29445892486 |
|
As a PR reviewer and CODEOWNER, I have reviewed this and have:
Additional detail section:
Signed: |
❌❌❌ REJECTED ❌❌❌@cquil11 — Check 3 fails: two deployment-defining kernel-selection args in this PR are not in the merged upstream recipe (vllm-project/recipes#645 / published DeepSeek-V4-Pro recipe): ✅ Check 0 (CODEOWNER): PASS — all changed paths are owned by |
…sweep Rebased onto main after PR #2224 merged the tuned B200 recipe. The LMCache points now live directly in the official dsv4-fp4-b200-vllm-agentic search space alongside the vllm-simple and Mooncake arms (no standalone config section). LMCache 0.5.1 MP server (lmcache_driven transfer mode) + LMCacheMPConnector per PR #2153; the lmcache arm drops --enable-cumem-allocator because cuMem/VMM allocations cannot be CUDA-IPC-exported to the LMCache server. Ladder validated in the PR #2231 bring-up sweep: peak ~25.7k total tok/s/GPU at DEP8 conc 72 with 96-98% cache hit. 中文：在 PR #2224 的调优配方合并后 rebase 到 main。LMCache 测试点直接并入 官方 dsv4-fp4-b200-vllm-agentic 搜索空间，与 vllm-simple、Mooncake 分支 并列（不再使用独立配置段）。LMCache 0.5.1 MP server（lmcache_driven 传输模式）+ LMCacheMPConnector（沿用 PR #2153）；lmcache 分支去掉 --enable-cumem-allocator（cuMem/VMM 分配无法通过 CUDA IPC 导出给 LMCache server）。测试点阶梯已在 PR #2231 调试扫描中验证：DEP8 并发 72 达到峰值约 25.7k total tok/s/GPU，缓存命中率 96–98%。 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Restructure so the sweep runs only the new MTP work, not the existing aggregate: - Revert dsv4-fp4-b200-vllm-agentic to its main (#2224) search space -- unchanged vs main, so it is not re-run. - Add dsv4-fp4-b200-vllm-agentic-mtp: MTP twins (num_speculative_tokens=3) of the aggregate arms (TP8 GPU-resident, TP8 SimpleCPU, DEP8 SimpleCPU, DEP8 Mooncake), each mirroring its non-MTP conc-list. - Point the perf-changelog entry at only the new key, so only it sweeps. (No separate TP8 key: B200's TP8 arms already exist in the #2224 aggregate.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
3 participants
Summary
origin/main.Original PR: #2202
Earlier source PR referenced by #2202: #2188
Testing
bash -n benchmarks/single_node/agentic/dsv4_fp4_b200_vllm.shpython -m pytest utils/matrix_logic/ -q— 221 passed.中文说明
origin/main。原 PR：#2202
#2202 引用的更早来源 PR：#2188
测试
bash -n benchmarks/single_node/agentic/dsv4_fp4_b200_vllm.shpython -m pytest utils/matrix_logic/ -q— 221 项测试通过。