[AgentX] vLLM DeepSeek-V4 B300 aggregate MTP#2258
Conversation
Signed-off-by: Jeff Ma <jeffjma@umich.edu>
Signed-off-by: Jeff Ma <jeffjma@umich.edu>
Signed-off-by: Jeff Ma <jeffjma@umich.edu>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LjHXfE1FRN9c5XQqgeTdez
Restore upstream lines whitespace-for-whitespace and append the B300 entry so the changelog diff contains no deletions. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LjHXfE1FRN9c5XQqgeTdez
# Conflicts: # configs/nvidia-master.yaml # perf-changelog.yaml
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
… the recipe Finalize the B300 AgentX vLLM search space (dsv4-fp4-b300-vllm-agentic): - Add a TP8 GPU-resident arm at conc [1,2,4,6,8]. - Trim TP4 GPU-resident to conc [1,2,4,6,8,12,16,20]; extend TP4 SimpleCPU lazy-offload down to conc [20,24,28,32,36,40]. - Add an MTP speculative-decoding twin (num_speculative_tokens=3) for every topology (TP8/TP4 GPU-resident, TP4 SimpleCPU, DEP4, DEP8), each mirroring its non-MTP conc-list, routed via the launcher's spec-decoding=mtp suffix to dsv4_fp4_b300_vllm_mtp.sh. MTP script: NUM_SPEC_TOKENS=3 -> TOKENS_PER_SEQ=4, so FULL_DECODE_ONLY cudagraph capture sizes (num_seqs*TOKENS_PER_SEQ) scale to num_seqs*4. Sync the tail with dsv4_fp4_b300_vllm.sh to restore the EVAL_ONLY branch (added upstream by #1947) so MTP configs also run the SWE-bench Lite accuracy eval. Add the perf-changelog entry for dsv4-fp4-b300-vllm-agentic (PR #2258). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…tion #1947 made single-node agentic recipes generate a SWE-bench eval row (run_eval/eval_only + agentic fields) but never widened the changelog matrix schema, so ChangelogMatrixEntry.evals (typed list[SingleNodeMatrixEntry], fixed-seq-len only) rejects every agentic eval row -- breaking check-changelog for any single-node agentic PR. Widen evals to the same Union single_node already uses, and give SingleNodeAgenticMatrixEntry optional run_eval/eval_only (None-default, so benchmark rows are unchanged). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29564785969 |
Restructure so the sweep runs only the new work, not the existing aggregate: - Revert dsv4-fp4-b300-vllm-agentic to its main (#2241) search space (TP4 [1..32], SimpleCPU [28..40], DEP4, DEP8) -- unchanged vs main, so it is not re-run. - Add dsv4-fp4-b300-vllm-agentic-mtp: MTP twins of the aggregate arms (num_speculative_tokens=3), reshaped TP4 [1..20] / SimpleCPU [20..40]. - Add dsv4-fp4-b300-vllm-agentic-tp8: new TP8 GPU-resident topology at conc [1,2,4,6,8], both non-MTP and MTP. - Point the perf-changelog entry at only the two new keys, so only they sweep. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29566371654 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29566417002 |
3 participants
No description provided.