Repository navigation
glm5.2 gb300 trt agentx: rc28 + agent bounce + Dynamo --publish-metrics / glm5.2 gb300 trt agentx：rc28 + agent bounce + Dynamo --publish-metrics - #3707
Conversation
|
Thanks for the contribution!
中文感谢你的贡献！
|
richardhuo-nv
added
the
full-sweep-fail-fast
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.
Additional findings (outside the current diff — GitHub can't attach inline comments there):
-
🟡
inferencex-e2e/configs/nvidia-master.yaml— Maintainers and dashboards now get a wrong Dynamo version recorded for every one of the six glm5.2 gb300 trt-agentx benchmark points, where one point used to be labeled correctly. router.version stays "1.4.0.dev20260807" while the paired recipe moves all six points, including the one that used to match this exact wheel, to ai-dynamo 1.6.0.dev20260922. generate.py's component_metadata() copies this field verbatim into every generated matrix entry/result record for the config, so all six runs now report the wrong Dynamo version with no validator catching the mismatch (ComponentMetadata only checks non-empty strings, not consistency with the recipe). Fix: when a recipe's dynamo.source changes, update the matching master-config router.version too; here set it to 1.6.0.dev20260922.Why this was flagged
Trigger: configs/nvidia-master.yaml:7731 keeps router.version="1.4.0.dev20260807" for glm5.2-fp4-gb300-dynamo-trt-agentic-disagg-mtp while disagg-variants.yaml now sets dynamo.source.wheel: 1.6.0.dev20260922 on all six override blocks (previously override_1p1d_tp8_c1_b1_mtp5 alone used wheel 1.4.0.dev20260807, matching this field). infx/matrix/generate.py:555 component_metadata() reads Fields.ROUTER straight from the master config and attaches it, unmodified, to every generated matrix entry/result label for this config. No code cross-checks router.version against the recipe's dynamo.source (ComponentMetadata in validation.py only enforces non-empty strings and rejects image: prefixes). Result: every one of the six benchmark runs for this config now has mislabeled Dynamo version provenance, regressing a previously-correct label for the c1 point.
Verification: nvidia-master.yaml:7731 keeps router.version at "1.4.0.dev20260807" while disagg-variants.yaml now sets dynamo.source.wheel to 1.6.0.dev20260922 on all six override blocks. generate.py:555-558 reads Fields.ROUTER verbatim from the master config with no cross-check against the recipe wheel, so every entry records the wrong Dynamo version. On base, c1 matched this field; the PR leaves router.version unbumped, so that one correct point is now mislabeled too.
| - "六个点的 Dynamo 均改为预编译的 ai-dynamo 1.6.0.dev20260922 wheel（包含 ai-dynamo/dynamo#12162，将性能指标与 KV 事件解耦），引擎仅以 --publish-metrics 发布指标（publish_metrics true，不发布 KV 事件）。AIPerf 通过 srtctl 公布的 worker 地址抓取 trtllm_* 指标（kv_cache_hit_rate、kv_cache_utilization），AIPERF_REQUIRED_SERVER_METRIC_PREFIX=trtllm_kv_cache_utilization 现适用于所有点。" | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXXX |
There was a problem hiding this comment.
🟡 (optional) New perf-changelog.yaml entry (lines 9296-9305) adds Chinese description lines, violating AGENTS.md:126: 'New perf-changelog.yaml entries must be English-only. Do not add Chinese translations or bilingual descriptions.' This mirrors the PR body's general bilingual convention, but AGENTS.md explicitly carves out changelog entries as English-only, unlike PR titles/descriptions. Fix: remove the two Chinese description strings (lines 9304-9305) and keep only the English description lines for this entry.
Why this was flagged
AGENTS.md:126 states 'New inferencex-e2e/perf-changelog.yaml entries must be English-only. Do not add Chinese translations or bilingual descriptions; the bilingual documentation and GitHub-content rules do not apply to these entries.' The new entry appended at perf-changelog.yaml:9293-9305 for glm5.2-fp4-gb300-dynamo-trt-agentic-disagg-mtp includes two Chinese description lines (9304-9305) mirroring the English ones. Every prior entry in the file is English-only (confirmed by scanning historical entries), so this is the first entry to break the convention. A maintainer or tooling that enforces this changelog convention will reject the PR as written, and the perf-changelog file (which is append-only/byte-sensitive per AGENTS.md) would need a follow-up edit to strip the Chinese lines.
Verification: nit — clear violation of an explicit documented invariant introduced by this diff; nothing breaks at runtime.
CLAUDE.md:3 delegates all guidance to AGENTS.md. AGENTS.md:126 (also listed under "Non-negotiable benchmark invariants") states verbatim: "New inferencex-e2e/perf-changelog.yaml entries must be English-only. Do not add Chinese translations or bilingual descriptions; the bilingual documentation and GitHub-content rules do not apply to these entries. Leave historical entries unchanged."
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=37558340091 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=37558340091 |
richardhuo-nv
added
full-sweep-enabled
Update the image, Dynamo wheel, bounce-buffer settings, and metric publication across all six existing points. Remove the redundant served-model-name patch. 更新 GLM-5.2 GB300 TRT AgentX 六个现有点的镜像、Dynamo wheel、bounce buffer 设置及指标发布方式，删除重复的 served-model-name 补丁。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Co-Authored-By: Cameron Quilici <60715037+cquil11@users.noreply.github.com>
daa6d07 to
ba33a0b
Compare
修复：固定 GLM-5.2 GB300 Dynamo 版本以选择可路由的节点地址。
This branch has not been deployed
4 participants
Summary
Update all six
glm5.2-fp4-gb300-dynamo-trt-agentic-disagg-mtppoints to TensorRT-LLM1.3.0rc28.dev202609210001. Enable the C++ transfer-agent bounce buffer, iteration statistics, and engine metric publication without KV events. Use srt-slurm v2.43.4 and remove the redundant served-model-name patch.Pin Dynamo source to
ee07d6b64f9ebd2cc9bf1972be29ac7d65a4f714, which includes ai-dynamo/dynamo#15628 and selects routed node addresses for metadata and transport advertisement. Record the same revision in router metadata. The launcher builds and caches the pinned runtime on a cold cache.Validation
All six srt-slurm v2.43.4 recipe dry-runs passed. Matrix generation selects six points with the same topology, concurrency, image, and cluster routing. Historical changelog bytes are preserved with one trailing newline. YAML parsing and
git diff --checkpassed. GPU execution of the pinned revision remains unverified.AI model disclosure
Original changes: Claude Opus 5.5, identified in existing commit attribution. This update: Codex (GPT-6 family); the exact model/version could not be verified from the runtime. Codex prepared the source pin, matching router metadata, changelog entry, and local checks. No delegated agents contributed.
中文
概要
将
glm5.2-fp4-gb300-dynamo-trt-agentic-disagg-mtp的六个点统一更新为 TensorRT-LLM1.3.0rc28.dev202609210001。启用 C++ transfer-agent bounce buffer、迭代统计以及不依赖 KV 事件的引擎指标发布。使用 srt-slurm v2.43.4，删除重复的 served-model-name 补丁。将 Dynamo 源码固定到
ee07d6b64f9ebd2cc9bf1972be29ac7d65a4f714。该提交包含 ai-dynamo/dynamo#15628，为元数据和传输端点通告选择可路由的节点地址。router 元数据记录相同提交。缓存为空时，启动器会构建并缓存固定版本的 runtime。验证
六个配置均通过 srt-slurm v2.43.4 dry-run。矩阵生成选出六个点，拓扑、并发、镜像和集群路由设置保持一致。变更记录的历史字节完整保留，文件末尾仅有一个换行符。YAML 解析和
git diff --check均通过。固定提交的 GPU 执行仍待验证。AI 模型使用说明
原始修改：Claude Opus 5.5，依据现有提交署名。本次更新：Codex（GPT-6 系列）；运行环境未提供可核实的完整模型标识。Codex 完成源码固定、对应的 router 元数据、变更记录及本地检查。没有委派其他 agent。