From f8ae1d186619d3febd14214c7bbb7d0cb57338d0 Mon Sep 17 00:00:00 2001 From: Rohit Pujar Nagraj Date: Tue, 6 Oct 2026 10:54:06 -0700 Subject: [PATCH 1/4] chore(srt): migrate AgentX recipes to schema 2 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 将 AgentX SRT 配置迁移至 schema 2，并更新共用的 srt-slurm 版本。 Signed-off-by: Rohit Pujar Nagraj --- .../multi_node/srt-slurm-recipes/RECIPES.md | 6 +- .../srt-slurm-recipes/RECIPES_zh.md | 6 +- .../mi355x-fp4/agentx/disagg-umbp-dspark.yaml | 1 + .../h200-fp8/agentx/agg-h200-tp8-mtp-c12.yaml | 65 ++++++++++--------- .../h200-fp8/agentx/agg-h200-tp8-mtp-c16.yaml | 65 ++++++++++--------- .../h200-fp8/agentx/agg-h200-tp8-mtp-c2.yaml | 65 ++++++++++--------- .../h200-fp8/agentx/agg-h200-tp8-mtp-c4.yaml | 65 ++++++++++--------- .../h200-fp8/agentx/agg-h200-tp8-mtp-c8.yaml | 65 ++++++++++--------- .../infx/tests/launch/test_srt_config.py | 2 +- inferencex-e2e/perf-changelog.yaml | 7 ++ inferencex-e2e/utils/srt-slurm | 2 +- 11 files changed, 194 insertions(+), 155 deletions(-) diff --git a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/RECIPES.md b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/RECIPES.md index 42c3c5bb73..e1437a10a0 100644 --- a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/RECIPES.md +++ b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/RECIPES.md @@ -4,7 +4,7 @@ InferenceX owns the recipes in this directory. For every NVIDIA srt-slurm launch, the srt driver ([`infx/launch/drivers/srt/checkout.py`](../../../infx/launch/drivers/srt/checkout.py)) makes a job-local Git clone of the pinned submodule and copies this entire tree into `recipes/`. It records the actual revision in `srt-slurm-sha.txt`; power lanes copy that revision into `power-producer-sha.txt` for result validation. -The shared version is the Git submodule pointer at [`utils/srt-slurm`](../../../utils/srt-slurm), currently [v2.36.0](https://github.com/NVIDIA/srt-slurm/releases/tag/v2.36.0) (`7b5863a7837673d81403b076be219bbf18a7700f`). Update that submodule pointer when upgrading, then run the recipe and integration checks. Do not add model-specific checkout branches to launchers. +The shared version is the Git submodule pointer at [`utils/srt-slurm`](../../../utils/srt-slurm), currently [v2.43.4](https://github.com/NVIDIA/srt-slurm/releases/tag/v2.43.4) (`848f72d45b05af0fc082a9d1df49a8b4e7e61507`). Update that submodule pointer when upgrading, then run the recipe and integration checks. Do not add model-specific checkout branches to launchers. InferenceX requires srt-slurm 2.0 or newer and `schema: 2` recipes. Legacy recipe layouts are unsupported; migrate them before adding them to this tree. @@ -54,8 +54,6 @@ All referenced recipes must be checked in: srt-slurm 2 ships curated examples in Install the shared pin in an isolated environment, then use its CLI: ```bash -# Verify each supported recipe directory before rewriting it. -srtctl migrate --verify -f benchmarks/multi_node/srt-slurm-recipes/dsr1/sglang srtctl migrate --in-place -f benchmarks/multi_node/srt-slurm-recipes/dsr1/sglang # Repeat for the other model/engine directories. python -m pytest infx/tests/matrix/ -q @@ -64,7 +62,7 @@ python -m infx.matrix.generate full-sweep \ --framework dynamo-sglang dynamo-trt dynamo-vllm --multi-node ``` -Validate recipes with the exact launcher pin, including all override variants. For a path-only reorganization, compare generated matrices before and after with the path mapping applied; all other fields, including eval selection and node counts, must match. A passing local schema check does not replace the full hardware sweep and evals. +Validate recipes with the exact launcher pin, including all override variants. The current migration CLI has no `--verify`; use `srtctl dry-run` for each selected recipe after migration, supplying launcher-provided values such as `benchmark.concurrencies` through `--set` when needed. For a path-only reorganization, compare generated matrices before and after with the path mapping applied; all other fields, including eval selection and node counts, must match. A passing local schema check does not replace the full hardware sweep and evals. The initial migration also resolves compatibility issues that `srtctl migrate` cannot fix itself: diff --git a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/RECIPES_zh.md b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/RECIPES_zh.md index 41d431845f..85586ff012 100644 --- a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/RECIPES_zh.md +++ b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/RECIPES_zh.md @@ -4,7 +4,7 @@ InferenceX 负责维护本目录中的配置。每次 NVIDIA srt-slurm 启动时，srt 驱动（[`infx/launch/drivers/srt/checkout.py`](../../../infx/launch/drivers/srt/checkout.py)）都会为作业创建固定版本子模块的本地 Git 克隆，并将整个目录复制到 `recipes/`。它将实际提交记录到 `srt-slurm-sha.txt`；功耗测试路径还会将其复制到 `power-producer-sha.txt`，供结果校验使用。 -统一版本由 [`utils/srt-slurm`](../../../utils/srt-slurm) 的 Git 子模块指针指定，目前为 [v2.36.0](https://github.com/NVIDIA/srt-slurm/releases/tag/v2.36.0)（`7b5863a7837673d81403b076be219bbf18a7700f`）。升级时更新该子模块指针，然后运行配置和集成检查。不要在启动器中新增按模型选择检出版本的分支。 +统一版本由 [`utils/srt-slurm`](../../../utils/srt-slurm) 的 Git 子模块指针指定，目前为 [v2.43.4](https://github.com/NVIDIA/srt-slurm/releases/tag/v2.43.4)（`848f72d45b05af0fc082a9d1df49a8b4e7e61507`）。升级时更新该子模块指针，然后运行配置和集成检查。不要在启动器中新增按模型选择检出版本的分支。 InferenceX 要求 srt-slurm 2.0 或更新版本，且配置必须声明 `schema: 2`。不支持旧版配置结构；加入本目录前必须先完成迁移。 @@ -54,8 +54,6 @@ TileRT 使用固定版本的上游 srt-slurm 子模块。配置指定 `roles.pre 在隔离环境中安装统一版本，然后使用其 CLI： ```bash -# 重写前先验证每个受支持的配置目录。 -srtctl migrate --verify -f benchmarks/multi_node/srt-slurm-recipes/dsr1/sglang srtctl migrate --in-place -f benchmarks/multi_node/srt-slurm-recipes/dsr1/sglang # 对其他模型/引擎目录重复执行。 python -m pytest infx/tests/matrix/ -q @@ -64,7 +62,7 @@ python -m infx.matrix.generate full-sweep \ --framework dynamo-sglang dynamo-trt dynamo-vllm --multi-node ``` -使用启动器指定的确切提交验证配置，包括全部覆盖变体。仅调整路径时，应按路径映射比较变更前后的生成矩阵；其他字段（包括评估选择和节点数）必须完全一致。本地配置校验通过不能替代完整硬件扫描和准确性评估。 +使用启动器指定的确切提交验证配置，包括全部覆盖变体。当前迁移 CLI 已不提供 `--verify`；迁移后对每个选中的配置运行 `srtctl dry-run`，必要时通过 `--set` 提供启动器注入的值，例如 `benchmark.concurrencies`。仅调整路径时，应按路径映射比较变更前后的生成矩阵；其他字段（包括评估选择和节点数）必须完全一致。本地配置校验通过不能替代完整硬件扫描和准确性评估。 本次迁移还修复了 `srtctl migrate` 无法自动处理的兼容性问题： diff --git a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/dsv4/sglang/mi355x-fp4/agentx/disagg-umbp-dspark.yaml b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/dsv4/sglang/mi355x-fp4/agentx/disagg-umbp-dspark.yaml index 1a2c163ed5..eb46cf57a9 100644 --- a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/dsv4/sglang/mi355x-fp4/agentx/disagg-umbp-dspark.yaml +++ b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/dsv4/sglang/mi355x-fp4/agentx/disagg-umbp-dspark.yaml @@ -2,6 +2,7 @@ # with DSpark. Every arm offloads prefill KV to UMBP as a direct external store # for the unified radix tree (no host cache tier), backed by a 1.5 TB # hugepage DRAM tier on the prefill node. +schema: 2 base: name: mi355x-dsv4-pro-0813-agentx-umbp model: diff --git a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/glm5.2/sglang/h200-fp8/agentx/agg-h200-tp8-mtp-c12.yaml b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/glm5.2/sglang/h200-fp8/agentx/agg-h200-tp8-mtp-c12.yaml index 69ddd78a1c..70790ad8f4 100644 --- a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/glm5.2/sglang/h200-fp8/agentx/agg-h200-tp8-mtp-c12.yaml +++ b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/glm5.2/sglang/h200-fp8/agentx/agg-h200-tp8-mtp-c12.yaml @@ -1,3 +1,4 @@ +schema: 2 name: agg-h200-tp8-mtp-c12 # Aggregated TP8 GLM-5.2 AgentX topology on one 8-GPU H200 node. MTP uses the @@ -27,14 +28,17 @@ dynamo: resources: gpu_type: h200 gpus_per_node: 8 - gpus_per_agg: 8 - agg_nodes: 1 - agg_workers: 1 - -infra: - etcd_nats_dedicated_node: false - nats_max_payload_mb: 64 - +services: + - name: etcd + type: etcd + placement: + node: infra + - name: nats + type: nats + placement: + node: infra + options: + max_payload_mb: 64 frontend: type: dynamo nginx_session_affinity: true @@ -49,10 +53,30 @@ frontend: active-prefill-tokens-threshold: None active-prefill-tokens-threshold-frac: None -backend: - type: sglang - sglang_config: - aggregated: +engine: sglang +roles: + agg: + nodes: 1 + workers: 1 + + gpus: 8 + env: + HF_HOME: /hf_hub_cache + HF_HUB_CACHE: /hf_hub_cache + PIP_BREAK_SYSTEM_PACKAGES: "1" + PYTHONUNBUFFERED: "1" + SGLANG_OPT_USE_TOPK_V2: "1" + SGLANG_TIMEOUT_KEEP_ALIVE: "900" + SGLANG_ENABLE_THINKING: "1" + SGLANG_REASONING_EFFORT: max + SGLANG_ENABLE_UNIFIED_RADIX_TREE: "1" + SGLANG_HICACHE_DEBUG_LOG: "1" + SGLANG_HICACHE_DEBUG_SAMPLE_RATE: "16384" + SGLANG_SIMULATE_ACC_LEN: "2.99" + SGLANG_SIMULATE_ACC_METHOD: match-expected + SGLANG_SIMULATE_ACC_TOKEN_MODE: real-draft-token + + args: served-model-name: zai-org/GLM-5.2-FP8 trust-remote-code: true tool-call-parser: glm47 @@ -83,22 +107,6 @@ backend: stream-interval: 60 enable-metrics: true enable-cache-report: true - aggregated_environment: - HF_HOME: /hf_hub_cache - HF_HUB_CACHE: /hf_hub_cache - PIP_BREAK_SYSTEM_PACKAGES: "1" - PYTHONUNBUFFERED: "1" - SGLANG_OPT_USE_TOPK_V2: "1" - SGLANG_TIMEOUT_KEEP_ALIVE: "900" - SGLANG_ENABLE_THINKING: "1" - SGLANG_REASONING_EFFORT: max - SGLANG_ENABLE_UNIFIED_RADIX_TREE: "1" - SGLANG_HICACHE_DEBUG_LOG: "1" - SGLANG_HICACHE_DEBUG_SAMPLE_RATE: "16384" - SGLANG_SIMULATE_ACC_LEN: "2.99" - SGLANG_SIMULATE_ACC_METHOD: match-expected - SGLANG_SIMULATE_ACC_TOKEN_MODE: real-draft-token - health_check: max_attempts: 1440 interval_seconds: 10 @@ -118,7 +126,6 @@ telemetry: benchmark: type: custom - use_chat_template: true command: bash /infmax-workspace/benchmarks/srt_agentic.sh env: INFMAX_CONTAINER_WORKSPACE: /infmax-workspace diff --git a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/glm5.2/sglang/h200-fp8/agentx/agg-h200-tp8-mtp-c16.yaml b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/glm5.2/sglang/h200-fp8/agentx/agg-h200-tp8-mtp-c16.yaml index b8884e2bc6..6c3066a6f8 100644 --- a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/glm5.2/sglang/h200-fp8/agentx/agg-h200-tp8-mtp-c16.yaml +++ b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/glm5.2/sglang/h200-fp8/agentx/agg-h200-tp8-mtp-c16.yaml @@ -1,3 +1,4 @@ +schema: 2 name: agg-h200-tp8-mtp-c16 # Aggregated TP8 GLM-5.2 AgentX topology on one 8-GPU H200 node. MTP uses the @@ -27,14 +28,17 @@ dynamo: resources: gpu_type: h200 gpus_per_node: 8 - gpus_per_agg: 8 - agg_nodes: 1 - agg_workers: 1 - -infra: - etcd_nats_dedicated_node: false - nats_max_payload_mb: 64 - +services: + - name: etcd + type: etcd + placement: + node: infra + - name: nats + type: nats + placement: + node: infra + options: + max_payload_mb: 64 frontend: type: dynamo nginx_session_affinity: true @@ -49,10 +53,30 @@ frontend: active-prefill-tokens-threshold: None active-prefill-tokens-threshold-frac: None -backend: - type: sglang - sglang_config: - aggregated: +engine: sglang +roles: + agg: + nodes: 1 + workers: 1 + + gpus: 8 + env: + HF_HOME: /hf_hub_cache + HF_HUB_CACHE: /hf_hub_cache + PIP_BREAK_SYSTEM_PACKAGES: "1" + PYTHONUNBUFFERED: "1" + SGLANG_OPT_USE_TOPK_V2: "1" + SGLANG_TIMEOUT_KEEP_ALIVE: "900" + SGLANG_ENABLE_THINKING: "1" + SGLANG_REASONING_EFFORT: max + SGLANG_ENABLE_UNIFIED_RADIX_TREE: "1" + SGLANG_HICACHE_DEBUG_LOG: "1" + SGLANG_HICACHE_DEBUG_SAMPLE_RATE: "16384" + SGLANG_SIMULATE_ACC_LEN: "2.99" + SGLANG_SIMULATE_ACC_METHOD: match-expected + SGLANG_SIMULATE_ACC_TOKEN_MODE: real-draft-token + + args: served-model-name: zai-org/GLM-5.2-FP8 trust-remote-code: true tool-call-parser: glm47 @@ -83,22 +107,6 @@ backend: stream-interval: 60 enable-metrics: true enable-cache-report: true - aggregated_environment: - HF_HOME: /hf_hub_cache - HF_HUB_CACHE: /hf_hub_cache - PIP_BREAK_SYSTEM_PACKAGES: "1" - PYTHONUNBUFFERED: "1" - SGLANG_OPT_USE_TOPK_V2: "1" - SGLANG_TIMEOUT_KEEP_ALIVE: "900" - SGLANG_ENABLE_THINKING: "1" - SGLANG_REASONING_EFFORT: max - SGLANG_ENABLE_UNIFIED_RADIX_TREE: "1" - SGLANG_HICACHE_DEBUG_LOG: "1" - SGLANG_HICACHE_DEBUG_SAMPLE_RATE: "16384" - SGLANG_SIMULATE_ACC_LEN: "2.99" - SGLANG_SIMULATE_ACC_METHOD: match-expected - SGLANG_SIMULATE_ACC_TOKEN_MODE: real-draft-token - health_check: max_attempts: 1440 interval_seconds: 10 @@ -118,7 +126,6 @@ telemetry: benchmark: type: custom - use_chat_template: true command: bash /infmax-workspace/benchmarks/srt_agentic.sh env: INFMAX_CONTAINER_WORKSPACE: /infmax-workspace diff --git a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/glm5.2/sglang/h200-fp8/agentx/agg-h200-tp8-mtp-c2.yaml b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/glm5.2/sglang/h200-fp8/agentx/agg-h200-tp8-mtp-c2.yaml index 05511596d1..57dc61f31b 100644 --- a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/glm5.2/sglang/h200-fp8/agentx/agg-h200-tp8-mtp-c2.yaml +++ b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/glm5.2/sglang/h200-fp8/agentx/agg-h200-tp8-mtp-c2.yaml @@ -1,3 +1,4 @@ +schema: 2 name: agg-h200-tp8-mtp-c2 # Aggregated TP8 GLM-5.2 AgentX topology on one 8-GPU H200 node. MTP uses the @@ -27,14 +28,17 @@ dynamo: resources: gpu_type: h200 gpus_per_node: 8 - gpus_per_agg: 8 - agg_nodes: 1 - agg_workers: 1 - -infra: - etcd_nats_dedicated_node: false - nats_max_payload_mb: 64 - +services: + - name: etcd + type: etcd + placement: + node: infra + - name: nats + type: nats + placement: + node: infra + options: + max_payload_mb: 64 frontend: type: dynamo nginx_session_affinity: true @@ -49,10 +53,30 @@ frontend: active-prefill-tokens-threshold: None active-prefill-tokens-threshold-frac: None -backend: - type: sglang - sglang_config: - aggregated: +engine: sglang +roles: + agg: + nodes: 1 + workers: 1 + + gpus: 8 + env: + HF_HOME: /hf_hub_cache + HF_HUB_CACHE: /hf_hub_cache + PIP_BREAK_SYSTEM_PACKAGES: "1" + PYTHONUNBUFFERED: "1" + SGLANG_OPT_USE_TOPK_V2: "1" + SGLANG_TIMEOUT_KEEP_ALIVE: "900" + SGLANG_ENABLE_THINKING: "1" + SGLANG_REASONING_EFFORT: max + SGLANG_ENABLE_UNIFIED_RADIX_TREE: "1" + SGLANG_HICACHE_DEBUG_LOG: "1" + SGLANG_HICACHE_DEBUG_SAMPLE_RATE: "16384" + SGLANG_SIMULATE_ACC_LEN: "2.99" + SGLANG_SIMULATE_ACC_METHOD: match-expected + SGLANG_SIMULATE_ACC_TOKEN_MODE: real-draft-token + + args: served-model-name: zai-org/GLM-5.2-FP8 trust-remote-code: true tool-call-parser: glm47 @@ -83,22 +107,6 @@ backend: stream-interval: 60 enable-metrics: true enable-cache-report: true - aggregated_environment: - HF_HOME: /hf_hub_cache - HF_HUB_CACHE: /hf_hub_cache - PIP_BREAK_SYSTEM_PACKAGES: "1" - PYTHONUNBUFFERED: "1" - SGLANG_OPT_USE_TOPK_V2: "1" - SGLANG_TIMEOUT_KEEP_ALIVE: "900" - SGLANG_ENABLE_THINKING: "1" - SGLANG_REASONING_EFFORT: max - SGLANG_ENABLE_UNIFIED_RADIX_TREE: "1" - SGLANG_HICACHE_DEBUG_LOG: "1" - SGLANG_HICACHE_DEBUG_SAMPLE_RATE: "16384" - SGLANG_SIMULATE_ACC_LEN: "2.99" - SGLANG_SIMULATE_ACC_METHOD: match-expected - SGLANG_SIMULATE_ACC_TOKEN_MODE: real-draft-token - health_check: max_attempts: 1440 interval_seconds: 10 @@ -118,7 +126,6 @@ telemetry: benchmark: type: custom - use_chat_template: true command: bash /infmax-workspace/benchmarks/srt_agentic.sh env: INFMAX_CONTAINER_WORKSPACE: /infmax-workspace diff --git a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/glm5.2/sglang/h200-fp8/agentx/agg-h200-tp8-mtp-c4.yaml b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/glm5.2/sglang/h200-fp8/agentx/agg-h200-tp8-mtp-c4.yaml index de35237e7d..410a1c3b88 100644 --- a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/glm5.2/sglang/h200-fp8/agentx/agg-h200-tp8-mtp-c4.yaml +++ b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/glm5.2/sglang/h200-fp8/agentx/agg-h200-tp8-mtp-c4.yaml @@ -1,3 +1,4 @@ +schema: 2 name: agg-h200-tp8-mtp-c4 # Aggregated TP8 GLM-5.2 AgentX topology on one 8-GPU H200 node. MTP uses the @@ -27,14 +28,17 @@ dynamo: resources: gpu_type: h200 gpus_per_node: 8 - gpus_per_agg: 8 - agg_nodes: 1 - agg_workers: 1 - -infra: - etcd_nats_dedicated_node: false - nats_max_payload_mb: 64 - +services: + - name: etcd + type: etcd + placement: + node: infra + - name: nats + type: nats + placement: + node: infra + options: + max_payload_mb: 64 frontend: type: dynamo nginx_session_affinity: true @@ -49,10 +53,30 @@ frontend: active-prefill-tokens-threshold: None active-prefill-tokens-threshold-frac: None -backend: - type: sglang - sglang_config: - aggregated: +engine: sglang +roles: + agg: + nodes: 1 + workers: 1 + + gpus: 8 + env: + HF_HOME: /hf_hub_cache + HF_HUB_CACHE: /hf_hub_cache + PIP_BREAK_SYSTEM_PACKAGES: "1" + PYTHONUNBUFFERED: "1" + SGLANG_OPT_USE_TOPK_V2: "1" + SGLANG_TIMEOUT_KEEP_ALIVE: "900" + SGLANG_ENABLE_THINKING: "1" + SGLANG_REASONING_EFFORT: max + SGLANG_ENABLE_UNIFIED_RADIX_TREE: "1" + SGLANG_HICACHE_DEBUG_LOG: "1" + SGLANG_HICACHE_DEBUG_SAMPLE_RATE: "16384" + SGLANG_SIMULATE_ACC_LEN: "2.99" + SGLANG_SIMULATE_ACC_METHOD: match-expected + SGLANG_SIMULATE_ACC_TOKEN_MODE: real-draft-token + + args: served-model-name: zai-org/GLM-5.2-FP8 trust-remote-code: true tool-call-parser: glm47 @@ -83,22 +107,6 @@ backend: stream-interval: 60 enable-metrics: true enable-cache-report: true - aggregated_environment: - HF_HOME: /hf_hub_cache - HF_HUB_CACHE: /hf_hub_cache - PIP_BREAK_SYSTEM_PACKAGES: "1" - PYTHONUNBUFFERED: "1" - SGLANG_OPT_USE_TOPK_V2: "1" - SGLANG_TIMEOUT_KEEP_ALIVE: "900" - SGLANG_ENABLE_THINKING: "1" - SGLANG_REASONING_EFFORT: max - SGLANG_ENABLE_UNIFIED_RADIX_TREE: "1" - SGLANG_HICACHE_DEBUG_LOG: "1" - SGLANG_HICACHE_DEBUG_SAMPLE_RATE: "16384" - SGLANG_SIMULATE_ACC_LEN: "2.99" - SGLANG_SIMULATE_ACC_METHOD: match-expected - SGLANG_SIMULATE_ACC_TOKEN_MODE: real-draft-token - health_check: max_attempts: 1440 interval_seconds: 10 @@ -118,7 +126,6 @@ telemetry: benchmark: type: custom - use_chat_template: true command: bash /infmax-workspace/benchmarks/srt_agentic.sh env: INFMAX_CONTAINER_WORKSPACE: /infmax-workspace diff --git a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/glm5.2/sglang/h200-fp8/agentx/agg-h200-tp8-mtp-c8.yaml b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/glm5.2/sglang/h200-fp8/agentx/agg-h200-tp8-mtp-c8.yaml index 3e733200a2..c6b20cf7ba 100644 --- a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/glm5.2/sglang/h200-fp8/agentx/agg-h200-tp8-mtp-c8.yaml +++ b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/glm5.2/sglang/h200-fp8/agentx/agg-h200-tp8-mtp-c8.yaml @@ -1,3 +1,4 @@ +schema: 2 name: agg-h200-tp8-mtp-c8 # Aggregated TP8 GLM-5.2 AgentX topology on one 8-GPU H200 node. MTP uses the @@ -27,14 +28,17 @@ dynamo: resources: gpu_type: h200 gpus_per_node: 8 - gpus_per_agg: 8 - agg_nodes: 1 - agg_workers: 1 - -infra: - etcd_nats_dedicated_node: false - nats_max_payload_mb: 64 - +services: + - name: etcd + type: etcd + placement: + node: infra + - name: nats + type: nats + placement: + node: infra + options: + max_payload_mb: 64 frontend: type: dynamo nginx_session_affinity: true @@ -49,10 +53,30 @@ frontend: active-prefill-tokens-threshold: None active-prefill-tokens-threshold-frac: None -backend: - type: sglang - sglang_config: - aggregated: +engine: sglang +roles: + agg: + nodes: 1 + workers: 1 + + gpus: 8 + env: + HF_HOME: /hf_hub_cache + HF_HUB_CACHE: /hf_hub_cache + PIP_BREAK_SYSTEM_PACKAGES: "1" + PYTHONUNBUFFERED: "1" + SGLANG_OPT_USE_TOPK_V2: "1" + SGLANG_TIMEOUT_KEEP_ALIVE: "900" + SGLANG_ENABLE_THINKING: "1" + SGLANG_REASONING_EFFORT: max + SGLANG_ENABLE_UNIFIED_RADIX_TREE: "1" + SGLANG_HICACHE_DEBUG_LOG: "1" + SGLANG_HICACHE_DEBUG_SAMPLE_RATE: "16384" + SGLANG_SIMULATE_ACC_LEN: "2.99" + SGLANG_SIMULATE_ACC_METHOD: match-expected + SGLANG_SIMULATE_ACC_TOKEN_MODE: real-draft-token + + args: served-model-name: zai-org/GLM-5.2-FP8 trust-remote-code: true tool-call-parser: glm47 @@ -83,22 +107,6 @@ backend: stream-interval: 60 enable-metrics: true enable-cache-report: true - aggregated_environment: - HF_HOME: /hf_hub_cache - HF_HUB_CACHE: /hf_hub_cache - PIP_BREAK_SYSTEM_PACKAGES: "1" - PYTHONUNBUFFERED: "1" - SGLANG_OPT_USE_TOPK_V2: "1" - SGLANG_TIMEOUT_KEEP_ALIVE: "900" - SGLANG_ENABLE_THINKING: "1" - SGLANG_REASONING_EFFORT: max - SGLANG_ENABLE_UNIFIED_RADIX_TREE: "1" - SGLANG_HICACHE_DEBUG_LOG: "1" - SGLANG_HICACHE_DEBUG_SAMPLE_RATE: "16384" - SGLANG_SIMULATE_ACC_LEN: "2.99" - SGLANG_SIMULATE_ACC_METHOD: match-expected - SGLANG_SIMULATE_ACC_TOKEN_MODE: real-draft-token - health_check: max_attempts: 1440 interval_seconds: 10 @@ -118,7 +126,6 @@ telemetry: benchmark: type: custom - use_chat_template: true command: bash /infmax-workspace/benchmarks/srt_agentic.sh env: INFMAX_CONTAINER_WORKSPACE: /infmax-workspace diff --git a/inferencex-e2e/infx/tests/launch/test_srt_config.py b/inferencex-e2e/infx/tests/launch/test_srt_config.py index 193281657e..273ae27422 100644 --- a/inferencex-e2e/infx/tests/launch/test_srt_config.py +++ b/inferencex-e2e/infx/tests/launch/test_srt_config.py @@ -179,7 +179,7 @@ def test_selected_partition_overrides_inherited_and_point_environment(tmp_path): def test_multinode_jobs_wait_at_least_the_health_floor_for_their_server(tmp_path, health, effective): recipe = tmp_path / "recipes/r.yaml" recipe.parent.mkdir() - recipe.write_text(yaml.safe_dump({"name": "r", **({"health_check": health} if health else {})})) + recipe.write_text(yaml.safe_dump({"schema": 2, "name": "r", **({"health_check": health} if health else {})})) prepare_recipe(tmp_path, "recipes/r.yaml", "job", None, None) diff --git a/inferencex-e2e/perf-changelog.yaml b/inferencex-e2e/perf-changelog.yaml index 5031b626c3..4961a1f8ce 100644 --- a/inferencex-e2e/perf-changelog.yaml +++ b/inferencex-e2e/perf-changelog.yaml @@ -9327,3 +9327,10 @@ description: - "Enable SGLANG_OPT_USE_TOPK_V2 (false -> true) so the DSA indexer uses the top-k v2 kernel." pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3724 + +- config-keys: + - "*" + description: + - "Update the shared srt-slurm pin from v2.39.1 to v2.43.4, including staged Dynamo wheel dependency resolution." + - "Migrate five GLM-5.2 H200 AgentX recipes and the DeepSeek-V4 MI355X AgentX override bundle to schema 2 without changing their selected topology or serving settings." + pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX diff --git a/inferencex-e2e/utils/srt-slurm b/inferencex-e2e/utils/srt-slurm index 098e15aca8..848f72d45b 160000 --- a/inferencex-e2e/utils/srt-slurm +++ b/inferencex-e2e/utils/srt-slurm @@ -1 +1 @@ -Subproject commit 098e15aca8504d4747c6f62c7a845e178b7f4b10 +Subproject commit 848f72d45b05af0fc082a9d1df49a8b4e7e61507 From fc29dd1c7fbc26fc0efa2635d5edb85f363fd946 Mon Sep 17 00:00:00 2001 From: Rohit Pujar Nagraj Date: Tue, 6 Oct 2026 10:54:51 -0700 Subject: [PATCH 2/4] chore(srt): link schema migration changelog entry MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 将 schema 迁移的变更记录条目关联到此 PR。 Signed-off-by: Rohit Pujar Nagraj --- inferencex-e2e/perf-changelog.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/inferencex-e2e/perf-changelog.yaml b/inferencex-e2e/perf-changelog.yaml index 4961a1f8ce..7fe221dc5c 100644 --- a/inferencex-e2e/perf-changelog.yaml +++ b/inferencex-e2e/perf-changelog.yaml @@ -9333,4 +9333,4 @@ description: - "Update the shared srt-slurm pin from v2.39.1 to v2.43.4, including staged Dynamo wheel dependency resolution." - "Migrate five GLM-5.2 H200 AgentX recipes and the DeepSeek-V4 MI355X AgentX override bundle to schema 2 without changing their selected topology or serving settings." - pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX + pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3755 From 99e79e87dae5312769a4909dbec3b5f687e69ae4 Mon Sep 17 00:00:00 2001 From: Rohit Pujar Nagraj Date: Tue, 6 Oct 2026 11:55:56 -0700 Subject: [PATCH 3/4] test(evals): stabilize archive test IDs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 为归档校验测试指定固定参数 ID，避免并行收集时因 gzip 时间戳不同而失败。 Signed-off-by: Rohit Pujar Nagraj --- inferencex-e2e/infx/tests/evals/test_kimi_verifier_archive.py | 1 + 1 file changed, 1 insertion(+) diff --git a/inferencex-e2e/infx/tests/evals/test_kimi_verifier_archive.py b/inferencex-e2e/infx/tests/evals/test_kimi_verifier_archive.py index 70d05dc9f3..ff0aff8a6b 100644 --- a/inferencex-e2e/infx/tests/evals/test_kimi_verifier_archive.py +++ b/inferencex-e2e/infx/tests/evals/test_kimi_verifier_archive.py @@ -144,6 +144,7 @@ def _escaping_member() -> tarfile.TarInfo: ), (_archive(extra=_escaping_member()), None, "unsafe archive member path"), ], + ids=("sha256-mismatch", "missing-required-file", "unsafe-member-path"), ) def test_rejected_archive_extracts_nothing( tmp_path: Path, From d0620e4fb1b11d3361c71ca186b483fd4d6dafc0 Mon Sep 17 00:00:00 2001 From: cquil11 <60715037+cquil11@users.noreply.github.com> Date: Tue, 6 Oct 2026 21:43:27 +0000 Subject: [PATCH 4/4] chore(srt): migrate Kimi-K3 AgentX benchmark placement for srt-slurm v2.43.4 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replace benchmark.client_placement with benchmark.placement.node in eleven Kimi-K3 vLLM GB200/GB300 AgentX recipes, which the v2.43.4 pin rejects as a v1 key. 将十一个 Kimi-K3 vLLM GB200/GB300 AgentX 配置中的 benchmark.client_placement 替换为 benchmark.placement.node；v2.43.4 固定版本会将前者视为 v1 字段并拒绝加载。 --- .../gb200-fp4/agentx/agg-dcp16-dspark4-maxseq2-mooncake.yaml | 3 ++- .../vllm/gb200-fp4/agentx/agg-dcp16-nospec-mooncake.yaml | 3 ++- .../kimik3/vllm/gb200-fp4/agentx/agg-tp8pp2-mooncake-c16.yaml | 3 ++- .../kimik3/vllm/gb200-fp4/agentx/agg-tp8pp2-mooncake-c32.yaml | 3 ++- .../kimik3/vllm/gb200-fp4/agentx/agg-tp8pp2-mooncake-c48.yaml | 3 ++- .../kimik3/vllm/gb200-fp4/agentx/agg-tp8pp2-mooncake-c72.yaml | 3 ++- .../kimik3/vllm/gb200-fp4/agentx/agg-tp8pp2-mooncake-c96.yaml | 3 ++- .../agentx/disagg-1p1d-dcp8-dcp8-dspark4-mooncake.yaml | 3 ++- .../agentx/disagg-1p2d-dcp8-dcp8-dspark4-mooncake.yaml | 3 ++- .../agentx/disagg-1p3d-dcp8-dcp8-dspark4-mooncake.yaml | 3 ++- .../agentx/disagg-1p3d-dcp8-dcp8-dspark7-mooncake.yaml | 3 ++- inferencex-e2e/perf-changelog.yaml | 1 + 12 files changed, 23 insertions(+), 11 deletions(-) diff --git a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb200-fp4/agentx/agg-dcp16-dspark4-maxseq2-mooncake.yaml b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb200-fp4/agentx/agg-dcp16-dspark4-maxseq2-mooncake.yaml index 8bb943ac50..5d511c84f1 100644 --- a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb200-fp4/agentx/agg-dcp16-dspark4-maxseq2-mooncake.yaml +++ b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb200-fp4/agentx/agg-dcp16-dspark4-maxseq2-mooncake.yaml @@ -158,7 +158,6 @@ telemetry: benchmark: type: custom - client_placement: head concurrencies: [1, 2, 4, 8, 16] command: bash /infmax-workspace/benchmarks/srt_agentic.sh env: @@ -176,3 +175,5 @@ benchmark: AIPERF_DATASET_MMAP_CACHE_DIR: "/aiperf_mmap_cache" HF_HUB_CACHE: "/hf_hub_cache" WEKA_LOADER_OVERRIDE: "semianalysis_cc_traces_weka_062126" + placement: + node: head diff --git a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb200-fp4/agentx/agg-dcp16-nospec-mooncake.yaml b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb200-fp4/agentx/agg-dcp16-nospec-mooncake.yaml index f97597d20c..95b951994e 100644 --- a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb200-fp4/agentx/agg-dcp16-nospec-mooncake.yaml +++ b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb200-fp4/agentx/agg-dcp16-nospec-mooncake.yaml @@ -154,7 +154,6 @@ telemetry: benchmark: type: custom - client_placement: head concurrencies: [8, 40, 48] command: bash /infmax-workspace/benchmarks/srt_agentic.sh env: @@ -172,3 +171,5 @@ benchmark: AIPERF_DATASET_MMAP_CACHE_DIR: "/aiperf_mmap_cache" HF_HUB_CACHE: "/hf_hub_cache" WEKA_LOADER_OVERRIDE: "semianalysis_cc_traces_weka_062126" + placement: + node: head diff --git a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb200-fp4/agentx/agg-tp8pp2-mooncake-c16.yaml b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb200-fp4/agentx/agg-tp8pp2-mooncake-c16.yaml index 0f2b295757..86ff822121 100644 --- a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb200-fp4/agentx/agg-tp8pp2-mooncake-c16.yaml +++ b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb200-fp4/agentx/agg-tp8pp2-mooncake-c16.yaml @@ -127,7 +127,6 @@ telemetry: benchmark: type: custom - client_placement: head concurrencies: [16] command: bash /infmax-workspace/benchmarks/srt_agentic.sh env: @@ -145,3 +144,5 @@ benchmark: AIPERF_DATASET_MMAP_CACHE_DIR: "/aiperf_mmap_cache" HF_HUB_CACHE: "/hf_hub_cache" WEKA_LOADER_OVERRIDE: "semianalysis_cc_traces_weka_062126" + placement: + node: head diff --git a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb200-fp4/agentx/agg-tp8pp2-mooncake-c32.yaml b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb200-fp4/agentx/agg-tp8pp2-mooncake-c32.yaml index ab1000d783..dfa17adfb6 100644 --- a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb200-fp4/agentx/agg-tp8pp2-mooncake-c32.yaml +++ b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb200-fp4/agentx/agg-tp8pp2-mooncake-c32.yaml @@ -127,7 +127,6 @@ telemetry: benchmark: type: custom - client_placement: head concurrencies: [32] command: bash /infmax-workspace/benchmarks/srt_agentic.sh env: @@ -145,3 +144,5 @@ benchmark: AIPERF_DATASET_MMAP_CACHE_DIR: "/aiperf_mmap_cache" HF_HUB_CACHE: "/hf_hub_cache" WEKA_LOADER_OVERRIDE: "semianalysis_cc_traces_weka_062126" + placement: + node: head diff --git a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb200-fp4/agentx/agg-tp8pp2-mooncake-c48.yaml b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb200-fp4/agentx/agg-tp8pp2-mooncake-c48.yaml index eb408121bc..25a20d1f04 100644 --- a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb200-fp4/agentx/agg-tp8pp2-mooncake-c48.yaml +++ b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb200-fp4/agentx/agg-tp8pp2-mooncake-c48.yaml @@ -126,7 +126,6 @@ telemetry: benchmark: type: custom - client_placement: head concurrencies: [48] command: bash /infmax-workspace/benchmarks/srt_agentic.sh env: @@ -144,3 +143,5 @@ benchmark: AIPERF_DATASET_MMAP_CACHE_DIR: "/aiperf_mmap_cache" HF_HUB_CACHE: "/hf_hub_cache" WEKA_LOADER_OVERRIDE: "semianalysis_cc_traces_weka_062126" + placement: + node: head diff --git a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb200-fp4/agentx/agg-tp8pp2-mooncake-c72.yaml b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb200-fp4/agentx/agg-tp8pp2-mooncake-c72.yaml index a77fdb6739..637b3d787c 100644 --- a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb200-fp4/agentx/agg-tp8pp2-mooncake-c72.yaml +++ b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb200-fp4/agentx/agg-tp8pp2-mooncake-c72.yaml @@ -126,7 +126,6 @@ telemetry: benchmark: type: custom - client_placement: head concurrencies: [72] command: bash /infmax-workspace/benchmarks/srt_agentic.sh env: @@ -144,3 +143,5 @@ benchmark: AIPERF_DATASET_MMAP_CACHE_DIR: "/aiperf_mmap_cache" HF_HUB_CACHE: "/hf_hub_cache" WEKA_LOADER_OVERRIDE: "semianalysis_cc_traces_weka_062126" + placement: + node: head diff --git a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb200-fp4/agentx/agg-tp8pp2-mooncake-c96.yaml b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb200-fp4/agentx/agg-tp8pp2-mooncake-c96.yaml index e2ee0bc73b..224f99b1dc 100644 --- a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb200-fp4/agentx/agg-tp8pp2-mooncake-c96.yaml +++ b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb200-fp4/agentx/agg-tp8pp2-mooncake-c96.yaml @@ -126,7 +126,6 @@ telemetry: benchmark: type: custom - client_placement: head concurrencies: [96] command: bash /infmax-workspace/benchmarks/srt_agentic.sh env: @@ -144,3 +143,5 @@ benchmark: AIPERF_DATASET_MMAP_CACHE_DIR: "/aiperf_mmap_cache" HF_HUB_CACHE: "/hf_hub_cache" WEKA_LOADER_OVERRIDE: "semianalysis_cc_traces_weka_062126" + placement: + node: head diff --git a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb300-fp4/agentx/disagg-1p1d-dcp8-dcp8-dspark4-mooncake.yaml b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb300-fp4/agentx/disagg-1p1d-dcp8-dcp8-dspark4-mooncake.yaml index 4a7c6f3da3..bd9fe13954 100644 --- a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb300-fp4/agentx/disagg-1p1d-dcp8-dcp8-dspark4-mooncake.yaml +++ b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb300-fp4/agentx/disagg-1p1d-dcp8-dcp8-dspark4-mooncake.yaml @@ -187,7 +187,6 @@ telemetry: benchmark: type: custom - client_placement: head concurrencies: [48, 52, 56] command: bash /infmax-workspace/benchmarks/srt_agentic.sh env: @@ -203,3 +202,5 @@ benchmark: AIPERF_DATASET_MMAP_CACHE_DIR: "/aiperf_mmap_cache" HF_HUB_CACHE: "/hf_hub_cache" WEKA_LOADER_OVERRIDE: "semianalysis_cc_traces_weka_062126" + placement: + node: head diff --git a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb300-fp4/agentx/disagg-1p2d-dcp8-dcp8-dspark4-mooncake.yaml b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb300-fp4/agentx/disagg-1p2d-dcp8-dcp8-dspark4-mooncake.yaml index 8072d8e35e..1ca55de7c9 100644 --- a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb300-fp4/agentx/disagg-1p2d-dcp8-dcp8-dspark4-mooncake.yaml +++ b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb300-fp4/agentx/disagg-1p2d-dcp8-dcp8-dspark4-mooncake.yaml @@ -187,7 +187,6 @@ telemetry: benchmark: type: custom - client_placement: head concurrencies: [32, 48, 64] command: bash /infmax-workspace/benchmarks/srt_agentic.sh env: @@ -203,3 +202,5 @@ benchmark: AIPERF_DATASET_MMAP_CACHE_DIR: "/aiperf_mmap_cache" HF_HUB_CACHE: "/hf_hub_cache" WEKA_LOADER_OVERRIDE: "semianalysis_cc_traces_weka_062126" + placement: + node: head diff --git a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb300-fp4/agentx/disagg-1p3d-dcp8-dcp8-dspark4-mooncake.yaml b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb300-fp4/agentx/disagg-1p3d-dcp8-dcp8-dspark4-mooncake.yaml index 5c10e7e694..4046e9b36f 100644 --- a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb300-fp4/agentx/disagg-1p3d-dcp8-dcp8-dspark4-mooncake.yaml +++ b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb300-fp4/agentx/disagg-1p3d-dcp8-dcp8-dspark4-mooncake.yaml @@ -187,7 +187,6 @@ telemetry: benchmark: type: custom - client_placement: head concurrencies: [32, 48] command: bash /infmax-workspace/benchmarks/srt_agentic.sh env: @@ -203,3 +202,5 @@ benchmark: AIPERF_DATASET_MMAP_CACHE_DIR: "/aiperf_mmap_cache" HF_HUB_CACHE: "/hf_hub_cache" WEKA_LOADER_OVERRIDE: "semianalysis_cc_traces_weka_062126" + placement: + node: head diff --git a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb300-fp4/agentx/disagg-1p3d-dcp8-dcp8-dspark7-mooncake.yaml b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb300-fp4/agentx/disagg-1p3d-dcp8-dcp8-dspark7-mooncake.yaml index 7fe377cf7d..24f6a8060c 100644 --- a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb300-fp4/agentx/disagg-1p3d-dcp8-dcp8-dspark7-mooncake.yaml +++ b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/gb300-fp4/agentx/disagg-1p3d-dcp8-dcp8-dspark7-mooncake.yaml @@ -187,7 +187,6 @@ telemetry: benchmark: type: custom - client_placement: head concurrencies: [1] command: bash /infmax-workspace/benchmarks/srt_agentic.sh env: @@ -203,3 +202,5 @@ benchmark: AIPERF_DATASET_MMAP_CACHE_DIR: "/aiperf_mmap_cache" HF_HUB_CACHE: "/hf_hub_cache" WEKA_LOADER_OVERRIDE: "semianalysis_cc_traces_weka_062126" + placement: + node: head diff --git a/inferencex-e2e/perf-changelog.yaml b/inferencex-e2e/perf-changelog.yaml index 8a7eba83d5..7c7b27d536 100644 --- a/inferencex-e2e/perf-changelog.yaml +++ b/inferencex-e2e/perf-changelog.yaml @@ -9333,4 +9333,5 @@ description: - "Update the shared srt-slurm pin from v2.39.1 to v2.43.4, including staged Dynamo wheel dependency resolution." - "Migrate five GLM-5.2 H200 AgentX recipes and the DeepSeek-V4 MI355X AgentX override bundle to schema 2; rely on automatic golden acceptance injection for the GLM-5.2 recipes." + - "Migrate eleven Kimi-K3 vLLM GB200/GB300 AgentX recipes from benchmark.client_placement to benchmark.placement.node for the new pin." pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3755