Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
89b9243
fix: remove moriio connector patches after bumping to new nightly vll…
simondanielsson May 29, 2026
d2aadee
fix: move read mode envvar to flag
simondanielsson Jun 2, 2026
24555d3
fix: unpin vllm-router image
simondanielsson Jun 2, 2026
dfdbc7d
fix: update to mori_low_latency backend after rename in nightly
simondanielsson Jun 2, 2026
0c16e44
fix: pin nightlyies
simondanielsson Jun 4, 2026
3c94a6f
fix: comments and add perf-changelog.yml
simondanielsson Jun 4, 2026
3412624
Merge remote-tracking branch 'upstream' into fix/remove-vllm-disagg-p…
simondanielsson Jun 4, 2026
f3b4132
fix: pint router iamge as well
simondanielsson Jun 4, 2026
e644fde
Merge remote-tracking branch 'upstream/main' into fix/remove-vllm-dis…
simondanielsson Jun 11, 2026
2b3c89a
fix: bump vllm nightly versions
simondanielsson Jun 11, 2026
556d926
Merge remote-tracking branch 'upstream/main' into fix/remove-vllm-dis…
simondanielsson Jun 15, 2026
d450957
Merge remote-tracking branch 'upstream/main' into fix/remove-vllm-dis…
simondanielsson Jun 30, 2026
23a9f07
Merge remote-tracking branch 'upstream/main' into fix/remove-vllm-dis…
simondanielsson Jul 6, 2026
aab3883
Merge remote-tracking branch 'upstream/main' into fix/remove-vllm-dis…
simondanielsson Jul 6, 2026
510642c
fix: add ulimit nofile
simondanielsson Jul 7, 2026
ac50349
Merge remote-tracking branch 'upstream/main' into fix/remove-vllm-dis…
simondanielsson Jul 8, 2026
4395ac1
Merge branch 'main' into fix/remove-vllm-disagg-patches
chunfangamd Jul 9, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion benchmarks/multi_node/amd_utils/job.slurm
Original file line number Diff line number Diff line change
Expand Up @@ -317,7 +317,7 @@ export DOCKER_CONT_NAME="container_${ENGINE}_${SANITIZED_USER}_${MODEL_NAME}_${S
# vLLM external router container.
# NOTE: vllm/vllm-router only retains ~16 recent nightlies on Docker Hub; older
# dated tags are garbage-collected (manifest unknown)
VLLM_ROUTER_IMAGE="${VLLM_ROUTER_IMAGE:-vllm/vllm-router:nightly-20260617-e667ebb}"
VLLM_ROUTER_IMAGE="${VLLM_ROUTER_IMAGE:-vllm/vllm-router:nightly-20260629-e667ebb}"
ROUTER_CONT_NAME="router_vllm_${SANITIZED_USER}_${SLURM_JOB_ID}"
export RUN_FILE_FULL="$WS_PATH/${RUN_FILE}"

Expand Down Expand Up @@ -532,6 +532,7 @@ if [[ \"$ENGINE\" == \"vllm-disagg\" && \"$ROUTER_TYPE\" == \"vllm-router\" && \
\$DOCKER_CMD run -d \
--name \"$ROUTER_CONT_NAME\" \
--network host \
--ulimit nofile=1048576:1048576 \
-v /tmp:/run_logs \
\"$VLLM_ROUTER_IMAGE\" \
bash -lc \"mkdir -p /run_logs/slurm_job-${SLURM_JOB_ID} && exec vllm-router \
Expand Down
4 changes: 1 addition & 3 deletions configs/amd-master.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -1177,7 +1177,7 @@ dsr1-fp8-mi355x-sglang-disagg-mtp:
- "DECODE_MTP_SIZE=2"

kimik2.5-fp4-mi355x-vllm-disagg:
Comment thread
simondanielsson marked this conversation as resolved.
image: vllm/vllm-openai-rocm:nightly-bf610c2f56764e1b30bc6065f4ceace3d6e59036
image: vllm/vllm-openai-rocm:v0.24.0
model: amd/Kimi-K2.5-MXFP4
model-prefix: kimik2.5
runner: mi355x-disagg
Comment thread
simondanielsson marked this conversation as resolved.
Expand All @@ -1200,7 +1200,6 @@ kimik2.5-fp4-mi355x-vllm-disagg:
dp-attn: false
additional-settings:
- "PREFILL_NODES=1"
- "VLLM_MORIIO_CONNECTOR_READ_MODE=1"
decode:
num-worker: 2
tp: 8
Expand All @@ -1221,7 +1220,6 @@ kimik2.5-fp4-mi355x-vllm-disagg:
dp-attn: false
additional-settings:
- "PREFILL_NODES=1"
- "VLLM_MORIIO_CONNECTOR_READ_MODE=1"
decode:
num-worker: 2
tp: 8
Expand Down
7 changes: 7 additions & 0 deletions perf-changelog.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4613,6 +4613,13 @@
- "1k1k: 1 wide-EP max-throughput + 6 low-latency (DEP16/DEP32 + TP=4) topologies."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1780

- config-keys:
- kimik2.5-fp4-mi355x-vllm-disagg
description:
- "Bump image to vllm/vllm-openai-rocm:v0.24.0 (MoRI-IO fixes upstreamed in vllm#40344)"
- "Rename a2a backend mori to mori_low_latency; set read_mode via kv_connector_extra_config instead of VLLM_MORIIO_CONNECTOR_READ_MODE env var"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1585

- config-keys:
- dsr1-fp4-b200-dynamo-sglang-mtp
description:
Expand Down