feature/mad-profiling - #207

Closed

mkuznet1 wants to merge 154 commits into

ROCm:mad-rcclfrom

mkuznet1:feature/mad-profiling

Commits on Oct 25, 2023

Initial commit
amathews-amd
authored
f83a846

Commits on Jul 15, 2024

Push prototype version of the code
gargrahul
committed
603dcaa

Commits on Sep 3, 2024

Update default timeout to 4 hrs
gargrahul
committed
68382ad

Commits on Sep 4, 2024

unified docker vllm (#3)

authored
58abaa0

Commits on Sep 5, 2024

Merge pull request #2 from gargrahul/gargrahul/increasetimeout
gargrahul
authored
43dd5e3
Use ROCm fork for transformers and add model caching for gpt2 (#1)
gargrahul
authored
b9d92e4

Commits on Sep 10, 2024

Added gitignore for project (#5)
coketaste
authored
33c8a40
Added gitattributes (#7)
coketaste
authored
3018614

Commits on Sep 18, 2024

skip torchaudio to avoid torch reinstall (#8)
gargrahul
authored
f020708

Commits on Sep 24, 2024

Update the input args (#6)

coketaste
and
gargrahul
authored
e744d4c
Add CODEOWNERS (#9)
gargrahul
authored
cd291f6

Commits on Sep 27, 2024

Update CODEOWNERS (#12)
dgaliffiAMD
authored
0f69b15

Commits on Oct 28, 2024

Added data.json for data provider (#4)

coketaste
and
gargrahul
authored
6afa99b

Commits on Oct 30, 2024

unified docker 2nd rel (#15)
seungrokj
authored
6e757ae

Commits on Nov 6, 2024

Support multiple results for perf reporting (#14)

gargrahul
and
coketaste
authored
8f4dcbb
Updated the timeout assignment through models.json (#16)

gargrahul
and
coketaste
authored
9430c01

Commits on Nov 17, 2024

Fixed the reading error of query gpu count (#17)

gargrahul
and
Stephen Shao
authored
76d936c

Commits on Nov 27, 2024

add gpu check for installed but not working (#18)

gargrahul
and
younseojava
authored
f42f39f

Commits on Feb 5, 2025

vllm rocm 6.3.1 release (#20)
seungrokj
authored
4a14b80

Commits on Feb 7, 2025

update vllm rocm 6.3.1 release (#21)
seungrokj
authored
9e4228b

Commits on Feb 11, 2025

Fixed the str of mount docker paths (#22)
coketaste
authored
28cd379

Commits on Feb 21, 2025

Add pyt-training docker v25.3 (#23)
gargrahul
authored
ccdb8da

Commits on Feb 28, 2025

Change docs links to prod URLs instead of preview (#24)
peterjunpark
authored
ccbac3c

Commits on Mar 12, 2025

unified vllm docker with v0.7.3 (#26)

seungrokj
and
peterjunpark
authored
8cb178a

Commits on Mar 13, 2025

pyt training v25.4 update (#27)

authored
607878f

Commits on Mar 14, 2025

unified vllm docker readme with v0.7.3 (pyt_vllm_llama-3.1-8b_fp8) (#28)

seungrokj
and
gargrahul
authored
7836dd8

Commits on Mar 21, 2025

Add megatron-lm docker v25.4 (#29)
gargrahul
authored
6a90ec3

Commits on Mar 28, 2025

Add README for jax-training:maxtext-v25.4 (#30)
gargrahul
authored
1cccc73

Commits on Apr 3, 2025

Update CODEOWNERS (#32)
gargrahul
authored
2f64bd1

Commits on Apr 10, 2025

Remove nvidia dockerfile (#34)
gargrahul
authored
e468b58

Commits on Apr 15, 2025

pyt training v25.5 update (#35)
gargrahul
authored
b61e61a

Commits on Apr 18, 2025

Update for vllm 0.8.3 inference docker (#36)
gargrahul
authored
2469ba0

Commits on Apr 21, 2025

Add chai-1 inference (#37)
gargrahul
authored
dbfaaac
Add CLIP inference model (#39)
michaelzhang-ai
authored
6df039c

Commits on Apr 24, 2025

Add Qwq32b (#40)
gargrahul
authored
793cbb7

Commits on Apr 28, 2025

Add jax-maxtext v25.5 (#41)
gargrahul
authored
6c9161d

Commits on May 2, 2025

Add Mosaic-ML MPT-30B training model (#42)

gargrahul
and
EricKing626
authored
104f2ec

Commits on May 7, 2025

Fix MAD_RUNTIME_NGPUS (#45)

authored
a68a971

Commits on May 9, 2025

Add vllm serving benchmark (#46)
gargrahul
authored
36d0e6b

Commits on May 12, 2025

CLIP update: move the py scripts in scripts folder (#47)
michaelzhang-ai
authored
e1e2aee

Commits on May 19, 2025

Use rocm/pytorch latest for chai1 inference (#48)
shikamd123
authored
113cf02

Commits on May 21, 2025

Update for vllm inference version 0.8.5 (#49)
gargrahul
authored
06eb6a1
[vllm online serving] adding ignore-eos (#50)
gargrahul
authored
e417bb0

Commits on May 22, 2025

Add megatron-lm training docker v25.5 (#51)
gargrahul
authored
e38777a

Commits on May 28, 2025

Add Falcon-180b model (#52)
gargrahul
authored
9cb2bae

Commits on May 30, 2025

Add Microsoft/Phi-4 model (#53)
gargrahul
authored
9a1e3f1

Commits on Jun 2, 2025

Remove enable-chunked-prefill (#54)
gargrahul
authored
3e89c20
Update path to use profiling scripts in VLLM (#55)
gargrahul
authored
fe06429
vLLM 05/27 release (#56)

gargrahul
and
Rohan138
authored
847c256

Commits on Jun 9, 2025

Add Mochi single gpu model (#57)
gargrahul
authored
6bb9819

Commits on Jun 20, 2025

vllm 0610 update (#58)
gargrahul
authored
2a743c9
Pytorch Training Docker v25.6 (#59)
gargrahul
authored
151b30f

Commits on Jun 25, 2025

Fixed the issue of multiple results checking (#61)
coketaste
authored
5ae5d06

Commits on Jul 1, 2025

Add the logic to parse and load docker build args (#62)
coketaste
authored
cb64202

Commits on Jul 8, 2025

Integrate Wan2.1 in MAD (#60)

gargrahul
and
coketaste
committed
f22b8be

Commits on Jul 9, 2025

Vllm 0624 update (#64)
gargrahul
authored
7e16c04

Commits on Jul 17, 2025

Vllm 0715 update (#66)
gargrahul
authored
e96648e

Commits on Jul 18, 2025

Megatron-LM v25.6 (#67)
gargrahul
authored
5ecb701

Commits on Jul 22, 2025

Add Janus-Pro-7B Model (#68)
gargrahul
authored
669b24a

Commits on Jul 23, 2025

Add madengine usage for MAD (#63)

coketaste
and
gargrahul
authored
89b67a9

Commits on Jul 24, 2025

Install madengine from source (#70)
gargrahul
authored
c93c14f

Commits on Jul 25, 2025

Add Deepseek-r1-distill-qwen-32b with SGLANG (#65)
gargrahul
authored
67ec9b1

Commits on Jul 29, 2025

Update --tags option to space-separated values from comma-separated (#72)
samuel-bohman
authored
109bde0

Commits on Aug 12, 2025

Add hy video (#69)

gargrahul
and
Emmanuel0612
authored
bfaebb8

Commits on Aug 14, 2025

VLLM refactor and 08/12 update (#76)
gargrahul
authored
95c6921

Commits on Aug 19, 2025

Fix throughput benchmark in vllm inf (#77)
Rohan138
authored
0625be2

Commits on Aug 22, 2025

Primus / Megatron-LM v25.7 (#78)
gargrahul
authored
0c4bd8c

Commits on Sep 3, 2025

Sglang Disagg Distributed Inference Release1 (#79)
gargrahul
authored
0f6d812

Commits on Sep 5, 2025

Pytorch v25.7 Docker update (#80)
gargrahul
authored
f0186e9

Commits on Sep 8, 2025

correct disagg name (#81)

gargrahul
and
raviguptaamd
authored
48d19a1
Jax maxtext training v25.7 jax0.6.0 (#82)
gargrahul
authored
c6e1e5f

Commits on Sep 9, 2025

Added Qwen3-30B-A3B to MAD, using single GPU (#83)

gargrahul
and
Rohan138
authored
782dd6d

Commits on Sep 11, 2025

Vllm 0909 update (#85)
gargrahul
authored
5d4b681

Commits on Sep 16, 2025

Add ncf training (#87)
gargrahul
authored
8da4b20
Pytorch training v25.8 update (#88)
gargrahul
authored
313ec81
Update sglang disagg to ROCm7.0 (#89)
gargrahul
authored
7329905
Add sdxl lora (#84)
gargrahul
authored
36d6af3

Commits on Sep 17, 2025

JAX-Maxtext training v25.7 JAX 0.6.0 patch (#90)
gargrahul
authored
f5ca83a

Commits on Sep 19, 2025

Megatron-LM training v25.8 Update (#92)
gargrahul
authored
92bcfaa

Commits on Sep 20, 2025

Add AITER experimental flag to docs note (#91)
Rohan138
authored
481c2f2

Commits on Oct 1, 2025

Replace huggingface-cli with hf (#95)
kchong12
authored
45c582d

Commits on Oct 8, 2025

vLLM 10/06 update (#96)

gargrahul
and
Rohan138
authored
50ae014

Commits on Oct 11, 2025

fix float4 and gpt-oss dtypes (#97)
Rohan138
authored
2aa6dc5

Commits on Oct 14, 2025

Update vLLM README for 10/04 (#98)
Rohan138
authored
60730cd

Commits on Oct 16, 2025

Pytorch training v25.9 (#100)
gargrahul
authored
4c075e3
Megatron-LM training v25.9 (#101)
gargrahul
authored
cbdb308
fix gpt-oss and qwen moe configs (#99)
Rohan138
authored
158ec85

Commits on Oct 17, 2025

MaxText Training v25.9 (#102)
gargrahul
authored
4accefd

Commits on Oct 28, 2025

Add training and inference tag (#103)
gargrahul
authored
1eb7fef

Commits on Nov 4, 2025

Vllm 10/24 update (#104)
gargrahul
authored
aef3fc9

Commits on Nov 5, 2025

Update docker to 10/24 release (#105)
gargrahul
authored
16d9d69

Commits on Nov 18, 2025

pytorch-xdit:v25.10 release (#106)

gargrahul
and
nsakkine
authored
0d95ac9
Update VLLM readme perf link (#107)
gargrahul
authored
80f5d8e

Commits on Nov 20, 2025

fix unified attention typo in vllm script (#108)
Rohan138
authored
636c888

Commits on Dec 4, 2025

Primus Pytorch Megatron-LM v25.10 with dual-architecture (MI300X/MI355X) image (#111)
gargrahul
authored
47544e7

Commits on Dec 8, 2025

pytorch-xdit:v25.11 release (#110)
gargrahul
authored
1077a44

Commits on Dec 13, 2025

Update training tags (#114)
gargrahul
authored
3880857
add missing training tag (#115)
gargrahul
authored
54b1385

Commits on Dec 17, 2025

vllm 1205 update (#113)
gargrahul
authored
8294c5c
Correct vllm dockername in readme (#116)
gargrahul
authored
c98a8ff
pytorch-xdit:v25.12 release (#117)
gargrahul
authored
f12c05a

Commits on Dec 18, 2025

jax-maxtext training v25.11 (#118)
gargrahul
authored
7be34e3

Commits on Dec 24, 2025

pytorch-xdit:v25.13 release (#120)
gargrahul
authored
aad7dc7

Commits on Dec 25, 2025

Pytorch/Megatron-LM training v25.11 (#121)
gargrahul
authored
393ec06

Commits on Jan 9, 2026

Add zip install for NCF (#123)
gargrahul
authored
3cad4a3
Fix pyt_mpt30b_training PYTORCH_VERSION (#122)
gargrahul
authored
326f9df

Commits on Jan 15, 2026

vLLM Refactor (#124)
gargrahul
authored
28e2458

Commits on Jan 29, 2026

Primus v26.1 (#127)
gargrahul
authored
8815f24

Commits on Feb 6, 2026

JAX MaxText v26.1 (#129)
gargrahul
authored
d282482

Commits on Feb 11, 2026

Update SGLang PD Disagg with sgl router and updated benchmarking (#128)

gargrahul
and
etemadiamd
authored
18490d8
pytorch-xdit:v26.1 release (#130)
gargrahul
authored
e624c09

Commits on Feb 23, 2026

jax-maxtext-v26.2 (#131)
gargrahul
authored
4d0d819

Commits on Feb 25, 2026

[REL2-Distinf-NIXL-VLLM] vLLM PD Disaggregation with RIXL (#132)
raviguptaamd
authored
4094c27

Commits on Feb 27, 2026

Add vLLM Router support for PD Disaggregation (#133)
raviguptaamd
authored
2b9e9b5
pytorch xdit:v26.2 release (#134)
gargrahul
authored
fa777f1

Commits on Mar 17, 2026

pytorch-xdit:v26.3 release (#135)
gargrahul
authored
6acc36c

Commits on Mar 19, 2026

Commits on Mar 20, 2026

remove deprecated --overwrite_output_dir from HuggingFace model scripts (#136)
leconcio
authored
718728f

Commits on Mar 23, 2026

vllm 0318 update (#138)
gargrahul
authored
0b753d4

Commits on Mar 25, 2026

Primus 26.2 update (#139)
gargrahul
authored
db20e9d

Commits on Apr 1, 2026

updated docker base container and set PYTHONPATH env for sglang_disagg_inf (#140)

basemam
and
Basem Barakat
authored
d499e3c

Commits on Apr 8, 2026

Commits on Apr 15, 2026

[DistInf] Unified vLLM disaggregated PD inference: Default, MoRI EP, and DeepEP modes (#143)
raviguptaamd
authored
a83eb96

Commits on Apr 24, 2026

Commits on Apr 28, 2026

Enable SGLang disagg with MoRI-io & MoRI-EP (#154)
lcskrishna
authored
f164b5f

Commits on May 11, 2026

pytorch-xdit:v26.5 release (#155)
gargrahul
authored
b1a4072

Commits on May 18, 2026

JAX MaxText v26.3 (#156)
gargrahul
authored
883aa8c

Commits on May 19, 2026

Add ttft back to CSV (#158)
gargrahul
authored
edde311

Commits on May 20, 2026

Run accuracy after bench serve on all models (#157)

gargrahul
and
Rohan138
authored
a4d47ab

Commits on May 23, 2026

Primus v26.3 (#159)

authored
1b4ec50

Commits on Jun 2, 2026

[sglang_disagg] mori IO optimization and proxy server relocation (#161)

authored
f7c8fa6

Commits on Jun 9, 2026

Pin pandas for NCF (#163)
gargrahul
authored
dea5967

Commits on Jun 10, 2026

[DistInf][SGLang] Enable WideEP/LargeEP for SGLang PD Disagg. (#164)
lcskrishna
authored
2ed7eca

Commits on Jun 16, 2026

JAX MaxText v26.4 (#165)
gargrahul
authored
5446600

Commits on Jun 17, 2026

Update pyt_chai1_inference for use main branch (#166)
jvadakat
authored
d4bd572

Commits on Jun 22, 2026

Commits on Jun 23, 2026

Update pyt_chai1_inference package required list (#170)
jvadakat
authored
cc60691

Commits on Jun 24, 2026

madengine new models cards supporting sglan_disagg, vllm_disagg and large_ep_benchmark (#167)

authored
188092c

Commits on Jun 30, 2026

Primus v26.4 (#172)

gargrahul
and
GeneDer
authored
33c13df

Commits on Jul 7, 2026

vllm_dissag: unify 3 launchers into one two-axis driver + models.yaml (#171)

raviguptaamd
and
claude
authored
d538f56

Commits on Jul 23, 2026

Jax maxtext training v26.5 (#180)
gargrahul
authored
b95d24b

Commits on Jul 27, 2026

Add new vLLM models e.g. Kimi K2, DSV4, GLM (#179)

gargrahul
and
Rohan138
authored
c132a35

Commits on Jul 29, 2026

Kimi-K3 (2.8T MoE) inference support on MI350X/MI355X (#186)

authored
a20c885

Commits on Aug 1, 2026

Primus v26.5 (#187)

authored
7a8ae96

Commits on Aug 3, 2026

Update pyt_chai1_inference docker file (#188)
jvadakat
authored
f7e2560

Commits on Aug 5, 2026

Add multi-node scaleout benchmarks (Primus Megatron-LM, SGLang disaggregated) and the mad-slurm-multinode skill (#194)

authored
688828b

Commits on Aug 13, 2026

sglang-disagg: fix QP creation on Thor2 and make overlay stages optional (#200)
mkuznet1
authored
6f9ea4c

Commits on Aug 17, 2026