Skip to content

Merge branch 'main' into ahmadki/fix_2770 #8767

Merge branch 'main' into ahmadki/fix_2770

Merge branch 'main' into ahmadki/fix_2770 #8767

Triggered via push February 26, 2026 16:56
Status Success
Total duration 1h 36m 10s
Artifacts 32

cicd-main.yml

on: push
is-not-external-contributor
20s
is-not-external-contributor
merge-queue-notification
merge-queue-notification
pre-flight  /  pre-flight
12s
pre-flight / pre-flight
cicd-wait-in-queue
3s
cicd-wait-in-queue
cicd-container-build
48s
cicd-container-build
cicd-parse-downstream-testing
8s
cicd-parse-downstream-testing
cicd-parse-unit-tests
5s
cicd-parse-unit-tests
cicd-mbridge-testing
1h 31m
cicd-mbridge-testing
Matrix: cicd-unit-tests-latest
cicd-parse-integration-tests
11s
cicd-parse-integration-tests
Matrix: cicd-integration-tests-latest
Nemo_CICD_Test
9s
Nemo_CICD_Test
Coverage_Fake
0s
Coverage_Fake
Matrix: Coverage
cleanup-taint-node
0s
cleanup-taint-node
Fit to window
Zoom out
Zoom in

Deployment protection rules

Reviewers, timers, and other rules protecting deployments in this run
Event Environments Comment
ko3n1g
approved Feb 26, 2026
test

Automatically approved by queue manager

Annotations

35 errors and 3 warnings
cicd-container-build
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
tests/unit_tests/test_model_configs.py - latest
DEPRECATION: --no-python-version-warning is deprecated. pip 25.1 will enforce this behaviour change. A possible replacement is to remove the flag as it's a no-op. Discussion can be found at https://github.com/pypa/pip/issues/13154
tests/unit_tests/test_model_configs.py - latest
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
tests/unit_tests/data/**/*.py - latest
DEPRECATION: --no-python-version-warning is deprecated. pip 25.1 will enforce this behaviour change. A possible replacement is to remove the flag as it's a no-op. Discussion can be found at https://github.com/pypa/pip/issues/13154
tests/unit_tests/data/**/*.py - latest
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
tests/unit_tests/test_fp8_param.py - latest
DEPRECATION: --no-python-version-warning is deprecated. pip 25.1 will enforce this behaviour change. A possible replacement is to remove the flag as it's a no-op. Discussion can be found at https://github.com/pypa/pip/issues/13154
tests/unit_tests/test_fp8_param.py - latest
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
tests/unit_tests/dist_checkpointing/models/**/*.py - latest
DEPRECATION: --no-python-version-warning is deprecated. pip 25.1 will enforce this behaviour change. A possible replacement is to remove the flag as it's a no-op. Discussion can be found at https://github.com/pypa/pip/issues/13154
tests/unit_tests/dist_checkpointing/models/**/*.py - latest
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
tests/unit_tests/dist_checkpointing/models/test_moe_experts.py - latest
DEPRECATION: --no-python-version-warning is deprecated. pip 25.1 will enforce this behaviour change. A possible replacement is to remove the flag as it's a no-op. Discussion can be found at https://github.com/pypa/pip/issues/13154
tests/unit_tests/dist_checkpointing/models/test_moe_experts.py - latest
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
tests/unit_tests/models/**/*.py - latest
DEPRECATION: --no-python-version-warning is deprecated. pip 25.1 will enforce this behaviour change. A possible replacement is to remove the flag as it's a no-op. Discussion can be found at https://github.com/pypa/pip/issues/13154
tests/unit_tests/models/**/*.py - latest
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
tests/unit_tests/pipeline_parallel/**/*.py - latest
DEPRECATION: --no-python-version-warning is deprecated. pip 25.1 will enforce this behaviour change. A possible replacement is to remove the flag as it's a no-op. Discussion can be found at https://github.com/pypa/pip/issues/13154
tests/unit_tests/pipeline_parallel/**/*.py - latest
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
tests/unit_tests/distributed/megatron_fsdp/**/*.py - latest
DEPRECATION: --no-python-version-warning is deprecated. pip 25.1 will enforce this behaviour change. A possible replacement is to remove the flag as it's a no-op. Discussion can be found at https://github.com/pypa/pip/issues/13154
tests/unit_tests/distributed/megatron_fsdp/**/*.py - latest
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
tests/unit_tests/dist_checkpointing/test_optimizer.py - latest
DEPRECATION: --no-python-version-warning is deprecated. pip 25.1 will enforce this behaviour change. A possible replacement is to remove the flag as it's a no-op. Discussion can be found at https://github.com/pypa/pip/issues/13154
tests/unit_tests/dist_checkpointing/test_optimizer.py - latest
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
tests/unit_tests/transformer/moe/**/*.py - latest
DEPRECATION: --no-python-version-warning is deprecated. pip 25.1 will enforce this behaviour change. A possible replacement is to remove the flag as it's a no-op. Discussion can be found at https://github.com/pypa/pip/issues/13154
tests/unit_tests/transformer/moe/**/*.py - latest
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
tests/unit_tests/dist_checkpointing/**/*.py - latest
DEPRECATION: --no-python-version-warning is deprecated. pip 25.1 will enforce this behaviour change. A possible replacement is to remove the flag as it's a no-op. Discussion can be found at https://github.com/pypa/pip/issues/13154
tests/unit_tests/dist_checkpointing/**/*.py - latest
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
tests/unit_tests/transformer/**/*.py - latest
DEPRECATION: --no-python-version-warning is deprecated. pip 25.1 will enforce this behaviour change. A possible replacement is to remove the flag as it's a no-op. Discussion can be found at https://github.com/pypa/pip/issues/13154
tests/unit_tests/transformer/**/*.py - latest
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
tests/unit_tests/**/*.py - latest
DEPRECATION: --no-python-version-warning is deprecated. pip 25.1 will enforce this behaviour change. A possible replacement is to remove the flag as it's a no-op. Discussion can be found at https://github.com/pypa/pip/issues/13154
tests/unit_tests/**/*.py - latest
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
gpt/gpt3_mcore_te_tp2_pp1_gdn - latest
DEPRECATION: --no-python-version-warning is deprecated. pip 25.1 will enforce this behaviour change. A possible replacement is to remove the flag as it's a no-op. Discussion can be found at https://github.com/pypa/pip/issues/13154
gpt/gpt3_mcore_te_tp2_pp1_gdn - latest
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
gpt/gpt3_mcore_te_tp1_pp4_vp1_dist_optimizer_overlap_grad_reduce_param_gather_overlap_optimizer - latest
DEPRECATION: --no-python-version-warning is deprecated. pip 25.1 will enforce this behaviour change. A possible replacement is to remove the flag as it's a no-op. Discussion can be found at https://github.com/pypa/pip/issues/13154
gpt/gpt3_mcore_te_tp1_pp4_vp1_dist_optimizer_overlap_grad_reduce_param_gather_overlap_optimizer - latest
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
gpt/gpt3_mcore_te_tp4_pp1_resume_torch_dist_dist_optimizer_overlap_grad_reduce_param_gather - latest
DEPRECATION: --no-python-version-warning is deprecated. pip 25.1 will enforce this behaviour change. A possible replacement is to remove the flag as it's a no-op. Discussion can be found at https://github.com/pypa/pip/issues/13154
gpt/gpt3_mcore_te_tp4_pp1_resume_torch_dist_dist_optimizer_overlap_grad_reduce_param_gather - latest
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
moe/gpt3_moe_mcore_te_tp4_ep2_etp2_pp2_resume_torch_dist_dist_optimizer - latest
DEPRECATION: --no-python-version-warning is deprecated. pip 25.1 will enforce this behaviour change. A possible replacement is to remove the flag as it's a no-op. Discussion can be found at https://github.com/pypa/pip/issues/13154
moe/gpt3_moe_mcore_te_tp4_ep2_etp2_pp2_resume_torch_dist_dist_optimizer - latest
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
Default value for global ARG results in an empty or invalid base image name: docker/Dockerfile.ci.dev#L5
InvalidDefaultArgInFrom: Default value for ARG ${FROM_IMAGE_NAME} results in empty or invalid base image name More info: https://docs.docker.com/go/dockerfile/rule/invalid-default-arg-in-from/
The 'as' keyword should match the case of the 'from' keyword: docker/Dockerfile.ci.dev#L84
FromAsCasing: 'as' and 'FROM' keywords' casing do not match More info: https://docs.docker.com/go/dockerfile/rule/from-as-casing/
The 'as' keyword should match the case of the 'from' keyword: docker/Dockerfile.ci.dev#L5
FromAsCasing: 'as' and 'FROM' keywords' casing do not match More info: https://docs.docker.com/go/dockerfile/rule/from-as-casing/

Artifacts

Produced during runtime
Name Size Digest
NVIDIA~Megatron-LM~PEYCH9.dockerbuild
120 KB
sha256:2a8c41ccb1673753017f6a34b12bdc7f1cf5a46f827c95ea9f5c58fd15e9bdfa
coverage-unit-test-22452294548-0b418636-bbd3-49d5-b157-be900786c455
139 KB
sha256:0474741022f42c69cf39730c7d81835d5b28fc2d2c683d7327dc6651fe61049d
coverage-unit-test-22452294548-32094648-2e20-4379-81e5-7a1bf1ca2014
138 KB
sha256:9519191eed4be11d0e948428441d6392fb3e268c0924a5b836e570a9b64888c4
coverage-unit-test-22452294548-35ee4ff4-f04b-4246-a6d0-eb5cfbb87069
142 KB
sha256:ec77b73695e2145498144ae54265f3e2e0fc2070d35a0740fd7e26c480ba0aa7
coverage-unit-test-22452294548-596bef28-f5b9-4c54-985c-34853d33d74e
137 KB
sha256:40d9844651b6258fef1107af8193762c3a7408306e3f1005ac71887fafd43277
coverage-unit-test-22452294548-769493f1-c669-490a-9543-b3207948a999
139 KB
sha256:2a32383a4507a8903d6f95349d03d3b03fe44291d4bc6d3d49aa489b84a076f2
coverage-unit-test-22452294548-789af5ca-cc33-41e1-8439-0ce9cfb74c9c
138 KB
sha256:ff55c02482922bca931a85a00248739880b2a8b3c840b345967c8ed28dbf9e64
coverage-unit-test-22452294548-9b2b4196-203b-427d-a7f1-1cffcd992a3d
138 KB
sha256:acdc7d11da1ff0df46fa64f8792e9ce82788963229bc4ba5acd001e55eea2c9c
coverage-unit-test-22452294548-bbad72e4-19b1-4bbb-be09-d17ff4a1090c
139 KB
sha256:aa7a8a62a28de81d1e4626bb50e98d7f667afbea6a704ee12b075a9335534e92
coverage-unit-test-22452294548-c1bc1d09-ba2a-40b9-97a7-626f8959cb2b
142 KB
sha256:b86b146242205071174bda3504fc08006f0a69b1e845e56d271d4864bf978f4a
coverage-unit-test-22452294548-c3490013-f48a-4924-9ea3-1588f6621d3a
141 KB
sha256:a1812ad190ed76a56344a47716f8d370eb426190847ed04db7db6698a8932a32
coverage-unit-test-22452294548-c58ed13e-6e22-426f-818d-ad4a31475789
140 KB
sha256:78645f64f4e337ab73e0c5e911a98fdfa509fb741a51f2ceb2c82d641342ca7b
coverage-unit-test-22452294548-e35a1ac0-d6cf-487a-861e-e4344595254c
149 KB
sha256:f660deac7f04a71074deb5037bc266349f378a7bb83ea63de381ec3d802c1290
coverage-unit-test-22452294548-e45de1a2-7cee-438b-b70a-baef9e10b634
135 KB
sha256:34f87387d3152ce8e71efdd88eae055ecc2c99767402dc1fad1d65edf5d67ef2
coverage-unit-test-aggregated
31.2 KB
sha256:4d56e71bb5782d2900edf1a17e9e3c3e66ce6546c794597fcf56628da17ad912
logs-gpt3_mcore_te_tp1_pp4_vp1_dist_optimizer_overlap_grad_reduce_param_gather_overlap_optimizer-22452294548-fd8f0d68-2d5d-45c9-89ca-7b4cb13f0150
270 KB
sha256:2f325fbe55c2566433868dcbd54248435d4460121ce588ec0ad0d12b9df563a2
logs-gpt3_mcore_te_tp2_pp1_gdn-22452294548-bd6dbd1e-9bdd-437e-a25d-219bfc26ef63
81.9 KB
sha256:fdcf5cddafc1250dd1eceba0da6a6a7c4f8e877bf92c627d46ec0dbc6d0af1c1
logs-gpt3_mcore_te_tp4_pp1_resume_torch_dist_dist_optimizer_overlap_grad_reduce_param_gather-22452294548-8426cff0-fa36-4319-92c8-69adf765765f
594 KB
sha256:d3ca75fabfc91275f90304596f5e1135721f505a4c7073d808fab58a944a0a3d
logs-gpt3_moe_mcore_te_tp4_ep2_etp2_pp2_resume_torch_dist_dist_optimizer-22452294548-6c9aac20-e7f5-4d86-9c77-145b337c2955
616 KB
sha256:3d027d3014ca210aa6e8b855fe1070fb2044857d99609f9053c4ad2ea36c1a7a
logs-tests-unit_tests-----.py-22452294548-5e6dd811-82cf-45ad-aaff-ba12df058db7
763 KB
sha256:d74335a7527d447abdac195b7cf4b55ca1184a5336fba1af0a1de97a7a611594
logs-tests-unit_tests-data-----.py-22452294548-9fa2c6e3-4571-47a3-9bbd-8cb45e608d7f
815 KB
sha256:977a73596dc0c17a4493a722677cf8e106c3e3d485eed7ab65ecfd289789060b
logs-tests-unit_tests-dist_checkpointing-----.py-22452294548-8e462dd9-426f-4257-83b2-51858fc98c87
530 KB
sha256:ec7be98370a4c15512029ca37e34f59f72e5f373d48189b0ca412da80234ea10
logs-tests-unit_tests-dist_checkpointing-models-----.py-22452294548-80bc7c58-4f6f-453b-bcd9-88aeef0de4db
83.3 KB
sha256:5ee3c7380dcd9480d0ebb0de7607f540782647c93decd99233d30b9e96967629
logs-tests-unit_tests-dist_checkpointing-models-test_moe_experts.py-22452294548-f61531c6-6576-4275-af70-e018e9131bfc
110 KB
sha256:c171d7dd2f25a7e048597090778cfe06b7db8e8fe54d50655eccd702b4dbfd44
logs-tests-unit_tests-dist_checkpointing-test_optimizer.py-22452294548-151f2d92-6112-45bd-bdfc-8860d5cb94e1
186 KB
sha256:5df6aa12cb1c7074dc714776ad47dd2581961bd26036d3a241450b9c628fc73f
logs-tests-unit_tests-distributed-megatron_fsdp-----.py-22452294548-2d4ea66b-bcaf-48ae-ad80-df349a51db1f
166 KB
sha256:15eb75b8e53918a0a621c4eea9426aec8d4b91ef2d357196e4ab8b8e9df268bf
logs-tests-unit_tests-models-----.py-22452294548-0ea51bf2-fe82-4a68-aac0-5a874cd44e47
117 KB
sha256:04d30a5a58af84a9605f508edf2660684c5d113d1c1dedd6125d7d0db9bc103d
logs-tests-unit_tests-pipeline_parallel-----.py-22452294548-b717cbb9-80c5-4d7f-a38f-e019d53dab0f
120 KB
sha256:fe6e44f83b8eb0e484ce784f0d15f4213823dd7895d50afdac4741bba92cefdf
logs-tests-unit_tests-test_fp8_param.py-22452294548-68e552df-9be8-495f-bcf6-9d1152c9f0a0
177 KB
sha256:535544fe54e087ad472cf1ca55c7f34a61fed08d927029ac665f64d597728be1
logs-tests-unit_tests-test_model_configs.py-22452294548-a463c39a-d52b-47a9-91b6-94c4ae4d684a
82.4 KB
sha256:017a90b309b019492faba72f1bb6667a0d732bb1d96747540c51c89bfbce3c60
logs-tests-unit_tests-transformer-----.py-22452294548-c11175f4-8ee6-435a-abb6-b6fd4219ea39
564 KB
sha256:6a6b2912f1e781f0b80690b1e738fc9e38f3148e181d3522b61345c54c27e2ed
logs-tests-unit_tests-transformer-moe-----.py-22452294548-c7823a72-4989-4e0e-9bd4-109b02bed224
256 KB
sha256:7e7c70aff6c9ead8b9eebd0a84bd680608c768ffcdb1bbdf53888bad1762e063