fix(locks): fixing onnx runtime cuda problem by knowicki-nvidia · Pull Request #162 · ai-dynamo/aitune

knowicki-nvidia · 2026-11-03T07:32:46Z

new modelopt requires new onnxruntime - PyPi has mixed CUDA version 1.24 cu12 1.30 cu13
new uv version - lock conflict fix

Summary by CodeRabbit

Dependency Updates
- PyTorch and Torch-TensorRT minimum versions now vary by Python version, with separate requirements for Python 3.14.
- PyTorch 2.8–2.13 extras, and Torch-TensorRT in the 2.8–2.12 extras, are limited to Python versions below 3.14.
- CUDA 12 ONNX Runtime GPU packages now use the nightly package index and are limited to Python versions below 3.14 on non-ARM64 platforms.

Signed-off-by: Konrad Nowicki <knowicki@nvidia.com>

…ing_onnx_runtime_cuda_problem

Signed-off-by: Konrad Nowicki <knowicki@nvidia.com>

coderabbitai · 2026-11-03T07:33:04Z

No actionable comments were generated in the recent review.

Walkthrough

The dependency declarations now select Torch and Torch-TensorRT version floors by Python version. The CUDA 12 ONNX Runtime package has Python and platform restrictions and uses a nightly index. The CUDA 12 compatibility workflow updates its extra arguments and exclusion list.

Changes: Dependency declarations

Layer / File(s)	Summary
Python-specific Torch requirements `pyproject.toml`	The base Torch and Torch-TensorRT requirements use different version floors below Python 3.14 and on Python 3.14. The torch28 through torch213 extras restrict Torch requirements to Python below 3.14. The torch-tensorrt requirements in torch28 through torch212 gain the same restriction.
CUDA 12 ONNX Runtime source `pyproject.toml`	The CUDA 12 `onnxruntime-gpu` extra is limited to Python below 3.14 and non-`aarch64` platforms. Its package source changes to the `ort-cuda-12-nightly` index, which is added with its feed URL.
CUDA 12 compatibility workflow `.github/workflows/dispatch.compatibility.yml`	The CUDA 12 matrix entry adds `cupy-cuda12x` to `uv_extra_args`. The install step adds `cupy-cuda13x` to `cu12-excludes.txt`; `tensorrt` remains excluded.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Other

Possibly related PRs

ai-dynamo/aitune#159: Changes Torch and Torch-TensorRT version constraints by Python version in the same project file.
ai-dynamo/aitune#126: Changes the uv version and lock-generation commands related to dependency resolution.

Suggested labels: review-effort/medium

Suggested reviewers: piotr-bazan-nv, jpohl-nv

Merge Risk: Moderate · up to f7ca8

Python 3.14 users can receive the wrong Torch minor, while the documented CUDA 12 installation can select the wrong ONNX Runtime source. Resolve both before merging.

Pre-merge checks: 4

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly identifies the main change: fixing the ONNX Runtime CUDA dependency and lock configuration.
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

Finishing Touches

Generate unit tests (beta)

Commit to this branch
Create a new PR

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Tip

🎯 Customize your risk assessments

Guide how CodeRabbit assesses merge risk and security architecture risk. Highlight sensitive changes and explain your security boundaries using Merge Risk Instructions and Security Risk Instructions in your review settings. Security architecture risk assessments require the Advanced plan or above.

✅ Automatic approvals, with your rules

Let CodeRabbit approve pull requests once required threads are resolved, pre-merge checks pass, and your risk limits are met. Enable Automatic approvals in review settings, with overrides for teams or file paths.

CodeRabbit Verify

Run CodeRabbit Verify to test the changes in this pull request.

Recent review info

Run configuration

Source mode: Auto

Repository: ai-dynamo/aitune

Branch: knowicki/fixing&`#95`;onnx&`#95`;runtime&`#95`;cuda&`#95`;problem

File: .coderabbit.yml

Commit: f7ca81b

Layers: Repository YAML

Review profile: CHILL

Plan: Enterprise

Run ID: `3b788f6e-22c2-4e34-8587-e3494715e58d`

Commits

Reviewing files that changed from the base of the PR and between `ef15f10` and `f7ca81b`.

Files ignored due to path filters: 13

`examples/BERT/uv.lock` is excluded by `!**/*.lock`

`examples/E5Large/uv.lock` is excluded by `!**/*.lock`

`examples/ESM2/uv.lock` is excluded by `!**/*.lock`

`examples/FLUX/uv.lock` is excluded by `!**/*.lock`

`examples/JitTuning/uv.lock` is excluded by `!**/*.lock`

`examples/LLM/uv.lock` is excluded by `!**/*.lock`

`examples/ParakeetCTC/uv.lock` is excluded by `!**/*.lock`

`examples/ParakeetRNNT/uv.lock` is excluded by `!**/*.lock`

`examples/ResNet/uv.lock` is excluded by `!**/*.lock`

`examples/StableDiffusion/uv.lock` is excluded by `!**/*.lock`

`examples/WAN/uv.lock` is excluded by `!**/*.lock`

`examples/YOLO/uv.lock` is excluded by `!**/*.lock`

`uv.lock` is excluded by `!**/*.lock`

Files selected for processing: 2

`.github/workflows/dispatch.compatibility.yml`

`pyproject.toml`

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

coderabbitai

Fix comments: Fix CodeRabbit comments on this PR

Signed-off-by: Konrad Nowicki <knowicki@nvidia.com>

knowicki-nvidia · 2026-11-03T10:14:30Z

        run: |
          uv venv
-          printf "tensorrt\n" > cu12-excludes.txt
+          printf "tensorrt\ncupy-cuda13x\n" > cu12-excludes.txt


https://github.com/NVIDIA/Model-Optimizer/blob/main/pyproject.toml#L66

knowicki-nvidia · 2026-11-03T10:15:24Z


+[[tool.uv.index]]
+name = "ort-cuda-12-nightly"
+url = "https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple/"


https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple/onnxruntime-gpu/

To look into the index: one needs to add a package name.

knowicki-nvidia · 2026-11-03T10:20:46Z

 onnxruntime-gpu = [
    { index = "ort-cuda-13-nightly", extra = "onnxruntime-gpu-cuda13" },
-    { index = "pypi", extra = "onnxruntime-gpu-cuda12" },
+    { index = "ort-cuda-12-nightly", extra = "onnxruntime-gpu-cuda12" },


pypi has mixed cuda versions for ort:
ort < 1.27 -> cu12
ort >= 1.27 -> cu13

https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements

jpohl-nv · 2026-11-03T10:30:45Z

            cuda_image: 12.9.2-devel-ubuntu24.04
            cuda_major: 12
-            uv_extra_args: nvidia-cuda-runtime-cu13==0.0.0a0 --excludes ./cu12-excludes.txt --reinstall-package tensorrt-cu12 --reinstall-package tensorrt-cu12-bindings --reinstall-package tensorrt-cu12-libs
+            uv_extra_args: nvidia-cuda-runtime-cu13==0.0.0a0 cupy-cuda12x --excludes ./cu12-excludes.txt --reinstall-package tensorrt-cu12 --reinstall-package tensorrt-cu12-bindings --reinstall-package tensorrt-cu12-libs


Suggestion (not for this PR, future enhancement): Given the complexity of the installation, it would be valuable to add an install.sh (or similarly named script) that:

Detects installed CUDA version

Includes those extra flags from here to support CUDA 12

Installs CUDA 13 dependencies otherwise

Then the whole CI calls install.sh and we wouldn't have to maintain installation paths at different places. The install.sh script could also be the starting point for users to install AITune.

szysad

Should this change also include updating https://github.com/ai-dynamo/aitune/blob/main/docs/learn/install.md?plain=1#L60?
I believe now --no-sources should be skipped.

knowicki-nvidia added 4 commits November 2, 2026 16:51

chore(locks): upgrade uv locks & fix uv conflict

43c6524

Signed-off-by: Konrad Nowicki <knowicki@nvidia.com>

fix(config): onnx runtime from correct cuda package index

1db43b3

Signed-off-by: Konrad Nowicki <knowicki@nvidia.com>

Merge branch 'knowicki/release-08-uv-locks-upgrade' into knowicki/fix…

7326fad

…ing_onnx_runtime_cuda_problem

chore: uv lock update after adding onnx

ef15f10

Signed-off-by: Konrad Nowicki <knowicki@nvidia.com>

github-actions Bot added dependencies Pull requests that update a dependency file examples labels Nov 3, 2026

coderabbitai Bot added the review-effort/medium label Nov 3, 2026

coderabbitai Bot reviewed Nov 3, 2026

View reviewed changes

Comment thread pyproject.toml Resolved

Comment thread pyproject.toml Resolved

fix: new url for ort & constraints

f7ca81b

Signed-off-by: Konrad Nowicki <knowicki@nvidia.com>

github-actions Bot added the ci label Nov 3, 2026

knowicki-nvidia marked this pull request as ready for review November 3, 2026 09:29

knowicki-nvidia requested review from a team, jakkisel, jkosek, jpohl-nv and kacper-kleczewski November 3, 2026 09:29

knowicki-nvidia commented Nov 3, 2026

View reviewed changes

jpohl-nv approved these changes Nov 3, 2026

View reviewed changes

jakkisel approved these changes Nov 3, 2026

View reviewed changes

knowicki-nvidia mentioned this pull request Nov 3, 2026

enhancement: Given the complexity of the installation, it would be valuable to add an install.sh (or similarly named script) that: #164

Open

knowicki-nvidia merged commit 8b2e4d7 into main Nov 3, 2026
23 of 39 checks passed

knowicki-nvidia deleted the knowicki/fixing_onnx_runtime_cuda_problem branch November 3, 2026 10:45

szysad reviewed Nov 3, 2026

View reviewed changes

coderabbitai Bot mentioned this pull request Nov 4, 2026

feat(ci): FLUX on GB200 and support for arm64 images #169

Open

Navigation Menu

Repository navigation

fix(locks): fixing onnx runtime cuda problem - #162

fix(locks): fixing onnx runtime cuda problem#162
knowicki-nvidia merged 5 commits into
mainfrom
knowicki/fixing_onnx_runtime_cuda_problem

knowicki-nvidia commented Nov 3, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Nov 3, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot left a comment •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

knowicki-nvidia Nov 3, 2026

Uh oh!

knowicki-nvidia Nov 3, 2026 •

edited

Loading

Uh oh!

knowicki-nvidia Nov 3, 2026

Uh oh!

jpohl-nv Nov 3, 2026

Uh oh!

Uh oh!

szysad left a comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

knowicki-nvidia commented Nov 3, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Nov 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CodeRabbit Verify

Uh oh!

coderabbitai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

knowicki-nvidia Nov 3, 2026

Choose a reason for hiding this comment

Uh oh!

knowicki-nvidia Nov 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

knowicki-nvidia Nov 3, 2026

Choose a reason for hiding this comment

Uh oh!

jpohl-nv Nov 3, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

szysad left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

knowicki-nvidia commented Nov 3, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Nov 3, 2026 •

edited

Loading

coderabbitai Bot left a comment •

edited

Loading

knowicki-nvidia Nov 3, 2026 •

edited

Loading