Skip to content

Conversation

@i-riyad
Copy link
Contributor

@i-riyad i-riyad commented Nov 3, 2025

What does this PR do?

Type of change: Bug

Overview: get_onnx_bytes api is error prone as it returns only the protobuf info. If model has any external data, they get discarded! We have to use get_onnx_bytes_and_metadata and provide example for users to correctly write ONNX model to disk.

Usage

This would be proper way to save ONNX model bytes with/without external data.

    onnx_bytes, _ = get_onnx_bytes_and_metadata(
        model=model,
        dummy_input=(input_tensor,),
        weights_dtype=weights_dtype,
        model_name=model_name,
    )
    onnx_bytes_obj = OnnxBytes.from_bytes(onnx_bytes)

    # Write the onnx model to the specified directory without cleaning it
    onnx_bytes_obj.write_to_disk(os.path.dirname(onnx_save_path), clean_dir=False)

Testing

N/A. Existing tests are modified.

Before your PR is "Ready for review"

  • Make sure you read and follow Contributor guidelines and your commits are signed.
  • Is this change backward compatible?: Yes
  • Did you write any new necessary tests?: Yes
  • Did you add or update any necessary documentation?: Yes
  • Did you update Changelog?: Yes

Additional Information

https://nvbugspro.nvidia.com/bug/5618246/4

@i-riyad i-riyad requested a review from a team as a code owner November 3, 2025 23:44
@i-riyad i-riyad requested a review from ajrasane November 3, 2025 23:44
@i-riyad i-riyad force-pushed the rislam/onnx-download-fix branch from 7210c14 to 31b23d6 Compare November 3, 2025 23:47
@i-riyad i-riyad requested review from a team as code owners November 3, 2025 23:47
@i-riyad i-riyad requested a review from realAsma November 3, 2025 23:47
@i-riyad i-riyad force-pushed the rislam/onnx-download-fix branch 2 times, most recently from e42bc6f to 8113df6 Compare November 4, 2025 02:19
@codecov
Copy link

codecov bot commented Nov 4, 2025

Codecov Report

❌ Patch coverage is 18.18182% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.43%. Comparing base (72f23dc) to head (eed985e).
⚠️ Report is 12 commits behind head on main.

Files with missing lines Patch % Lines
modelopt/torch/_deploy/utils/torch_onnx.py 18.18% 9 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #496      +/-   ##
==========================================
- Coverage   73.43%   73.43%   -0.01%     
==========================================
  Files         180      180              
  Lines       18146    18144       -2     
==========================================
- Hits        13326    13324       -2     
  Misses       4820     4820              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@i-riyad i-riyad force-pushed the rislam/onnx-download-fix branch 6 times, most recently from 12f93d7 to d1652e2 Compare November 4, 2025 03:50
Signed-off-by: Riyad Islam <[email protected]>
@i-riyad i-riyad force-pushed the rislam/onnx-download-fix branch from d1652e2 to eed985e Compare November 4, 2025 03:51
@kevalmorabia97 kevalmorabia97 merged commit c3f6cef into main Nov 5, 2025
30 of 36 checks passed
@kevalmorabia97 kevalmorabia97 deleted the rislam/onnx-download-fix branch November 5, 2025 03:48
kevalmorabia97 pushed a commit that referenced this pull request Nov 5, 2025
## What does this PR do?

**Type of change:** Bug <!-- Use one of the following: Bug fix, new
feature, new example, new tests, documentation. -->

**Overview:** `get_onnx_bytes` api is error prone as it returns only the
protobuf info. If model has any external data, they get discarded! We
have to use `get_onnx_bytes_and_metadata` and provide example for users
to correctly write ONNX model to disk.

## Usage
This would be proper way to save ONNX model bytes with/without external
data.
```python
    onnx_bytes, _ = get_onnx_bytes_and_metadata(
        model=model,
        dummy_input=(input_tensor,),
        weights_dtype=weights_dtype,
        model_name=model_name,
    )
    onnx_bytes_obj = OnnxBytes.from_bytes(onnx_bytes)

    # Write the onnx model to the specified directory without cleaning it
    onnx_bytes_obj.write_to_disk(os.path.dirname(onnx_save_path), clean_dir=False)
```

## Testing
N/A. Existing tests are modified.

## Before your PR is "*Ready for review*"
<!-- If you haven't finished some of the above items you can still open
`Draft` PR. -->

- **Make sure you read and follow [Contributor
guidelines](https://github.com/NVIDIA/TensorRT-Model-Optimizer/blob/main/CONTRIBUTING.md)**
and your commits are signed.
- **Is this change backward compatible?**: Yes <!--- If No, explain why.
-->
- **Did you write any new necessary tests?**: Yes
- **Did you add or update any necessary documentation?**: Yes
- **Did you update
[Changelog](https://github.com/NVIDIA/TensorRT-Model-Optimizer/blob/main/CHANGELOG.rst)?**:
Yes<!--- Only for new features, API changes, critical bug fixes or bw
breaking changes. -->

## Additional Information
https://nvbugspro.nvidia.com/bug/5618246/4

Signed-off-by: Riyad Islam <[email protected]>
mxinO pushed a commit that referenced this pull request Nov 11, 2025
## What does this PR do?

**Type of change:** Bug <!-- Use one of the following: Bug fix, new
feature, new example, new tests, documentation. -->

**Overview:** `get_onnx_bytes` api is error prone as it returns only the
protobuf info. If model has any external data, they get discarded! We
have to use `get_onnx_bytes_and_metadata` and provide example for users
to correctly write ONNX model to disk.

## Usage
This would be proper way to save ONNX model bytes with/without external
data.
```python
    onnx_bytes, _ = get_onnx_bytes_and_metadata(
        model=model,
        dummy_input=(input_tensor,),
        weights_dtype=weights_dtype,
        model_name=model_name,
    )
    onnx_bytes_obj = OnnxBytes.from_bytes(onnx_bytes)

    # Write the onnx model to the specified directory without cleaning it
    onnx_bytes_obj.write_to_disk(os.path.dirname(onnx_save_path), clean_dir=False)
```

## Testing
N/A. Existing tests are modified.

## Before your PR is "*Ready for review*"
<!-- If you haven't finished some of the above items you can still open
`Draft` PR. -->

- **Make sure you read and follow [Contributor
guidelines](https://github.com/NVIDIA/TensorRT-Model-Optimizer/blob/main/CONTRIBUTING.md)**
and your commits are signed.
- **Is this change backward compatible?**: Yes <!--- If No, explain why.
-->
- **Did you write any new necessary tests?**: Yes
- **Did you add or update any necessary documentation?**: Yes
- **Did you update
[Changelog](https://github.com/NVIDIA/TensorRT-Model-Optimizer/blob/main/CHANGELOG.rst)?**:
Yes<!--- Only for new features, API changes, critical bug fixes or bw
breaking changes. -->

## Additional Information
https://nvbugspro.nvidia.com/bug/5618246/4

Signed-off-by: Riyad Islam <[email protected]>
Signed-off-by: mxin <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants