Skip to content

Cannot create TensorRT session in Java without using addCUDA first #26493

@znadrich-qf

Description

@znadrich-qf

Describe the issue

I'm attempting to load a model to the TensorRTExecutionProvider using the Java API but I am getting an error unless I use addCUDA prior to addTensorrt. Though this avoids the error, the model is loaded to the CUDAExecutionProvider instead of TensorRTExecutionProvider

The error I get is below

2025-11-04 11:10:47.790429743 [E:onnxruntime:ort-java, provider_bridge_ort.cc:2195 TryGetProviderInfo_CUDA] /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:1778 onnxruntime::Provider& onnxruntime::ProviderLibrary::Get() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libonnxruntime_providers_cuda.so: cannot open shared object file: No such file or directory

Exception in thread "main" ai.onnxruntime.OrtException: Error code - ORT_FAIL - message: /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:2203 onnxruntime::ProviderInfo_CUDA& onnxruntime::GetProviderInfo_CUDA() CUDA Provider not available, can't get interface for it

	at ai.onnxruntime.providers.OrtTensorRTProviderOptions.applyToNative(Native Method)
	at ai.onnxruntime.providers.StringConfigProviderOptions.applyToNative(StringConfigProviderOptions.java:90)
	at ai.onnxruntime.OrtSession$SessionOptions.addTensorrt(OrtSession.java:1182)
...

To reproduce

Attempt to load a model using SessionOptions.addTensorrt

Urgency

No response

Platform

Linux

OS Version

Ubuntu 22.04.5 LTS

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.22.0

ONNX Runtime API

Java

Architecture

X86

Execution Provider

TensorRT

Execution Provider Library Version

TensorRT 10.9

Metadata

Metadata

Assignees

No one assigned

    Labels

    api:Javaissues related to the Java APIep:TensorRTissues related to TensorRT execution providerstaleissues that have not been addressed in a while; categorized by a bot

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions