ONNX for TensorRT — Day 6: ONNX Functions, Custom Operators, and TensorRT Portability

[rui-ren]

Day 6/42 — Phase 1: ONNX one-on-one fundamentals

Objective

Distinguish ONNX functions from custom operators and assess their portability to NVIDIA TensorRT and TensorRT-RTX.

Concept

An ONNX function provides a reusable graph body composed of other operators. A custom operator introduces a domain/operator contract that TensorRT must recognize through parser support or a plugin. A custom op carrying a standard-ONNX function body may be expanded before TensorRT parsing; a custom op without an expandable body creates a deployment dependency.

Tasks (30–60 minutes)

• Confirm the checkout is on ruiren/tensorRT-mobius.
• Read Three-Tier Support Strategy and Design Decisions in docs/execution_providers.md.
• Inspect EpCapabilities and _register_builtins() in src/mobius/_execution_providers.py, focusing on supports_skip_layer_norm and the trt-rtx entry.
• Create a three-row comparison covering:
  • standard ONNX operator
  • custom operator with an ir.Function fallback
  • custom operator without a function body
• For each row, record its graph representation, TensorRT parser risk, required deployment handling, and portability.
• Explain why TensorRT-RTX expands unsupported skip-normalization operations into standard primitives, while a fusion natively supported by TensorRT—or implemented by a required plugin—may be retained.
• Explain why onnx-standard proactively removes custom-domain dependencies.

Deliverable

Add a comment to this issue containing:

1. the three-row comparison
2. a recommendation of about 100 words for choosing ONNX functions versus custom operators in Mobius

No Mobius source-code change is required for this lesson.

Verification

Your answer must correctly state that:

• ir.Function bodies can express an operator using standard graph operations
• unknown custom operators may require TensorRT parser support or plugins
• expanding a function body can improve portability at the cost of losing a fused representation
• onnx-standard proactively removes custom-domain dependencies

Completion criteria

The comparison clearly separates graph representation from runtime support and gives a defensible TensorRT deployment recommendation.

Robotics relevance

Minimizing parser and plugin dependencies makes edge-robot deployments easier to validate, migrate, and recover under strict latency constraints.