ONNX for TensorRT — Day 5: Topological Graph Tracing, Residual Paths, and Dead Values

[rui-ren]

Day 5/42 — Phase 1: ONNX one-on-one fundamentals

Objective

Read an ONNX graph topologically and identify producer-consumer edges, branches, residual paths, and dead values.

Concept

In topological order, every node appears after the nodes producing its inputs. A value can feed multiple consumers, creating a branch. Residual blocks preserve an earlier value and later merge it with a transformed path using Add. A dead value has no path to a graph output and should normally be removed before TensorRT engine construction.

Tasks (40–50 minutes)

• Confirm the checkout is on ruiren/tensorRT-mobius.
• Read src/mobius/components/_decoder.py, focusing on _forward_pre_norm().
• Draw the first residual path: hidden_states → RMSNormalization → Attention, alongside the bypassed residual, ending at the first Add.
• Repeat the trace for the MLP residual path.
• For each operation, list its producer and consumer(s). Mark the branch where hidden_states is both saved as residual and sent through normalization.
• Read src/mobius/models/base.py around TextModel.forward(). Explain why discarded RoPE Gather outputs would be dead values in the fused-GQA path and how direct use of cos_cache and sin_cache avoids creating them.
• Read src/mobius/components/_decoder_test.py::test_residual_connections and relate its “at least two Add nodes” assertion to your diagram.

Deliverable

Add a comment to this issue containing a one-page graph trace with:

1. the ordered operations
2. producer → consumer edges
3. both residual branches
4. a two-sentence explanation of the hypothetical dead RoPE Gather values

Do not modify Mobius for this lesson.

Verification

Starting from the graph outputs hidden_states and present_key_value, trace backward through your diagram:

• every listed live value must reach an output
• each producer must appear before its consumer
• the hypothetical discarded Gather output must not reach an output

Completion criteria

The trace is topologically ordered, identifies both residual branches and their merge points, and correctly distinguishes live values from dead values.

Robotics relevance

Removing dead work and understanding residual dependencies helps TensorRT produce smaller, more predictable engines for latency-sensitive robot perception and control loops.