Skip to content

Conversation

@evkotov
Copy link
Contributor

@evkotov evkotov commented Nov 4, 2025

Details:

This PR adds rank alignment for ONNX If nodes and rank reduction for LSTM operations to enable conversion of models like silero_vad.onnx.

Problem 1: If Node Rank Mismatch

ONNX If nodes with branches producing outputs of different ranks were causing dynamic rank in the output, which prevented further graph optimizations and caused runtime errors in operations requiring static rank (Conv, ReduceMean).

Example:

If (condition) {
  then_branch: ReduceMean(x, axes=[1], keepdims=1) → output shape [2, 1]
  else_branch: Identity(x) → output shape [2, 4]
}
Result: Dynamic rank output [?, ?]

Before fix:

   If Node
      |
     ┌────┴────┐
     |         |
     v         v
  then: [2,1] else: [2,4]
     |         |
     └────┬────┘
          v
     Output: [?, ?]  (dynamic rank)
          v
     Conv/ReduceMean
          v
     ERROR: "rank must be static"

After fix:

   If Node
      |
     ┌────┴────┐
     |         |
     v         v
  then: [2,1] else: [2,4]
     |         |
     v         |
  Unsqueeze   |
  (axis=0)    |
     |         |
  [1,2,1]  [1,2,4]
     |         |
     └────┬────┘
          v
     Output: [1,2,?]  (static rank 3)
          v
     Conv/ReduceMean
          v
     SUCCESS

Problem 2: LSTM High-Rank Inputs

Models like silero_vad.onnx have LSTM layers where input tensors have rank > 3 (e.g., [1,1,?,?,?]), but OpenVINO LSTM expects exactly rank 3 [batch, seq, features].

Example from silero_vad:

X: [1, 1, ?, ?, ?]  (rank 5)
initial_h: [1, 1, 2, 1, 64]  (rank 5)
initial_c: [1, 1, 2, 1, 64]  (rank 5)

Before fix:

  X [1, 1, ?, ?, ?]
        |
        v
     ┌──┴──────────────┐
     | LSTM Operator   |
     | Expects rank=3  |
     | Got rank=5      |
     └─────────────────┘
  Error: "Shape rank of input at 0 is incompatible.
          Expected rank: 3, actual shape: [1,1,?,?,?]"

After fix:

  X [1, 1, ?, ?, ?]
        |
        v
  reduce_tensor_rank(target=3)
        |
        v
  Squeeze leading dims = 1
   or Reshape if needed
        |
        v
  X [batch, seq, features]  (rank 3)
        |
        v
     ┌──┴──────────────┐
     | LSTM Operator   |
     |   rank=3       |
     └─────────────────┘

Tickets:

  • 162986

@evkotov evkotov requested a review from a team as a code owner November 4, 2025 17:54
@github-actions github-actions bot added category: build OpenVINO cmake script / infra category: ONNX FE OpenVINO ONNX FrontEnd labels Nov 4, 2025
@evkotov evkotov self-assigned this Nov 4, 2025
@evkotov evkotov changed the title Add rank alignment for If nodes and batch broadcasting for LSTM to en… Add rank alignment for If nodes and batch broadcasting for LSTM to enable silero_vad conversion Nov 4, 2025
Copy link
Contributor

@mvafin mvafin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You comminted .onnx models instead of .prototxt

@mvafin
Copy link
Contributor

mvafin commented Nov 19, 2025

@evkotov Not very clear why we need this change for If, what is the reason for rank mismatch? Why we add Unsqueeze? Can this be done in a separate PR?

@evkotov
Copy link
Contributor Author

evkotov commented Nov 21, 2025

You comminted .onnx models instead of .prototxt

I updated it

@evkotov evkotov changed the title Add rank alignment for If nodes and batch broadcasting for LSTM to enable silero_vad conversion Add If rank alignment and LSTM rank reduction to enable silero_vad conversion Nov 21, 2025
@github-actions github-actions bot removed the category: build OpenVINO cmake script / infra label Nov 21, 2025
@evkotov
Copy link
Contributor Author

evkotov commented Nov 25, 2025

moved LSTM fix part to #33023

@evkotov
Copy link
Contributor Author

evkotov commented Nov 25, 2025

moved if fix to #33027

@evkotov evkotov closed this Nov 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

category: ONNX FE OpenVINO ONNX FrontEnd

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants