Skip to content

test_runtime_checker.py can hang on CI due to WinML hardware probing #250

Description

@timenick

Problem

tests/unit/analyze/core/test_runtime_checker.py occasionally hangs on CI, causing the test (analyze) job to hit the 30-minute timeout. Observed on multiple runs:

In all cases, tests pass up to 18% (test_rule_loader_suffix.py), then produce no output for 27+ minutes until timeout cancels the run.

Root cause

The tests create a RuntimeChecker with ep="QNNExecutionProvider", device="NPU" and call op_support(). This triggers the following call chain:

checker.op_support()
  → RuntimeCheckerQuery (runtime_checker_query.py)
    → _try_local_ep_check()
      → _is_ep_available_locally() (line 1010)
        → winml.register_execution_providers() (line 1021)
          → WinML.__init__() (winml.py:27)
            → provider.ensure_ready_async().get() (line 46)  ← BLOCKS
        → ort.get_ep_devices() (line 1031)                   ← BLOCKS

Three potential hang points:

  1. winml.py:46provider.ensure_ready_async().get() blocks waiting for each EP provider to initialize. On runners without QNN hardware, this async wait can hang indefinitely.
  2. winml.py:40initialize(options=InitializeOptions.ON_NO_MATCH_SHOW_UI) may hang on runners without Windows App SDK.
  3. runtime_checker_query.py:1031ort.get_ep_devices() can hang when querying for unavailable hardware.

These are hardware-dependent operations that should not be called in unit tests. On most CI runners WinML initialization fails fast, but runners with partial WinML installations can hang.

Suggested fix

Mock _is_ep_available_locally() (or the WinML initialization path) in the unit tests to avoid triggering real hardware probing. The tests are verifying type hints and caching behavior — they don't need actual EP hardware.

Metadata

Metadata

Labels

No labels
No labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions