Problem
tests/unit/analyze/core/test_runtime_checker.py occasionally hangs on CI, causing the test (analyze) job to hit the 30-minute timeout. Observed on multiple runs:
In all cases, tests pass up to 18% (test_rule_loader_suffix.py), then produce no output for 27+ minutes until timeout cancels the run.
Root cause
The tests create a RuntimeChecker with ep="QNNExecutionProvider", device="NPU" and call op_support(). This triggers the following call chain:
checker.op_support()
→ RuntimeCheckerQuery (runtime_checker_query.py)
→ _try_local_ep_check()
→ _is_ep_available_locally() (line 1010)
→ winml.register_execution_providers() (line 1021)
→ WinML.__init__() (winml.py:27)
→ provider.ensure_ready_async().get() (line 46) ← BLOCKS
→ ort.get_ep_devices() (line 1031) ← BLOCKS
Three potential hang points:
winml.py:46 — provider.ensure_ready_async().get() blocks waiting for each EP provider to initialize. On runners without QNN hardware, this async wait can hang indefinitely.
winml.py:40 — initialize(options=InitializeOptions.ON_NO_MATCH_SHOW_UI) may hang on runners without Windows App SDK.
runtime_checker_query.py:1031 — ort.get_ep_devices() can hang when querying for unavailable hardware.
These are hardware-dependent operations that should not be called in unit tests. On most CI runners WinML initialization fails fast, but runners with partial WinML installations can hang.
Suggested fix
Mock _is_ep_available_locally() (or the WinML initialization path) in the unit tests to avoid triggering real hardware probing. The tests are verifying type hints and caching behavior — they don't need actual EP hardware.
Problem
tests/unit/analyze/core/test_runtime_checker.pyoccasionally hangs on CI, causing thetest (analyze)job to hit the 30-minute timeout. Observed on multiple runs:In all cases, tests pass up to 18% (
test_rule_loader_suffix.py), then produce no output for 27+ minutes until timeout cancels the run.Root cause
The tests create a
RuntimeCheckerwithep="QNNExecutionProvider",device="NPU"and callop_support(). This triggers the following call chain:Three potential hang points:
winml.py:46—provider.ensure_ready_async().get()blocks waiting for each EP provider to initialize. On runners without QNN hardware, this async wait can hang indefinitely.winml.py:40—initialize(options=InitializeOptions.ON_NO_MATCH_SHOW_UI)may hang on runners without Windows App SDK.runtime_checker_query.py:1031—ort.get_ep_devices()can hang when querying for unavailable hardware.These are hardware-dependent operations that should not be called in unit tests. On most CI runners WinML initialization fails fast, but runners with partial WinML installations can hang.
Suggested fix
Mock
_is_ep_available_locally()(or the WinML initialization path) in the unit tests to avoid triggering real hardware probing. The tests are verifying type hints and caching behavior — they don't need actual EP hardware.