Skip to content

fix(perf): poll the right adapter under --device gpu/--monitor (#445)#467

Merged
xieofxie merged 10 commits into
mainfrom
hualxie/fix_gpu
May 11, 2026
Merged

fix(perf): poll the right adapter under --device gpu/--monitor (#445)#467
xieofxie merged 10 commits into
mainfrom
hualxie/fix_gpu

Conversation

@xieofxie

@xieofxie xieofxie commented May 7, 2026

Copy link
Copy Markdown
Contributor

Fixes #445winml perf --monitor --device gpu now shows GPU utilization instead of silently falling back to NPU.

Summary

  • Adapter routing. PdhPoller / HWMonitor now accept device= (npu | gpu | cpu | auto) and ep_name=. Per-device queries go through build_npu_query (Compute engine) or build_gpu_query (3D engine).
  • ORT-first LUID resolution. Added resolve_adapter_luid(device_kind, ep_name=None) in sysinfo/pdh_adapters.py. It walks ort.get_ep_devices(), filters by EP name + device type, reads OrtHardwareDevice.metadata["LUID"], and converts the decimal LUID to the PDH 0xHHHHHHHH_0xHHHHHHHH form via (v >> 32, v & 0xFFFFFFFF). ORT-resolved LUIDs are sanity-checked against enumerate_adapters() — if PDH doesn't enumerate the LUID (some drivers register the adapter only for autoEP), we skip it and fall through to the PDH-fingerprint helpers (discover_npu_luid, discover_gpu_luid) instead of raising later in build_adapter_query. Same fallback applies when ORT isn't installed, when get_ep_devices() raises, when no matching ep_device exists, when LUID metadata is missing, or when an individual ep_device entry is malformed (each is wrapped in try/except so one bad entry can't break resolution).
  • Graceful build-query degradation. PdhPoller.start() wraps build_npu_query / build_gpu_query in a local try/except ValueError. If a resolved LUID still proves unusable, it clears device_kind / adapter_luid and degrades to CPU/RAM-only — CPU/RAM counters still register and the benchmark continues.
  • perf.py plumbing. _run_benchmark_monitored (HF), _run_onnx_benchmark (ONNX direct), and _perf_modules (per-module) all pass the resolved device + normalized EP name into HWMonitor. --monitor help text updated to "Show live hardware utilization chart for the benchmarked device (NPU, GPU, or CPU)".
  • HWMonitor.to_dict() schema. Always emits an npu block (zeros when monitoring GPU — back-compat with existing consumers/tests). When device_kind == "gpu" a parallel gpu block is added. New top-level keys device_kind and adapter_luid.
  • Live chart labels follow what's actually polled. Centralised adapter_label(device_kind) helper in hw_monitor.py. LiveMonitorDisplay accepts device_kind= (threaded through from _run_monitored_loop as hw.device_kind); HWLiveDisplay derives the label dynamically from self._hw.device_kind. So auto resolving to GPU now labels "GPU", and CPU-only mode no longer falsely says "NPU".
  • CPU-only display drops the adapter row entirely. When no adapter is polled (--device cpu or auto resolving to nothing), LiveMonitorDisplay / HWLiveDisplay skip the green plot, drop the swatch from the chart legend, and collapse the status row to CPU: ... | Sys Mem: ... (no adapter cell, no Device Mem cell). The post-benchmark display_console_report in perf.py likewise prints CPU/RAM only — no zeroed NPU: 0.0% avg, 0.0% peak.

Verification

  • Targeted suite: tests/unit/session/test_ep_monitor.py (84 passed, +15 new) plus test_perf_cli.py and test_perf_module.py (26 passed).
  • Full unit suite green: 4665 passed, 73 skipped, 2 xfailed.
  • ruff check clean on every modified file.
  • Smoke-checked on a Snapdragon X laptop after winml.register_execution_providers(ort=True):
    GPU (no ep_name)  -> 0x00000000_0x000121CF   # Adreno X1-85 via DML and QNN
    GPU (ep=DML)      -> 0x00000000_0x000121CF
    GPU (ep=QNN)      -> 0x00000000_0x000121CF
    NPU (ep=QNN)      -> 0x00000000_0x00012C89   # authoritative, not just fingerprinted
    NPU (no ep_name)  -> 0x00000000_0x00012C89
    

@xieofxie xieofxie requested a review from a team as a code owner May 7, 2026 10:04
Comment thread src/winml/modelkit/commands/_live_chart.py
Comment thread src/winml/modelkit/session/monitor/hw_monitor.py Outdated
Comment thread src/winml/modelkit/commands/_live_chart.py
Comment thread src/winml/modelkit/session/monitor/hw_monitor.py
Comment thread src/winml/modelkit/session/monitor/_pdh.py Outdated
Comment thread src/winml/modelkit/session/monitor/hw_monitor.py
@xieofxie xieofxie merged commit 4139951 into main May 11, 2026
9 checks passed
@xieofxie xieofxie deleted the hualxie/fix_gpu branch May 11, 2026 07:59
ssss141414 pushed a commit that referenced this pull request May 15, 2026
…#467)

Fixes #445 — `winml perf --monitor --device gpu` now shows GPU
utilization instead of silently falling back to NPU.

## Summary

- **Adapter routing.** `PdhPoller` / `HWMonitor` now accept `device=`
(`npu` | `gpu` | `cpu` | `auto`) and `ep_name=`. Per-device queries go
through `build_npu_query` (Compute engine) or `build_gpu_query` (3D
engine).
- **ORT-first LUID resolution.** Added
`resolve_adapter_luid(device_kind, ep_name=None)` in
`sysinfo/pdh_adapters.py`. It walks `ort.get_ep_devices()`, filters by
EP name + device type, reads `OrtHardwareDevice.metadata["LUID"]`, and
converts the decimal LUID to the PDH `0xHHHHHHHH_0xHHHHHHHH` form via
`(v >> 32, v & 0xFFFFFFFF)`. ORT-resolved LUIDs are sanity-checked
against `enumerate_adapters()` — if PDH doesn't enumerate the LUID (some
drivers register the adapter only for autoEP), we skip it and fall
through to the PDH-fingerprint helpers (`discover_npu_luid`,
`discover_gpu_luid`) instead of raising later in `build_adapter_query`.
Same fallback applies when ORT isn't installed, when `get_ep_devices()`
raises, when no matching ep_device exists, when LUID metadata is
missing, or when an individual ep_device entry is malformed (each is
wrapped in `try/except` so one bad entry can't break resolution).
- **Graceful build-query degradation.** `PdhPoller.start()` wraps
`build_npu_query` / `build_gpu_query` in a local `try/except
ValueError`. If a resolved LUID still proves unusable, it clears
`device_kind` / `adapter_luid` and degrades to CPU/RAM-only — CPU/RAM
counters still register and the benchmark continues.
- **`perf.py` plumbing.** `_run_benchmark_monitored` (HF),
`_run_onnx_benchmark` (ONNX direct), and `_perf_modules` (per-module)
all pass the resolved device + normalized EP name into `HWMonitor`.
`--monitor` help text updated to *"Show live hardware utilization chart
for the benchmarked device (NPU, GPU, or CPU)"*.
- **`HWMonitor.to_dict()` schema.** Always emits an `npu` block (zeros
when monitoring GPU — back-compat with existing consumers/tests). When
`device_kind == "gpu"` a parallel `gpu` block is added. New top-level
keys `device_kind` and `adapter_luid`.
- **Live chart labels follow what's actually polled.** Centralised
`adapter_label(device_kind)` helper in `hw_monitor.py`.
`LiveMonitorDisplay` accepts `device_kind=` (threaded through from
`_run_monitored_loop` as `hw.device_kind`); `HWLiveDisplay` derives the
label dynamically from `self._hw.device_kind`. So `auto` resolving to
GPU now labels "GPU", and CPU-only mode no longer falsely says "NPU".
- **CPU-only display drops the adapter row entirely.** When no adapter
is polled (`--device cpu` or `auto` resolving to nothing),
`LiveMonitorDisplay` / `HWLiveDisplay` skip the green plot, drop the
swatch from the chart legend, and collapse the status row to `CPU: ... |
Sys Mem: ...` (no adapter cell, no Device Mem cell). The post-benchmark
`display_console_report` in `perf.py` likewise prints CPU/RAM only — no
zeroed `NPU: 0.0% avg, 0.0% peak`.

## Verification

- Targeted suite: `tests/unit/session/test_ep_monitor.py` (84 passed,
+15 new) plus `test_perf_cli.py` and `test_perf_module.py` (26 passed).
- Full unit suite green: 4665 passed, 73 skipped, 2 xfailed.
- `ruff check` clean on every modified file.
- Smoke-checked on a Snapdragon X laptop after
`winml.register_execution_providers(ort=True)`:
  ```
GPU (no ep_name) -> 0x00000000_0x000121CF # Adreno X1-85 via DML and QNN
  GPU (ep=DML)      -> 0x00000000_0x000121CF
  GPU (ep=QNN)      -> 0x00000000_0x000121CF
NPU (ep=QNN) -> 0x00000000_0x00012C89 # authoritative, not just
fingerprinted
  NPU (no ep_name)  -> 0x00000000_0x00012C89
  ```

---------

Co-authored-by: hualxie <hualxie@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

winml perf --monitor ignores --device gpu: GPU utilization never shown

2 participants