Skip to content

[Code Health] Nothing records whether a run paid torch_npu's device autoload #1950

Description

@ChaoWao

Category

Robustness (potential edge-case failure)

Component

Tests

Description

torch >= 2.5 runs every registered torch.backends entry point on import torch.
On this box that entry point exists and belongs to torch_npu, verified without
importing anything:

$ python3 -c "import importlib.metadata as md; [print(e.name,'->',e.value,'from',e.dist.name) for e in md.entry_points().select(group='torch.backends')]"
torch_npu -> torch_npu:_autoload from torch_npu
$ python3 -c "import torch, importlib.metadata as md; print(torch.__version__, md.version('torch_npu'))"
2.7.1+cpu 2.7.1

So every import torch in this repo runs torch_npu._autoload() unless
TORCH_DEVICE_BACKEND_AUTOLOAD=0 is set, and that autoload acquires an NPU device.
The repo imports torch in 21+ example/test files plus the golden modules and the
scene-test harness itself.

Two things make this worth tracking:

Nothing in this repo uses torch on the NPU. There is no import torch_npu, no
.npu(), and no device="npu" anywhere in the tree — torch is used purely for
CPU golden math (positive control: 21 files under examples/ + tests/ call CPU
torch ops). Whatever the autoload costs, it buys this repo nothing.

The condition is unobservable after the fact. No code here reads
TORCH_DEVICE_BACKEND_AUTOLOAD, so no log line records whether a given run paid
for the autoload. Two logs therefore cannot be compared for "same ruler" without
out-of-band knowledge of how each was launched.

That is not hypothetical. During the host_build_graph bind-phase measurement work
one comparison arm lacked TORCH_DEVICE_BACKEND_AUTOLOAD=0, paid for torch_npu
grabbing a device at import, and the difference was credited to the branch under
test — a plausible-looking wrong number. The trap is currently only documented
(docs/dfx/hbg-bind-phases.md), and the bind-phase recipe works around it by
echoing its own command into the log, which covers only runs launched that way.

Scope note: the harm here is to measurement, not to any product output. No test
result is wrong because of this; a timing number can be.

Location

  • simpler_setup/scene_test.py:180, :319, :370, :1209 — the harness's own import torch
  • simpler_setup/torch_interop.py:45
  • simpler_setup/goldens/{paged_attention,qwen3_14b_decode,deepseek_v4_flash_decode,mx_fp_gemm}.py
  • python/simpler/env_manager.py — the existing env-reading surface, 29 lines, no recording of effective values
  • docs/dfx/hbg-bind-phases.md:97, :156 — where the trap is documented today

Proposed Fix

Record it; do not change behavior.

Log the effective TORCH_DEVICE_BACKEND_AUTOLOAD at LOG_TIMING once per run, and
— more useful than the intent — whether torch_npu actually ended up in
sys.modules after the harness imports torch, which is the observed outcome. That
makes any number auditable from its own log instead of from memory, and it is purely
diagnostic, so it needs no new behavior gate
(.claude/rules/env-macro-gating.md §1 exempts diagnostic-only knobs).

Deliberately not proposed here: having the harness set
TORCH_DEVICE_BACKEND_AUTOLOAD=0 itself. It would be free for this repo's own
torch usage, but it is a behavior change for in-process callers that do want
torch_npu (pypto-serving), so it deserves its own decision rather than riding on a
logging change.

Priority

Medium (minor risk, should fix in next few releases)


Context: #617 (closed) consolidated the torch interop helpers into simpler_setup;
this is about the import side effect rather than the helpers.

Metadata

Metadata

Assignees

No one assigned

    Labels

    code healthTechnical debt, robustness, code quality

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions