feat(perf): cache per-module builds and honor --rebuild/--ignore-cache

[xieofxie]

Summary

winml perf --module built every submodule into a throwaway tempfile.TemporaryDirectory with no cache_key and no rebuild flag, so per-module artifacts were rebuilt on every run and --rebuild / --ignore-cache were silently ignored. The single-model path already honored these flags via WinMLAutoModel → auto.py.

This mirrors auto.py's cache logic in _perf_modules:

• Default: each submodule builds into the model's persistent cache dir (~/.cache/winml/artifacts/<model_slug>/) and is reused on the next run.
• --rebuild: reuses the cache dir but overwrites artifacts.
• --ignore-cache: falls back to a temp dir + forced rebuild (prior behavior, now opt-in).

Why per-module caching is collision-free

Each submodule config carries a unique loader.module_path (plus its traced I/O shapes). cfg.generate_cache_key() hashes the full config, so every instance gets a distinct key — even shape-identical siblings — and build_hf_model's cache_key filename prefixing keeps each instance's model.onnx separate within one shared model dir.

Behavior change worth noting

Per-module artifacts now persist by default (more disk use), which is the intended consequence of matching the single-model / auto.py behavior.

Tests

Added TestPerfModuleCache (default → persistent dir + rebuild=False; --rebuild → rebuild=True in cache dir; --ignore-cache → temp dir + rebuild=True). Full tests/unit/commands/test_perf_module.py: 15 passed.

Closes #918

🤖 Generated with Claude Code

[DingmaomaoBJTU]

Reviewed the per-module cache changes. The logic cleanly mirrors auto.py/the single-model path (use_cache = not ignore_cache, force_rebuild = rebuild or ignore_cache, persistent model dir + per-module cache_key, temp-dir fallback on --ignore-cache), and build_hf_model creates output_dir and scopes its rebuild-glob to {cache_key}_*.onnx, so sibling modules coexist and --rebuild won't wipe each other's artifacts. Ran tests/unit/commands/test_perf_module.py locally: 15 passed. Two notes inline — one behavioral nuance and one optional test-coverage suggestion; neither is blocking.

[DingmaomaoBJTU]

Re-reviewed the new commit (53a8f26) — it directly resolves both points from my previous review:

1. Optimize/analyze toggles now affect the cache key. get_cache_key folds skip_optimize / hack_max_optim_iterations in, applied at all four call sites (auto.py HF + ONNX, perf module, build single-model). I checked key/build consistency: the same controls folded into the key are exactly what reach build_hf_model / build_onnx_model, so a later --no-optimize run no longer silently reuses an optimized artifact. Nice that path.py's stdlib-only / zero-internal-imports contract is preserved (hashlib/json/typing are all stdlib).
2. Collision-free claim now has a real test. test_sibling_instances_get_distinct_cache_keys uses two distinct config hashes (no longer mocked to a constant) and asserts distinct cache_keys reach build_hf_model; test_no_optimize_changes_cache_key guards the flag→key coupling.

Verified locally: applied the PR head and ran tests/unit/cache/test_path.py + tests/unit/commands/test_perf_module.py → 33 passed. One optional, non-blocking note inline. LGTM otherwise.

[DingmaomaoBJTU]

Approving. The follow-up commit cleanly resolves both points from my earlier reviews: optimize/analyze toggles now fold into the cache key at all four call sites (verified key/build consistency), and the collision-free claim has real test coverage. Verified locally — tests/unit/cache/test_path.py + tests/unit/commands/test_perf_module.py: 33 passed. The two inline notes are minor/optional and not blocking.