[Feature] Support a cache-access policy (L2 bypass) on GM loads

[zhangqi-chen]

Summary

There is currently no way to ask for a cache-access policy on a GM load —
for example "read this operand around L2" — anywhere in the pipeline that
reaches the emitted TLOAD. The only way to obtain the behavior today is to
hand-edit the generated C++ and add an architecture-specific address alias,
which cannot ship.

We would like this to become reachable through the normal compilation path. How
it is represented and lowered — and whether PTOAS is even the right layer — is
for PTOAS to decide; this issue states the need and the evidence, not a design.

Motivation / use case

A large class of GEMM operands are streamed once and never revisited: the
weight tile of a grouped/batched matmul, any operand whose working set exceeds
L2, and one-shot activation reloads. For these, every L2 allocation is a
guaranteed dead allocation — allocate, fill, evict, hit rate identically zero.
That is not merely neutral overhead:

1. the fill costs L2 tag and bandwidth that is never repaid; and
2. it evicts the operand that does have reuse. In a typical tiled GEMM the
   left operand is re-read by every N-block task while the weight tile is read
   exactly once, so the zero-reuse stream displaces the high-reuse one.

There is currently no way to express this. The only way to obtain the behavior
today is to edit the generated C++ after compilation and add an
architecture-specific address alias by hand:

// hand-edited into kernels/aic/*.cpp after codegen — not expressible in PTO
constexpr uint64_t L2_CACHE_DISABLE_OFFSET = 0x80000000000ULL;   // A2/A3 only
__gm__ int8_t* weightL2Bypass = reinterpret_cast<__gm__ int8_t*>(
    reinterpret_cast<uint64_t>(v2) + L2_CACHE_DISABLE_OFFSET);

This is unusable as a supported path: the constant is architecture-specific,
appears in no header in the toolchain, cannot be validated from inside an
AICore kernel, and does not survive recompilation.

Measured effect. On an INT8 grouped matmul ([E, N, K] weights, 23 active
experts, 368 cube tasks, A2/A3), applying the bypass to the weight loads only,
on top of an already NZ-packed weight layout:

PMU counter (sum over 368 tasks)	cached	bypass	ratio
mte2_busy_cycles	6,528,588	6,300,856	0.965
cube_busy_cycles	1,260,032	1,260,032	1.000
mte1_busy_cycles	3,535,290	3,534,611	1.002
pmu_total_cycles	9,704,218	9,480,931	0.977

cube_busy_cycles is identical to the cycle and mte1 moves 0.2%, so the whole
effect is on the GM→L1 path. The per-task distribution shows where it comes
from — a small uniform gain plus a large tail cut:

per-task mte2_busy_cycles	cached	bypass	ratio
p50	17,848	16,862	0.945
p90	19,518	18,719	0.959
p95	24,006	20,355	0.848
p99	25,930	21,318	0.822
coefficient of variation	0.170	0.077	—

The coefficient of variation more than halves. That is the signature of removing
cross-core interference: many cores each streaming a distinct, zero-reuse tile
through a shared L2 evict one another, and the unlucky tasks are pulled back to
the median once the stream stops competing for the cache. End-to-end this was
worth ~3% of device wall time on the same case, and it is expected to matter
more as core count and weight working set grow.

Proposed API / behavior

We are not proposing a design. The representation, the lowering, and the
layer this ultimately belongs in are PTOAS's to decide. What we need is that the
behavior becomes reachable through the normal pipeline. The constraints we can
state from the pypto-lib side:

• the policy has to be expressible before codegen and survive lowering to the
  emitted TLOAD, so it is not lost between PTO bytecode and generated C++;
• it has to distinguish operands that want different treatment. In the
  motivating kernel the weight operand is streamed once and never revisited
  while the left operand is re-read by every N-block task, so a single
  whole-kernel or whole-buffer setting would not capture it — but if PTOAS
  concludes the right place for this is the allocator or another layer, that
  conclusion is just as useful to us as a PTOAS-side feature;
• no architecture-specific constant should end up in PTOAS output or in
  pypto-lib;
• existing bytecode must keep its current meaning.

Our current understanding of the split is below, and we would welcome being
corrected on it — the boundary is exactly what we are unsure of:

Layer	Responsibility
pypto	frontend surface to declare the intent on an operand, reaching implicit matmul weight loads
PTOAS	carry the intent through PTO bytecode and lowering to the emitted TLOAD
pto-isa / simpler	provide the actual mechanism for the target (aliased view, page attribute, or descriptor bit) and own any coherency requirement

Alternatives considered

• Keep editing generated C++. Proves the optimization but is unsafe under
  recompilation, hardcodes an architecture constant, and cannot ship.
• Mark the buffer uncached at allocation time. A considerably smaller ask,
  and possibly sufficient for a purely streaming weight buffer. Listed here
  because if PTOAS judges this the right layer, we would rather pursue that than
  add anything to the bytecode.

Additional context

Measured on A2/A3 with RunConfig(enable_pmu=2); PMU counters are per-task
sums over one invocation, and the three variants differ only in the weight
GlobalTensor descriptor and the address used for the weight loads.

The three kernel binaries were compiled in an earlier session and replayed
unmodified for this PMU capture, so the counters reflect those binaries executed
on: pypto c7ba9fb0, simpler 93adc386, pto-isa cd4a3d3f, PTOAS 0.57,
CANN 9.0.0 — all components on their pins.

This is a general capability request, not a defect report — the manual C++
experiment does not establish a PTOAS bug.

Related: hw-native-sys/pypto-lib#1039 — the umbrella issue this came out of.
It tracks end-to-end direct-NZ and L2-bypass support for a DeepSeek-V4 Flash
decode grouped matmul, and carries the full experiment, the CANN comparison, and
the acceptance criteria. It currently assigns the runtime mechanism to simpler
and the frontend surface to pypto; this issue is us asking whether a piece of it
belongs in PTOAS.

Also related: #527 (NZ layout inference), which the same experiment depends on.