Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
107 changes: 107 additions & 0 deletions docs/en/dev/ir/05-operators.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,113 @@ auto dynamic_dim = make_int(kDynamicDim);
| `f_deduce_type(fn)` | Type deduction function | `.f_deduce_type(DeduceAddType)` |
| `set_core_affinity(a)` | Which core executes the op (**placement**) | `.set_core_affinity(core_affinity::CoreAffinity::VECTOR)` |
| `set_no_duplicate()` | Op must not run on a second core (**replication**) | `.set_no_duplicate()` |
| `set_arg_effect(i, e)` | What the op does to argument `i`'s buffer | `.set_arg_effect(2, ArgEffect::Write)` |
| `set_arg_effect(i, fn)` | Same, when a kwarg decides it | `.set_arg_effect(2, [](const auto& kw) { ... })` |
| `no_arg_writes()` | Classified: writes through no argument | `.no_arg_writes()` |
| `set_write_channel(c)` | Hardware path the op's writes take | `.set_write_channel(WriteChannel::Dma)` |

### Argument effects

An operator that updates one of its arguments in place must say so. Direction
inference, dependency analysis and the parameter-direction verifier all ask the
registry the same question — *does this call write the buffer this argument
names?* — and an operator that never answered reads as a pure consumer:

```text
tile.mscatter writes output_tensor, but never declared it
→ the parameter it writes keeps direction In
→ no RAW edge is emitted against the kernel that reads it
→ the scheduler is free to run the reader first
→ stale data, or a deadlock waiting on a signal nobody wrote
```

| Effect | Meaning | Examples |
| ------ | ------- | -------- |
| `ArgEffect::Read` | Read, never written. Default for an unnamed argument | `tile.store`'s source tile |
| `ArgEffect::Write` | Overwritten without being read first | `tile.store`'s `output_tensor`, `pld.tile.get`'s `dst` |
| `ArgEffect::ReadWrite` | Read *and* written | `tile.matmul_acc`'s accumulator, an atomic store's destination |

**Partial overwrite is still `Write`.** A store that lands on a sub-region does
not read the untouched remainder — nothing moves *into* the kernel — so its
destination is a pure write. Declaring it `ReadWrite` is not a harmless
approximation: it makes the enclosing parameter `InOut`, which stages the buffer
host→device and, across ranks, invents a dependency between two ranks writing
disjoint rows.

Whether a destination is read is decided per operator, not per family: a
gather or exchange destination is pushed into and never loaded from, so it is
`Write`, while a reduce destination has its running value loaded back and is
`ReadWrite`.

**`ReadWrite` is for operators that genuinely read the slot**: an accumulator
(`out += x` reads the running sum), an atomic store or assemble, or a
destination-passing operator whose untouched positions flow through to its SSA
result (`tile.scatter`, `array.update_element`).

**Kwarg-dependent effects.** When a kwarg decides the answer, pass a resolver
instead of a constant. A kwarg can decide *whether* an argument is written at
all, not only how: `tile.mgather`'s third operand is a written GM scratch tensor
in Mat element mode and a read-only `valid_shape` in Mat row mode. The other
live cases are the `atomic` kwarg on the store family and the `op` kwarg on
`pld.system.notify`, whose default is atomic-add — so an unannotated notify
reads the slot it adds into:

```cpp
REGISTER_OP("tile.store")
// ... arguments, memory spec ...
.set_arg_effect(2,
[](const std::vector<std::pair<std::string, std::any>>& kwargs) {
return GetIntKwarg(kwargs, "atomic", static_cast<int>(AtomicType::kNone)) ==
static_cast<int>(AtomicType::kNone)
? ArgEffect::Write
: ArgEffect::ReadWrite;
})
.set_write_channel(WriteChannel::Dma)
```

**Declared-read-only is not the same as unclassified.** `HasDeclaredArgEffects()`
distinguishes "a human decided this operator writes nothing" (`no_arg_writes()`,
e.g. `pld.system.wait`) from "nobody has looked at this operator yet". An
analysis that needs the answer can then refuse to guess instead of defaulting an
unclassified writer to read-only.

**Enforcement.** `OpRegistry::ValidateArgEffects()` runs at import and rejects
two shapes, naming every offender and the fix rather than failing on first use:

- an operator declaring `set_output_reuses_input(N)` — its SSA result *is*
argument N's buffer, so it writes through it — without a verdict about
argument N specifically. Classification is what is required, not a particular
answer: an operator whose in-place slot is metadata may declare it read-only.
- an operator declaring a write channel while writing through no argument. A
channel says *how* an operator writes, so one without a write is either a
stray declaration or a missing one.

The second rule matters more than it looks. `set_write_channel()` creates the
effect spec as a side effect, so "the spec exists" cannot stand in for "a human
decided" — otherwise an operator that declared a channel and forgot its
`set_arg_effect` would pass the first rule with the argument it updates still
defaulting to `Read`. `no_arg_writes()` records the all-arguments verdict
explicitly, and combining it with `set_arg_effect` is rejected as
contradictory.

`set_write_channel` records whether the writes travel the MTE3/DMA path or the
scalar D-cache path. PyPTO cannot order the two against one GM tensor, so a
function mixing them on one buffer is rejected; the channel lets that diagnostic
read the registry instead of re-listing operators.

Declare it only for an operator whose writes really are one of those two paths,
and leave it unset otherwise — an unset channel keeps the operator out of that
diagnostic, which is where an operator belongs when neither path describes it:

- `pld.system.notify` emits `pto.comm.tnotify`, a distinct comm instruction.
Claiming either channel would let the diagnostic reject a valid program.
- `system.set_ffts` hands the workspace *pointer* to the FFTS unit rather than
moving data; the hardware writes that region on its own schedule, which no
dependency edge models. It declares `no_arg_writes()`.
- A composite collective updates a data window and a signal through different
mechanisms, and one operator-level channel cannot describe both. Per-argument
channels would, but no case yet needs the distinction, and a wrong single
answer is worse than none.

**`set_core_affinity` vs `set_no_duplicate`** — two orthogonal axes, and picking
the wrong one makes a false claim about the ISA:
Expand Down
88 changes: 88 additions & 0 deletions docs/zh/dev/ir/05-operators.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,94 @@ auto dynamic_dim = make_int(kDynamicDim);
| `f_deduce_type(fn)` | 类型推导函数 | `.f_deduce_type(DeduceAddType)` |
| `set_core_affinity(a)` | 算子在哪个核上执行(**放置**) | `.set_core_affinity(core_affinity::CoreAffinity::VECTOR)` |
| `set_no_duplicate()` | 算子不得在第二个核上运行(**复制**) | `.set_no_duplicate()` |
| `set_arg_effect(i, e)` | 算子对第 `i` 个参数缓冲区做了什么 | `.set_arg_effect(2, ArgEffect::Write)` |
| `set_arg_effect(i, fn)` | 同上,但由 kwarg 决定 | `.set_arg_effect(2, [](const auto& kw) { ... })` |
| `no_arg_writes()` | 已分类:不通过任何参数写入 | `.no_arg_writes()` |
| `set_write_channel(c)` | 算子写入所走的硬件通路 | `.set_write_channel(WriteChannel::Dma)` |

### 参数效应(Argument effects)

原地更新某个参数的算子必须显式声明。方向推导(direction inference)、依赖分析
(dependency analysis)和参数方向验证器都向注册表询问同一个问题——*这次调用是否
写入该参数所指的缓冲区?*——而从未回答过的算子会被读成纯消费者:

```text
tile.mscatter 写 output_tensor,却从未声明
→ 它写入的参数方向停留在 In
→ 不会对读取它的 kernel 发出 RAW 边
→ 调度器可以先运行读方
→ 读到陈旧数据,或等待一个无人写入的信号而死锁
```

| 效应 | 含义 | 示例 |
| ---- | ---- | ---- |
| `ArgEffect::Read` | 只读,从不写入。未声明参数的默认值 | `tile.store` 的源 tile |
| `ArgEffect::Write` | 覆盖写,写前不读 | `tile.store` 的 `output_tensor`、`pld.tile.get` 的 `dst` |
| `ArgEffect::ReadWrite` | 既读*又*写 | `tile.matmul_acc` 的累加器、原子 store 的目的操作数 |

**部分覆盖仍然是 `Write`。** 只写入子区域的 store 不会读取未触及的其余部分——没有
任何数据流*进*内核——所以其目的操作数是纯写。把它声明成 `ReadWrite` 并非无害的保守
近似:这会让外层参数变成 `InOut`,从而触发 host→device 搬运,并且在跨 rank 场景下
为两个写入不相交行的 rank 凭空造出一条依赖。

目的操作数是否被读取按**算子**判定,而非按家族:gather / exchange 的目的窗口只被推入、
从不被 load,因此是 `Write`;而 reduce 的目的窗口会把运行值 load 回来,因此是 `ReadWrite`。

**`ReadWrite` 留给真正会读取该槽位的算子**:累加器(`out += x` 会读取运行中的和)、
原子 store/assemble,或那些未触及位置会流入 SSA 结果的 destination-passing 算子
(`tile.scatter`、`array.update_element`)。

**由 kwarg 决定的效应。** 当答案取决于某个 kwarg 时,传入 resolver 而非常量。kwarg 不
仅能决定*怎么*写,还能决定某个实参*是否*被写:`tile.mgather` 的第三个操作数在 Mat elem
模式下是被写的 GM scratch,在 Mat row 模式下则是只读的 `valid_shape`。另外两处是 store
家族的 `atomic` kwarg,以及 `pld.system.notify` 的 `op` kwarg——后者默认是 atomic-add,
因此未加标注的 notify 会读取它累加的槽位:

```cpp
REGISTER_OP("tile.store")
// ... arguments, memory spec ...
.set_arg_effect(2,
[](const std::vector<std::pair<std::string, std::any>>& kwargs) {
return GetIntKwarg(kwargs, "atomic", static_cast<int>(AtomicType::kNone)) ==
static_cast<int>(AtomicType::kNone)
? ArgEffect::Write
: ArgEffect::ReadWrite;
})
.set_write_channel(WriteChannel::Dma)
```

**"已声明为只读"不等于"未分类"。** `HasDeclaredArgEffects()` 区分"有人判定该算子不
写入任何参数"(`no_arg_writes()`,例如 `pld.system.wait`)与"还没有人看过这个算子"。
需要答案的分析因而可以拒绝猜测,而不是把未分类的写者默认成只读。

**强制约束。** `OpRegistry::ValidateArgEffects()` 在 import 时运行,拒绝两种形态,
一次性列出所有违规算子和修复方式,而不是等到首次使用才失败:

- 声明了 `set_output_reuses_input(N)`(其 SSA 结果*就是*第 N 个参数的缓冲区,因此会
通过它写入),却没有专门对第 N 个参数作出裁决。要求的是"做出分类"而非某个特定答案:
原地槽位是元数据的算子可以声明为只读。
- 声明了写通路却不通过任何参数写入。通路描述的是"怎么写",因此没有写的通路要么是多余
声明,要么是漏了声明。

第二条比看上去重要。`set_write_channel()` 会顺带创建效应 spec,因此"spec 存在"不能
等同于"有人做过判断"——否则一个声明了通路却忘了 `set_arg_effect` 的算子会通过第一条
检查,而它原地更新的那个参数仍然默认为 `Read`。`no_arg_writes()` 显式记录"对所有参数
的裁决",且与 `set_arg_effect` 同时使用会被判为自相矛盾而拒绝。

`set_write_channel` 记录写入走的是 MTE3/DMA 通路还是标量 D-cache 通路。PyPTO 无法为
同一个 GM tensor 排序这两者,因此会拒绝在同一缓冲区上混用两者的函数;有了通路声明,
该诊断可以查询注册表而不必再列一遍算子清单。

只为写入确实走这两条通路之一的算子声明它,其余一律留空——留空会把该算子排除在该诊断
之外,而当两条通路都无法描述它时,这正是它该待的位置:

- `pld.system.notify` 发出的是 `pto.comm.tnotify`,一条独立的 comm 指令。声明任一通路
都会让该诊断拒绝合法程序。
- `system.set_ffts` 是把 workspace *指针*交给 FFTS 单元,而非搬运数据;该区域由硬件按
自己的节奏写入,没有任何依赖边能建模它。它声明 `no_arg_writes()`。
- 复合集合通信通过不同机制更新数据窗口与 signal,单个算子级通路无法同时描述两者。
按参数记录通路可以做到,但目前没有任何用例需要这种区分,而一个错误的单一答案比没有
答案更糟。

**`set_core_affinity` 与 `set_no_duplicate`** —— 两个正交的维度,选错会对 ISA
做出错误的断言:
Expand Down
Loading
Loading