[Feature] Lift the HOST-rail in-loop pld.tensor.allreduce restriction via shared-signal synthesis

[georgebisbas]

Summary

SynthesizeAllReduceSignals currently rejects pld.tensor.allreduce inside a for/while loop (CheckAllReduceCall, repeating_scope_depth_ == 0). With the host collective builtin kernels now self-clearing (#2279 — every kernel restores its signal cells to zero after each call), the restriction is no longer justified by the barrier protocol: it exists only because the pass mints a fresh synthesized signal allocation per call, which cannot be placed per dynamic iteration. This issue tracks lifting the restriction — a single per-host_orch shared signal plus in-loop support — originally deferred from #2279 as a follow-up.

Motivation / Use Case

• Training / decode loops call allreduce on every iteration. Users must today unroll the loop or hoist the call out manually, and an explicit signal bound before the loop is also rejected (the repeating_scope_depth_ == 0 check runs before the args_.size() == 2 branch).
• The InCore composite rail is already loop-safe (fix(ir): make composite collective barrier signals reusable #2175 / Make composite collective barrier signals reusable #2156); the HOST rail is the only remaining inconsistency.
• feat(distributed): self-clearing barrier signals in host collective kernels #2279's self-clearing epilogues make the signal provably all-zero after each call, so reuse across dynamic iterations is correct — the same bounded-skew (≤ 1 credit) argument the InCore rail already relies on.

Proposed API / Behavior

for step in pl.range(num_steps):
    data = pld.tensor.allreduce(data, signal, op=pld.ReduceOp.Sum)  # explicit signal — allowed

and, once shared-signal synthesis lands, the fully ergonomic form:

for step in pl.range(num_steps):
    data = pld.tensor.allreduce(data, op=pld.ReduceOp.Sum)  # synthesized signal, hoisted to host_orch entry

Pass change in SynthesizeAllReduceSignals:

• Emit one synthesized signal per host_orch function, allocated before the body exactly as MakeSignalBinding does today (world_size * 4 bytes → [world_size, 1] INT32 window), and pass it to every implicit-signal allreduce.
• Delete the repeating_scope_depth_ == 0 check (explicit signals pass through unchanged).
• Keep the mesh [NR, 1] vs ring [2*(NR-1)+1, NR] signal-shape guard (ValidateMeshSignalShape).
• Update the pass doc and the debugging "rejected inside loop" row (en/zh), and add loop + implicit-signal STs (convert the existing test_host_tensor_allreduce_signal_reuse to a live pl.range loop).

Alternatives Considered

• Keep the restriction and require users to manage signals manually: rejected — signal-window management is the most error-prone part of the collective API and the HOST rail is the only rail still loop-restricted.
• Hoist only the explicit-signal case (move the check after the arg-count branch): a reasonable minimal first step, but the synthesized path needs the shared-signal synthesis to be fully ergonomic, so the shared-signal approach is preferred.

Additional Context

• Tracked in plan 83 Phase 2 / PR-2 (feat/synthesize-allreduce-shared-signal on fork-gbisbas, previously verified in sim: UTs + allreduce basic/signal-reuse/loop STs P=2/P=4).
• Related: [Feature] Add orchestration-level collective communication operators to PyPTO compiler #1189 (umbrella collective ops), Make composite collective barrier signals reusable #2156 (InCore reusable barrier signals, closed by fix(ir): make composite collective barrier signals reusable #2175), feat(distributed): self-clearing barrier signals in host collective kernels #2279 (HOST self-clearing epilogues — the prerequisite for this issue).