Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion docs/comm-domain.md
Original file line number Diff line number Diff line change
Expand Up @@ -237,6 +237,19 @@ the resident `KernelArgs`, and injects it into every run's kernel
SDMA streams and its kernels read a zero workspace address. The workspace is
released at Worker finalize by ordinary stream/manager teardown.

**`enable_sdma` is a defect quarantine, not a capability switch, and it has an
exit condition.** Selecting an engine already belongs to the kernel, which reads
whichever addresses were injected; the runtime would otherwise provision every
engine the platform supports. SDMA is the exception because
`SdmaWorkspaceManager::Init()` is indivisible — the 16 KB workspace *is* the
descriptor table for 48 CP-process STARS streams, so there is no way to have the
address without holding the streams — and a Worker holding those streams gets a
single device-reset attempt instead of three after an AICore fault. Defaulting it
on would therefore put every ordinary Worker in that population, which is exactly
the regression [the investigation](investigations/2026-07-a2a3-sdma-fault-teardown.md)
records. The flag disappears once CANN bounds the final CP-process stream release,
or once pto-isa offers an `Init()` that separates the workspace from the streams.

Provisioning also warms the SDMA control path once, in the same call: a
vector-only AICore ELF (`sdma_warmup_kernel.o`, staged per arch under
`build/lib/<arch>/sdma_warmup/`) walks every channel so the first
Expand All @@ -260,7 +273,7 @@ and issue #1425. `enable_sdma` is currently honored only by the a2a3 onboard
`tensormap_and_ringbuffer` runtime; host-build-graph, simulation, a5, and
provider-disabled builds fail Worker init fast when it is set. A5 provisions
its communication-context SDMA workspace by default; this is separate from
the callable-declared workspace mechanism controlled by `enable_sdma`.
the Worker-level workspace mechanism controlled by `enable_sdma`.

---

Expand Down
7 changes: 4 additions & 3 deletions src/common/platform/include/aicpu/aicpu_device_config.h
Original file line number Diff line number Diff line change
Expand Up @@ -55,9 +55,10 @@ int get_scheduler_timeout_ms();
* kind (see DmaWorkspaceKind). Published by simpler_aicpu_init (from
* InitArgs.dma_workspace_addr[]) into a resident-SO array; the scheduler
* copies each slot into every core's GlobalContext, so kernels read it via
* get_dma_workspace(args, kind). 0 means no callable has provisioned that
* engine. A callable that declares the engine is rejected before launch if the
* platform cannot provide a non-zero address. Out-of-range kinds are ignored.
* get_dma_workspace(args, kind). 0 means this Worker did not provision that
* engine — the platform does not support it, or the Worker declined it. A
* kernel that reads 0 is responsible for its own fallback; nothing rejects it
* before launch. Out-of-range kinds are ignored.
*/
void set_dma_workspace_addr(int kind, unsigned long long addr);

Expand Down
26 changes: 19 additions & 7 deletions src/common/platform/include/common/dma_workspace.h
Original file line number Diff line number Diff line change
Expand Up @@ -11,22 +11,34 @@
/**
* Async-DMA engine workspace kinds.
*
* The runtime provisions a per-device scratch workspace for each async-DMA
* engine it supports and injects the device addresses into every core's
* GlobalContext, so kernels obtain them via get_dma_workspace(args, kind)
* without threading them as user args. One slot per engine, indexed by this
* enum; DMA_WORKSPACE_KIND_COUNT sizes the injected array end to end
* (InitArgs, the resident AICPU config, and GlobalContext).
* The runtime provisions a per-device scratch workspace and injects the device
* addresses into every core's GlobalContext, so kernels obtain them via
* get_dma_workspace(args, kind) without threading them as user args. Selecting
* an engine is the kernel's decision, not the runtime's: every provisioned
* address is injected, and a kind the runtime did not provision reads back 0.
* One slot per engine, indexed by this enum; DMA_WORKSPACE_KIND_COUNT sizes the
* injected array end to end (InitArgs, the resident AICPU config, and
* GlobalContext).
*
* simpler-owned and deliberately independent of pto-isa's comm::DmaEngine —
* the kernel maps this kind to the pto-isa engine tag at the call boundary.
* Shared by host (provisioning + InitArgs) and device (scheduler + kernels),
* so it carries no dependencies beyond the enum itself.
*
* A Worker provisions an engine only when it both requested that engine and the
* device supports it (dma_workspace_supported_mask()), so an unrequested or
* unsupported kind stays 0. SDMA is the only engine anyone declines today, and
* the only one worth declining: provisioning it is inseparable from holding 48
* CP-process STARS streams whose post-fault release CANN bounds neither with a
* completion fence nor with a portable timeout — see
* docs/investigations/2026-07-a2a3-sdma-fault-teardown.md. A Worker declines it
* by leaving `enable_sdma` off, which is the default.
*
* Current support matrix: SDMA is available only on a2a3 onboard with the
* tensormap_and_ringbuffer runtime. URMA is reserved for the future a5
* per-domain provider. Host-build-graph, simulation, a5, and builds without
* the a2a3 PTO-SDMA provider reject non-empty requirements at registration.
* the a2a3 PTO-SDMA provider reject a request for SDMA during Worker
* initialization, when the workspace would be provisioned.
*/

#ifndef PLATFORM_COMMON_DMA_WORKSPACE_H_
Expand Down
Loading