diff --git a/tech-specs/2026-07-14-worker-compose/README.md b/tech-specs/2026-07-14-worker-compose/README.md new file mode 100644 index 000000000..4ccdab2d1 --- /dev/null +++ b/tech-specs/2026-07-14-worker-compose/README.md @@ -0,0 +1,86 @@ +--- +title: worker compose — distributed worker lifecycle +tagline: one daemon per machine supervises the workers a compose file declares — namespaced, config-fed, crash-cascading, engine-coordinated. +date: 2026-07-14 +tags: [compose, workers, lifecycle, dx] +status: draft +--- + +# Worker Compose + +A standalone **compose daemon** that supervises iii workers declared in one +`worker-compose.yaml`. The daemon is not the engine: `iii compose` starts only +the daemon, which connects to an engine like any worker, registers the +`compose::up/down/list/status/logs/validate` control functions, and owns +exactly the child processes it spawned on its own machine. Several daemons — +several machines — attach to one engine; a trigger addresses one of them by +argument: `iii trigger compose::up id=host-a`. + +The load-bearing separation: **the engine routes and arbitrates; the daemon +supervises.** Worker lifecycle today is welded to the machine the engine runs +on (`iii worker` + iii-worker-ops). Compose breaks that weld so workers can +run where their resources are, while the engine stays the single coordination +point: it rejects duplicate registrations inside a namespace, holds triggers +for workers that have not arrived yet, and answers "who is running what". + +## What this pack proposes + +Six contracts, each in its own doc: + +| Doc | Contract | +| --- | --- | +| [compose-file.md](./compose-file.md) | `worker-compose.yaml` v1 — an id-keyed `containers:` object; `depends_on` inside one file only | +| [scripts.md](./scripts.md) | per-container `scripts:` — `pre_start` (blocking, timed), `run` (supervised), `post_run` (fires after the run exits) | +| [namespace.md](./namespace.md) | namespace as a runtime argument — never a function-name prefix; engine rejects same-name collisions inside a namespace | +| [configuration.md](./configuration.md) | the daemon fetches base config, applies compose overrides, hands the worker its final config at start | +| [cli-contract.md](./cli-contract.md) | one CLI/env contract for every worker binary: `--url`, `--namespace`, `--config`; `registerWorker()` reads env | +| [lifecycle.md](./lifecycle.md) | up/down over a validated DAG, readiness = engine registration, cascading failure, optimistic trigger buffer | + +## Why now + +Three facts from the current codebase force each major piece: + +1. **Duplicate registration is a silent overwrite.** The engine warns and + replaces on a name collision (`engine/src/services.rs:106-111` — and a unit + test codifies it: `registry_insert_service_duplicate_overwrites`). Running + two state workers means the last one wins and the first keeps running + blind. Namespaces + hard rejection replace that with a deterministic error. +2. **No two worker binaries agree on how to find the engine.** `http`, `state`, + `storage`, `database` take only a `--url` flag with a hardcoded + `ws://127.0.0.1:49134` default and read no env; `llm-router` reads + `III_WS_URL`; `shell` and friends read `III_URL`. No SDK reads any address + env at all. A supervisor cannot inject a contract that does not exist — + the CLI/env standard creates it. +3. **Config delivery is first-boot-only.** A worker's `--config` seeds the + configuration entry once; afterwards "the configuration worker is the + authoritative source and `--config` is ignored" (`workers/http/src/main.rs` + doc comment). Compose-managed config therefore cannot be a file passed at + spawn — the daemon has to resolve base + overrides and deliver the final + value through the standard contract. + +## Principles (decided 2026-07-13, team review) + +- **compose over docker**: bare-metal child processes locally; the sandbox + path stays the validated route to cloud. Docker is not the default runtime. +- **fail early over silent overwrite**: same name + same namespace = rejected + connection, not a zombie. +- **one file, one dependency scope**: `depends_on` never crosses compose + files. Sharing across files is a namespace decision, not a dependency edge. +- **worker knowledge stays with the worker**: compose does not carry full + configurations, and `setup`/`install` remain the worker/sandbox contract. +- **zero-code onboarding**: `registerWorker()` with no arguments must work — + the daemon injects url/namespace/config through env; developers learn + namespaces only when they need two of something. + +## Honest trade-offs + +- The namespace model requires protocol + all-three-SDK changes + (register/trigger carry a namespace; env contract) — the widest surface in + the pack. +- `run` in the compose file partially reverses the 2026-07-13 "no scripts in + compose" decision; the counter-argument is scoped: three hooks, `run` only + for local workers, no setup/install override. +- Per-target artifacts (a real release ships 9 platform binaries with 9 + digests) mean reproducibility metadata must be per-target from day one. +- The optimistic trigger buffer trades "fail loud on missing worker" for + "wait with a bounded timeout" — the timeout knob is what keeps that honest. diff --git a/tech-specs/2026-07-14-worker-compose/cli-contract.md b/tech-specs/2026-07-14-worker-compose/cli-contract.md new file mode 100644 index 000000000..527e86dd1 --- /dev/null +++ b/tech-specs/2026-07-14-worker-compose/cli-contract.md @@ -0,0 +1,52 @@ +# the worker CLI/env contract + +A supervisor can only inject what workers agree to read. Today they agree on +nothing: + +| Worker | Engine address | Env read | +| --- | --- | --- | +| http / state / storage / database / harness | `--url` flag, default `ws://127.0.0.1:49134` | none | +| llm-router | `--url` flag | `III_WS_URL` | +| shell / codex / devin / email / bridge / lsp / hermes | varies | `III_URL` | +| any SDK (`register_worker`) | explicit code argument | none | + +(`workers/http/src/main.rs:28`, `workers/llm-router/src/main.rs:25`, +`sdk/packages/rust/iii/src/lib.rs:96`.) + +## The standard + +Every worker binary accepts the same three parameters, each with a flag and +an env fallback, flags winning: + +| Parameter | Flag | Env | +| --- | --- | --- | +| engine address | `--url` | `III_URL` | +| namespace | `--namespace` | `III_NAMESPACE` | +| configuration | `--config` | `III_CONFIG` (resolved payload or handle — pinned with the config library) | + +- `registerWorker()` / `register_worker()` with **no arguments** reads the + env contract. Explicit arguments still win — but the zero-arg form is the + documented default, so tutorial code contains no addresses and no + namespaces, and the same code runs under compose, under `iii worker`, or by + hand. +- The daemon injects all three into every child (hooks included). Reserved + keys: a compose file cannot override them. +- Published binary packages get the flags for free once the shared **config + registration library** lands (one crate/package each SDK wraps — the + extraction is a listed next step from the 2026-07-13 review, so the fleet + is not 41 hand-rolled argument parsers). + +## Implicit start for packages + +With the contract in place, a `package://` binary needs no start script: +compose execs the resolved artifact with the standard flags. That is the +entire start story for the 34/41 fleet workers that today ship no `scripts:` +at all — and why `run` in scripts.md exists only for local workers. + +## Migration + +Additive: binaries keep their current flags; the standard adds env fallbacks +and normalizes names. The compose daemon only requires the env contract — +old workers run under compose the day they read `III_URL`/`III_NAMESPACE`/ +`III_CONFIG`, which the shared library gives them in one dependency bump. +`iii worker` and iii-worker-ops keep working unchanged during coexistence. diff --git a/tech-specs/2026-07-14-worker-compose/compose-file.md b/tech-specs/2026-07-14-worker-compose/compose-file.md new file mode 100644 index 000000000..0b44b9619 --- /dev/null +++ b/tech-specs/2026-07-14-worker-compose/compose-file.md @@ -0,0 +1,79 @@ +# worker-compose.yaml v1 + +One file describes one project's workers. One daemon binds to one file for its +whole lifetime; the daemon's `--id` is how remote control addresses it +(`iii trigger compose::up id=compose-hostb`). + +## Canonical shape + +```yaml +name: orders + +containers: + database: + worker: package://workers.iii.dev/database + version: 1.4.2 + config_name: orders-database + scripts: + post_run: "./backup-on-exit.sh" + + api: + worker: path://./workers/api # no iii.worker.yaml needed — run: is enough + depends_on: [database] + config_name: orders-api + config_override: + server: + port: 3000 + scripts: + pre_start: "npx prisma migrate deploy" + pre_start_timeout: 120s + run: "pnpm dev" + post_run: "./cleanup-tmp.sh" +``` + +## Container fields + +| Field | Required | Rule | +| --- | --- | --- | +| `worker` | yes | `package://` (registry) or `path://` (local directory) | +| `version` | package only | exact or range, resolved into the lockfile | +| `depends_on` | no | container ids **from the same file only** | +| `config_name` / `config_uri` | no | where the daemon fetches base config (see configuration.md) | +| `config_override` | no | sparse map merged over the fetched base | +| `scripts` | no (`run` required for manifest-less `path://`) | see scripts.md | +| `working_dir` | no | default: worker dir for `path://`, compose-file dir for `package://` | + +## Identity + +- `containers` is an **id-keyed object**; the key is the worker's lifecycle id + and its registered name. +- For `path://` with a manifest, the key must equal the manifest `name`. For + manifest-less `path://` (compose declares `run`), the key **is** the + identity. +- No key order semantics: start order comes only from the `depends_on` DAG. + +## Dependency scope + +`depends_on` resolves inside the same file, full stop. Two compose files that +want to share one database do it by **namespace** (both point at the same +namespace, one of them owns the process — see namespace.md), never by a +cross-file dependency edge. This keeps ownership, rollback, and teardown +decidable by one daemon looking at one file. + +## Validation (hard errors) + +- empty `containers`; unknown or self `depends_on`; dependency cycle (error + prints the full path: `api -> queue -> database -> api`); +- unknown field anywhere (strict schema); +- `run` on a `package://` worker; `pre_start_timeout` without `pre_start`; +- `path://` with neither manifest nor `run`; +- key ≠ manifest `name` when a manifest exists. + +`iii compose validate` runs the whole ruleset offline — no engine required. + +## Not in v1 + +Cross-file `depends_on`, docker runtime keys (`ports`, `image`), full inline +configuration bodies, hot reload, `environment`/`env_file` (children get the +standard injected contract plus the host env they inherit; per-container env +maps return if a concrete need appears). diff --git a/tech-specs/2026-07-14-worker-compose/configuration.md b/tech-specs/2026-07-14-worker-compose/configuration.md new file mode 100644 index 000000000..bc1d8efef --- /dev/null +++ b/tech-specs/2026-07-14-worker-compose/configuration.md @@ -0,0 +1,71 @@ +# configuration — fetched by the daemon, finalized before spawn + +The compose file never carries a full configuration and the worker never does +a startup dance (boot → connect → query → maybe restart). The **daemon** +resolves the final config and the worker receives it ready at start. + +## The flow + +``` +configuration worker (base, by name) + │ fetch + ▼ +compose daemon ── merge ──► final config ── standard contract ──► worker start + ▲ + │ sparse override +worker-compose.yaml (config_override) +``` + +1. The container names its base config: `config_name: orders-api` (or a + `config_uri` for explicit sources). Names resolve through the + **configuration worker**, whose adapter decides storage — local fs today, + secrets manager or a separate iii instance in cloud. File paths are not + the contract; names are, so the same compose file works when storage + moves. +2. The daemon fetches the base **before** anything else runs for that + container. Fetch failure = the container does not start (`up` fails rather + than booting on wrong defaults — an http worker on the wrong port is worse + than no http worker). +3. `config_override` merges over the base: maps merge per key, arrays and + scalars replace, `null` is an explicit value. +4. The finalized config reaches the worker through the standard CLI/env + contract (`--config` / env; see cli-contract.md). + +## Why the daemon delivers values (not a pointer) + +Today `--config` is a **first-boot seed**: it populates the configuration +entry only when nothing is stored, and afterwards "the configuration worker +is the authoritative source and `--config` is ignored" +(`workers/http/src/main.rs` doc comment). If compose passed only a pointer, +every restart after first boot would silently ignore the compose overrides. +Delivering resolved values keeps `worker-compose.yaml` honest: what the file +says is what the process got — while the configuration worker remains the +authority for everything the override does not touch. + +## Boundaries + +- **env vs config is a hard line**: env is machine/deployment context + (engine url, namespace); config is worker behavior (ports, adapters, + origins). The compose file does not blur them. +- No full config bodies in the compose file — one source of truth, no + "edited the YAML and nothing happened" trap. +- Runtime config reload is out of scope here: config is fixed for the life of + the process (the configuration worker's watch/reload path is a separate + track). + +## Example + +```yaml +containers: + state: + worker: package://workers.iii.dev/state + version: 0.21.x + config_name: orders-state + config_override: + cors: + allowed_origins: ["https://app.example.com"] +``` + +The state worker's base `orders-state` config lives with the configuration +worker (any adapter); the compose file pins one field; the process starts +with the merged result and never re-fetches. diff --git a/tech-specs/2026-07-14-worker-compose/lifecycle.md b/tech-specs/2026-07-14-worker-compose/lifecycle.md new file mode 100644 index 000000000..900cb5d19 --- /dev/null +++ b/tech-specs/2026-07-14-worker-compose/lifecycle.md @@ -0,0 +1,82 @@ +# lifecycle — up, readiness, cascade, teardown + +The daemon owns its children; the engine is the source of truth for +readiness. Every lifecycle answer follows from those two facts. + +## up + +1. Validate file + dependency DAG (offline ruleset — cycles print their full + path). +2. Resolve every container's config (fetch base + merge overrides). Any fetch + failure fails `up` before a single process exists. +3. Walk the DAG in topological order. Per container: + `pre_start` (blocking, timed) → spawn `run` in its own process group / + Job Object → wait for **readiness**. +4. On a startup failure: capture the failing worker's logs, roll back the + containers **this operation** started (reverse order), leave everything + that was already running untouched. +5. `up` against an already-running file is a no-op success (`changed: false` + per container). + +**Readiness = the worker's registration is visible on the engine** — not a +live PID. A process that boots but never registers is a startup failure at +its `startup_timeout`, and the client timeout of the invoking `iii trigger` +must sit above the graph's budget (a 30s client default under a 60-90s +startup budget loses the result mid-operation). + +## Optimistic trigger buffer + +Startup order breaks registrations today: a worker that registers a route +before its trigger-type worker (e.g. the http router) is up gets a warning +and a dropped route. The engine gains a **buffer**: trigger registrations +that reference an absent worker wait in memory and flush when that worker +arrives (or re-arrives after a restart). Compose exposes the knobs — how long +a registration may wait, how many retries — so the buffer cannot silently +hold a route forever. Ordering becomes a performance concern, not a +correctness one; `depends_on` stays about *data* dependencies, not +registration order. + +## Cascading failure + +If a worker crashes post-ready, its local transitive dependents stop, in +reverse dependency order, states set to `failed`, cause in the logs. The +alternative — a queue worker happily pushing into a dead database — corrupts +work invisibly. Softer edges (systemd-style `wants` vs `needs`) can come +later for optional side-processes; v1 has one edge type and it is strict. + +## down + +- `down` (bare): stops everything this daemon owns, dependents before + dependencies. Never anything owned by another daemon. +- `down `: the target plus its local transitive dependents. +- Termination per child: SIGTERM to the process group → bounded grace → + SIGKILL (Job Object termination on windows). Claims/registrations release + only after the process is confirmed dead. Once an exit is confirmed the + child's `post_run` fires (not awaited — teardown proceeds). +- Intentional daemon shutdown (SIGINT/SIGTERM) performs a local `down`. + Engine disconnect does **not** stop children: they keep serving and the + daemon reconciles when the engine returns. + +## Crash and restart + +State on disk per child (pid + process birth identity + group id + log +paths). On daemon restart: verify each recorded process against its birth +identity — reconcile the live ones, clean the dead ones, never signal a pid +it cannot verify (pid recycling is real). A worker that dies while the daemon +is down is detected here and cascades then. + +## Remote surface + +`compose::up / down / list / status / logs / validate` — one function family, +registered by the daemon like any worker's; the target daemon travels as an +argument: + +```bash +iii trigger compose::up id=host-a +iii trigger compose::down id=host-a +iii trigger compose::status id=host-a +``` + +`list`/`status` answer for **the addressed daemon's compose project only**; +the engine-wide view stays on `worker::list` (which is also being unified: +every connected entity reports as a worker with full metadata). diff --git a/tech-specs/2026-07-14-worker-compose/namespace.md b/tech-specs/2026-07-14-worker-compose/namespace.md new file mode 100644 index 000000000..b1c5ebdf2 --- /dev/null +++ b/tech-specs/2026-07-14-worker-compose/namespace.md @@ -0,0 +1,64 @@ +# namespace — a runtime argument, not a rename + +Two compose files that both declare a `state` worker must be able to coexist +on one engine. Renaming functions (`gdpr::state::get`) was rejected: code +would behave differently depending on where it runs. Instead, **namespace is a +separate dimension** carried alongside the function id. + +## The model + +- A worker process receives its namespace at start — an argument or an env + var injected by the daemon (`--namespace` / standard env). Code never + changes: `registerWorker()` picks it up from the environment. +- Function ids stay exactly what the worker registered (`state::get` is + `state::get` everywhere). +- `trigger()` gains an optional namespace argument, and routing is strict: + `trigger("state::set")` resolves **only in the default namespace**; + `trigger("state::set", namespace: "analytics")` resolves only in + `analytics`. No best-fit guessing — a miss is a clear FUNCTION_NOT_FOUND + that lists the namespaces where the id does exist. +- Every SDK (node / rust / python) implements the same surface — the + register/trigger protocol messages carry the namespace field. + +## Collision = rejection + +Inside one namespace, one worker name maps to one live instance. Today the +engine warns and overwrites (`engine/src/services.rs:106-111`; +`Engine::claim_function` transfers ownership between live workers with only a +WARN). That becomes a **rejected registration**: the second worker's +connection fails with a structured error, and its process exits instead of +running blind behind an engine that forgot it. + +`compose up` twice against the same running file: the daemon sees its own +live children and no-ops; a *second* daemon (or a hand-started worker) +claiming the same name in the same namespace is rejected. A future +`on_conflict: fail | ignore | replace` per container can soften this — the +default is `fail`. + +## Defaults that keep it invisible + +| Situation | Namespace | +| --- | --- | +| `compose up` with no namespace set | autogenerated per compose project — two files never collide by accident | +| worker started by hand, no env | default namespace — single-user flow unchanged | +| two files sharing one database | both declare the shared namespace explicitly; exactly one file owns the process | + +Onboarding never mentions namespaces: the first user runs one file, gets the +autogenerated isolation, and everything routes. The concept surfaces only +when someone wants two of the same thing — which is exactly when they need +it. + +## What this replaces + +The previous plan deferred duplicates entirely ("one worker name = one +instance per engine") and dropped namespaces as purposeless without them. +That left a real gap: package manifests pin their `name:` (`name: http`), so +two projects composing the published `http` worker could only collide. +Namespaces restore multi-instance without renaming anything. + +## Cost (stated plainly) + +Protocol change (register + trigger messages), engine routing change +(namespace-aware function table + strict resolution + rejection path), and +all three SDKs. This is the widest-surface piece of the pack and the reason +it is specified here rather than folded silently into compose. diff --git a/tech-specs/2026-07-14-worker-compose/scripts.md b/tech-specs/2026-07-14-worker-compose/scripts.md new file mode 100644 index 000000000..17f4b07e2 --- /dev/null +++ b/tech-specs/2026-07-14-worker-compose/scripts.md @@ -0,0 +1,72 @@ +# scripts: pre_start / run / post_run + +Per-container scripts in the **compose file**. Three hooks, one supervised +process, no stop-hooks: teardown stays signal-based (SIGTERM → grace → +SIGKILL on the process group). + +## Contract + +| Hook | Blocking | When | On failure | +| --- | --- | --- | --- | +| `pre_start` | yes | every time compose is about to spawn the worker; after config resolution, before spawn | exit ≠ 0 or timeout → container `failed`, worker never spawns, `up` rolls back what this operation started | +| `run` | supervised | the worker process itself | crash → cascading stop of local dependents (see lifecycle.md) | +| `post_run` | no | once, after the `run` process **exits** — any exit path (down, post-ready crash, `up` rollback) | warning + `last_error` in status; teardown proceeds without waiting | + +- `pre_start_timeout`: default **60s**; set the field only to change it. + Timeout kills the hook's process group. +- Hooks run for **any** worker type (they execute on the daemon's host). + `run` is only meaningful for `path://` — on `package://` binaries the start + is implicit: exec the resolved artifact with the standard CLI contract + (`--url`, `--namespace`, `--config`). +- Precedence for `path://`: `run` in compose **>** `scripts.start` in + `iii.worker.yaml`. With neither → validation error ("add run: or create + iii.worker.yaml"). +- `setup` / `install` from the manifest are **not executed by compose** — they + remain the sandbox/registry contract. Dependency install is the developer's + job or the `pre_start`'s. + +## Execution context + +- env: the container's resolved env — the same the `run` gets (host env + + injected url/namespace/config). A migration in `pre_start` sees the same + database the worker will see. +- cwd: the container's `working_dir`. +- shell: `sh -c` on unix, `cmd /C` on windows; single string, no array form. +- isolation: each hook in its own process group / Job Object — killing a hook + never touches the worker; stdout/stderr land in the container log with a + phase prefix (`[pre_start]`, `[post_run]`), visible through + `compose::logs id=`. + +## Order + +``` +resolve config → pre_start (blocking) → spawn run → readiness +(engine registration) → … worker lifetime … → run EXITS +(down, crash, or rollback) → post_run (fire, don't wait) +``` + +`post_run` is **not awaited by ordered teardown**: it fires once the process +exit is confirmed, and the dependent cascade and registration release proceed +without waiting for it. A dependent that needs preparation before its own +start puts that logic in its own `pre_start`. + +## Why hooks in the compose file (and not only the manifest) + +The 2026-07-13 review removed `scripts:` from compose to keep it worker- +focused. This proposal brings back a **scoped** version because two real +cases have nowhere else to live: + +1. **project-level preparation** — `prisma migrate` belongs to the project + composing the worker, not to the published worker package; +2. **manifest-less local workers** — the monorepo dev case (`run: pnpm dev`) + should not require authoring a manifest first. + +The original objection (precedence stacking, compose bloat) is addressed by +the cut: three hooks, `run` only for local workers, no `setup`/`install` +override, single explicit precedence rule. + +## Deferred + +a `stop` hook (custom graceful-stop command — teardown stays signal-based; +`post_run` already covers after-exit), array-form commands, project-level +(non-container) hooks, hooks inside the sandbox path. diff --git a/website/roadmap/2026-07-14-worker-compose/src/App.tsx b/website/roadmap/2026-07-14-worker-compose/src/App.tsx new file mode 100644 index 000000000..1ea9151db --- /dev/null +++ b/website/roadmap/2026-07-14-worker-compose/src/App.tsx @@ -0,0 +1,84 @@ +import { Sheet } from '@lib/components/schematic/Sheet' +import { TopNav } from '@lib/components/TopNav' +import { useHashRoute } from '@lib/hooks/useHashRoute' +import { SpecPage } from '@lib/pages/SpecPage' +import type { ComponentType } from 'react' +import { NAV } from './content/deck' +import { CrashRestartPage } from './pages/CrashRestartPage' +import { CliSection } from './sections/CliSection' +import { ConfigSection } from './sections/ConfigSection' +import { FailureSection } from './sections/FailureSection' +import { Hero } from './sections/Hero' +import { MapSection } from './sections/MapSection' +import { NamespaceSection } from './sections/NamespaceSection' +import { OrderingSection } from './sections/OrderingSection' +import { PayoffSection } from './sections/PayoffSection' +import { RunItSection } from './sections/RunItSection' +import { ScriptsSection } from './sections/ScriptsSection' +import { WhySection } from './sections/WhySection' +import { SPEC_DOCS } from './spec-docs' + +/** + * Ordered home-page sections. The first is the hero; the rest each carry a DOM + * id matching a NAV entry in content/deck.ts for scroll-spy. + */ +const SECTIONS: ComponentType[] = [ + Hero, + WhySection, + RunItSection, + MapSection, + NamespaceSection, + ScriptsSection, + ConfigSection, + CliSection, + OrderingSection, + FailureSection, + PayoffSection, +] + +// The built-in spec viewer — never remove. It renders every markdown file of +// the paired tech-specs// directory with a file sidebar. +const Spec = () => + +/** deep-dive pages, keyed by the `#/` route slug. */ +const PAGES: Record = { + 'crash-restart': CrashRestartPage, + spec: Spec, +} + +function Home() { + return ( +
+ {SECTIONS.map((Component, i) => ( + + ))} +
+ ) +} + +function NotFound() { + return ( +
+

+ nothing here.{' '} + + ← back to the overview + +

+
+ ) +} + +export default function App() { + const route = useHashRoute() + const Page = route.kind === 'page' ? PAGES[route.slug] : undefined + + return ( +
+ + + {route.kind === 'home' ? : Page ? : } + +
+ ) +} diff --git a/website/roadmap/2026-07-14-worker-compose/src/content/cli-tracks.ts b/website/roadmap/2026-07-14-worker-compose/src/content/cli-tracks.ts new file mode 100644 index 000000000..3bac659fc --- /dev/null +++ b/website/roadmap/2026-07-14-worker-compose/src/content/cli-tracks.ts @@ -0,0 +1,79 @@ +/* run-it — the golden path and its three edges, as playable cli tracks. */ + +import type { CliTrack } from '@lib/components/diagrams/CliPlayground' + +export const CLI_TRACKS: CliTrack[] = [ + { + id: 'up', + label: 'up', + lines: [ + { + cmd: 'iii compose --id host-a --file worker-compose.yaml', + out: ['→ waiting for engine … connected', '✓ registered compose::up · down · list · status · logs · validate'], + }, + { + cmd: 'iii trigger compose::up id=host-a', + fn: 'compose::up', + out: [ + '→ config orders-database fetched · override applied', + '[pre_start] database: (none)', + '✓ database ready · ns orders', + '[pre_start] api: npx prisma migrate deploy … done', + '✓ api ready · ns orders', + '{ "status": "success", "changed": true }', + ], + exit: 0, + }, + ], + }, + { + id: 'again', + label: 'up, again', + lines: [ + { + cmd: 'iii trigger compose::up id=host-a', + fn: 'compose::up', + out: ['→ database already running · api already running', '{ "status": "success", "changed": false }'], + exit: 0, + }, + ], + }, + { + id: 'collision', + label: 'collision', + lines: [ + { + cmd: 'iii compose --id host-b --file worker-compose.yaml # same file, same namespace', + out: ['→ waiting for engine … connected'], + }, + { + cmd: 'iii trigger compose::up id=host-b', + fn: 'compose::up', + out: [ + '✗ rejected: database already registered in namespace orders (owner: host-a)', + '→ no silent overwrite. the second process exits instead of running blind.', + ], + exit: 1, + }, + ], + }, + { + id: 'down', + label: 'down', + lines: [ + { + cmd: 'iii trigger compose::down id=host-a', + fn: 'compose::down', + out: [ + '→ stopping api (dependents first)', + '[post_run] api: ./cleanup-tmp.sh fired (not awaited)', + '→ stopping database', + '[post_run] database: ./backup-on-exit.sh fired', + '✓ 0 processes left behind', + '{ "status": "success", "changed": true }', + ], + exit: 0, + }, + ], + }, +] diff --git a/website/roadmap/2026-07-14-worker-compose/src/content/cli.ts b/website/roadmap/2026-07-14-worker-compose/src/content/cli.ts new file mode 100644 index 000000000..320541511 --- /dev/null +++ b/website/roadmap/2026-07-14-worker-compose/src/content/cli.ts @@ -0,0 +1,21 @@ +/* cli contract — today's divergence vs the standard. data only. */ + +export const CLI_TODAY = [ + { worker: 'http · state · storage · database · harness', reads: '--url flag only', env: 'none', default: 'ws://127.0.0.1:49134 hardcoded' }, + { worker: 'llm-router', reads: '--url flag', env: 'III_WS_URL', default: 'ws://127.0.0.1:49134' }, + { worker: 'shell · codex · devin · email · bridge · lsp', reads: 'varies', env: 'III_URL', default: 'varies' }, + { worker: 'every sdk (register_worker)', reads: 'explicit code argument', env: 'none', default: '—' }, +] as const + +export const CLI_STANDARD = [ + { param: 'engine address', flag: '--url', env: 'III_URL' }, + { param: 'namespace', flag: '--namespace', env: 'III_NAMESPACE' }, + { param: 'configuration', flag: '--config', env: 'III_CONFIG' }, +] as const + +export const CLI_NOTES = [ + 'flags win over env; env wins over defaults.', + 'registerWorker() with no arguments reads the env contract — tutorial code carries no addresses and no namespaces.', + 'the daemon injects all three into every child and every hook; a compose file cannot override them.', + 'one shared config-registration library gives the whole fleet the contract in a dependency bump — not 41 hand-rolled parsers.', +] as const diff --git a/website/roadmap/2026-07-14-worker-compose/src/content/config.ts b/website/roadmap/2026-07-14-worker-compose/src/content/config.ts new file mode 100644 index 000000000..43ad6cf7f --- /dev/null +++ b/website/roadmap/2026-07-14-worker-compose/src/content/config.ts @@ -0,0 +1,26 @@ +/* config — three sources converge on one started process (A8). data only. */ + +import type { FunnelPath } from '@lib/components/diagrams/Funnel' + +export const CONFIG_PATHS: FunnelPath[] = [ + { id: 'defaults', label: 'worker defaults', desc: 'shipped with the package' }, + { id: 'base', label: 'base config', desc: 'configuration worker · by name' }, + { id: 'override', label: 'config_override', desc: 'sparse map in the compose file' }, +] + +export const CONFIG_TARGET = { + label: 'the process starts configured', + sub: 'daemon merges, then spawns — no startup dance', +} + +export const CONFIG_REJECT = { + label: 'full config in yaml', + desc: 'two sources of truth — rejected', +} + +export const CONFIG_RULES = [ + { name: 'reference by name', desc: 'config_name resolves through the configuration worker; storage is an adapter (fs, secrets manager, another iii). file paths are not the contract.' }, + { name: 'fetch or fail', desc: 'a container whose base fetch fails does not start — up fails before a process exists rather than booting on wrong defaults.' }, + { name: 'merge semantics', desc: 'maps merge per key; arrays and scalars replace; null is an explicit value.' }, + { name: 'why values, not a pointer', desc: '--config is a first-boot seed today ("thereafter the configuration worker is the authoritative source"). delivering resolved values keeps the compose file honest on every restart.' }, +] as const diff --git a/website/roadmap/2026-07-14-worker-compose/src/content/crash-restart.ts b/website/roadmap/2026-07-14-worker-compose/src/content/crash-restart.ts new file mode 100644 index 000000000..5d8e0b7a1 --- /dev/null +++ b/website/roadmap/2026-07-14-worker-compose/src/content/crash-restart.ts @@ -0,0 +1,52 @@ +/* crash & restart — the daemon's durable ownership record (A16). data only. */ + +import type { TimelineStage } from '@lib/components/diagrams/DurabilityTimeline' + +export const CRASH_STAGES: TimelineStage[] = [ + { + id: 'running', + label: 'running', + sub: 'steady state', + tone: 'ink', + title: 'every child has a durable record', + desc: 'the daemon persists pid, process birth identity, process-group id, and log paths per child — before anything can go wrong.', + record: { status: 'ready', step: 'supervise', calls: 'database · api', note: 'state.json: pid + birth identity + group' }, + }, + { + id: 'crash', + label: 'daemon dies', + sub: 'kill -9', + tone: 'alert', + title: 'children outlive the daemon', + desc: 'a sudden daemon crash stops nothing else. the children keep serving; the state file keeps the ownership claim.', + record: { status: 'unsupervised', step: 'children alive', calls: 'database · api', note: 'engine connection gone · state on disk' }, + gapAfter: true, + }, + { + id: 'restart', + label: 'restart', + sub: 'same id, same file', + tone: 'ink', + title: 'verify before touching anything', + desc: 'on start the daemon reads each record and verifies the live process against its birth identity. a recycled pid is never signaled.', + record: { status: 'reconciling', step: 'verify identities', calls: 'pid 48213 ✓ · pid 48214 ?', note: 'unverifiable → report, never kill' }, + }, + { + id: 'reconcile', + label: 'reconcile', + sub: 'live child', + tone: 'accent', + title: 'live and valid → managed again', + desc: 'a verified child is re-adopted: claim reconciled with the engine, supervision resumes, no restart needed.', + record: { status: 'ready', step: 'adopted', calls: 'database · api', note: 'zero downtime through the daemon crash' }, + }, + { + id: 'cleanup', + label: 'cleanup', + sub: 'dead child', + tone: 'warn', + title: 'dead → clean, cascade if needed', + desc: 'a child that died while the daemon was down is detected here: stale state removed, registration released, its post_run fired, local dependents cascaded.', + record: { status: 'failed', step: 'cascade dependents', calls: 'api stopped', note: 'post_run fired · cause in the logs' }, + }, +] diff --git a/website/roadmap/2026-07-14-worker-compose/src/content/deck.ts b/website/roadmap/2026-07-14-worker-compose/src/content/deck.ts new file mode 100644 index 000000000..4470c3411 --- /dev/null +++ b/website/roadmap/2026-07-14-worker-compose/src/content/deck.ts @@ -0,0 +1,30 @@ +/** + * deck.ts — the one file that describes THIS presentation. + * Pure data; TopNav/Footer receive it from App.tsx. + */ + +import type { DeckMeta, FooterSpec, NavItem } from '@lib/lib/deck-types' + +export const DECK_META: DeckMeta = { + wordmarkLabel: 'worker compose', +} + +/** top-nav section links — each id matches the `id` passed to a
. */ +export const NAV: NavItem[] = [ + { id: 'why', label: 'why' }, + { id: 'run-it', label: 'run it' }, + { id: 'map', label: 'map' }, + { id: 'namespace', label: 'namespace' }, + { id: 'scripts', label: 'scripts' }, + { id: 'config', label: 'config' }, + { id: 'cli', label: 'contract' }, + { id: 'payoff', label: 'payoff' }, +] + +export const FOOTER: FooterSpec = { + eyebrow: 'get started', + headline: 'one file. one contract. zero zombies.', + command: 'iii compose --id host-a --file worker-compose.yaml', + attribution: 'worker compose · distributed worker lifecycle', + source: 'source of truth: tech-specs/2026-07-14-worker-compose', +} diff --git a/website/roadmap/2026-07-14-worker-compose/src/content/failure.ts b/website/roadmap/2026-07-14-worker-compose/src/content/failure.ts new file mode 100644 index 000000000..5e4eb1484 --- /dev/null +++ b/website/roadmap/2026-07-14-worker-compose/src/content/failure.ts @@ -0,0 +1,20 @@ +/* failure — one crash fans out to an ordered stop (A7). data only. */ + +import type { FanHandler } from '@lib/components/diagrams/FanOut' + +export const FAILURE_SOURCE = { label: 'database', sub: 'crashes post-ready' } + +export const FAILURE_TRIGGER = 'cascade stop' + +export const FAILURE_HANDLERS: FanHandler[] = [ + { id: 'api', label: 'api', desc: 'stopped first — depends_on database' }, + { id: 'cron', label: 'cron-reports', desc: 'stopped next — transitive dependent' }, + { id: 'log', label: 'operator log', desc: 'cause recorded · states set to failed' }, +] + +export const FAILURE_NOTES = [ + 'the alternative is a queue worker pushing into a dead database — corruption you find later.', + 'the cascade is local: one daemon, its own children, reverse dependency order.', + 'each stopped worker’s post_run fires once its exit is confirmed — cleanup runs even on a crash.', + 'softer edges (systemd wants vs needs) can come later; v1 has one edge type and it is strict.', +] as const diff --git a/website/roadmap/2026-07-14-worker-compose/src/content/hero.ts b/website/roadmap/2026-07-14-worker-compose/src/content/hero.ts new file mode 100644 index 000000000..be19828e8 --- /dev/null +++ b/website/roadmap/2026-07-14-worker-compose/src/content/hero.ts @@ -0,0 +1,27 @@ +/* hero — the win, quantified. data only. */ + +export const HERO_STATS = [ + { value: '41→1', label: 'binaries on one cli contract' }, + { value: '2× up', label: 'one deterministic error' }, + { value: '3', label: 'hooks, one supervised run' }, + { value: '0', label: 'function ids renamed' }, +] as const + +export const HERO_CLAIMS = [ + { + title: 'a daemon per machine', + body: 'iii compose starts a supervisor, not an engine. it owns exactly the processes it spawned; several daemons attach to one engine.', + }, + { + title: 'namespace, not rename', + body: 'two projects run the same state worker side by side. function ids never change; the namespace travels as an argument.', + }, + { + title: 'config arrives finished', + body: 'the daemon fetches the base, applies the file overrides, and hands the worker its final config at start. no startup dance.', + }, + { + title: 'failure is loud and ordered', + body: 'a name collision is a rejected connection. a crashed dependency stops its dependents. nothing keeps running blind.', + }, +] as const diff --git a/website/roadmap/2026-07-14-worker-compose/src/content/map.ts b/website/roadmap/2026-07-14-worker-compose/src/content/map.ts new file mode 100644 index 000000000..318f8b9a0 --- /dev/null +++ b/website/roadmap/2026-07-14-worker-compose/src/content/map.ts @@ -0,0 +1,246 @@ +/* architecture — the system map (A4). nodes + edges + per-node datasheets. */ +import type { MapEdge, MapNode, MapNodeInfo } from '@lib/components/diagrams/SystemMap' + +/* coordinate space: 1030 x 600 (the SystemMap default viewBox) */ + +export const MAP_NODES: MapNode[] = [ + { + id: 'engine', + x: 415, + y: 48, + w: 200, + h: 70, + title: 'iii engine', + sub: 'route · arbitrate · buffer', + kind: 'primary', + tag: 'engine', + }, + { + id: 'config', + x: 770, + y: 52, + w: 200, + h: 62, + title: 'configuration', + sub: 'base configs · any adapter', + kind: 'secondary', + tag: 'builtin', + }, + { + id: 'daemon-a', + x: 120, + y: 240, + w: 210, + h: 70, + title: 'compose daemon a', + sub: 'id host-a · owns its children', + kind: 'primary', + tag: 'daemon', + }, + { + id: 'daemon-b', + x: 690, + y: 240, + w: 200, + h: 70, + title: 'compose daemon b', + sub: 'id host-b · another machine', + kind: 'secondary', + tag: 'daemon', + }, + { + id: 'database', + x: 60, + y: 460, + w: 160, + h: 58, + title: 'database', + sub: 'ns orders', + kind: 'secondary', + tag: 'child', + }, + { id: 'api', x: 250, y: 460, w: 160, h: 58, title: 'api', sub: 'ns orders', kind: 'secondary', tag: 'child' }, + { + id: 'state', + x: 740, + y: 460, + w: 160, + h: 58, + title: 'state', + sub: 'ns analytics', + kind: 'secondary', + tag: 'child', + }, +] + +export const MAP_EDGES: MapEdge[] = [ + { + id: 'a-engine', + from: 'daemon-a', + to: 'engine', + d: 'M 225 240 L 225 180 L 468 180 L 468 118', + label: 'control · compose::* (id host-a)', + lx: 250, + ly: 172, + anchor: 'start', + }, + { + id: 'b-engine', + from: 'daemon-b', + to: 'engine', + d: 'M 790 240 L 790 180 L 562 180 L 562 118', + label: 'control · compose::* (id host-b)', + lx: 782, + ly: 172, + anchor: 'end', + }, + { + id: 'engine-config', + from: 'engine', + to: 'config', + d: 'M 615 83 L 770 83', + label: 'fetch base', + lx: 693, + ly: 73, + anchor: 'middle', + dashed: true, + }, + { + id: 'a-database', + from: 'daemon-a', + to: 'database', + d: 'M 180 310 L 180 400 L 140 400 L 140 460', + label: 'supervise', + lx: 168, + ly: 394, + anchor: 'end', + }, + { id: 'a-api', from: 'daemon-a', to: 'api', d: 'M 270 310 L 270 400 L 330 400 L 330 460' }, + { id: 'b-state', from: 'daemon-b', to: 'state', d: 'M 790 310 L 790 385 L 820 385 L 820 460' }, + { + id: 'api-engine', + from: 'api', + to: 'engine', + d: 'M 410 489 L 495 489 L 495 118', + label: 'register · ns orders', + lx: 505, + ly: 420, + anchor: 'start', + dashed: true, + dur: 3.2, + }, + { + id: 'state-engine', + from: 'state', + to: 'engine', + d: 'M 900 460 L 900 205 L 585 205 L 585 118', + label: 'register · ns analytics', + lx: 908, + ly: 350, + anchor: 'start', + dashed: true, + dur: 3.2, + }, +] + +export const MAP_INFO: Record = { + engine: { + id: 'engine', + kindLabel: 'engine · coordinator', + role: 'the shared router and the single arbitration point. it never supervises a process.', + sections: [ + { + heading: 'does', + dotted: true, + items: [ + { name: 'route triggers', desc: 'function ids stay verbatim; namespace travels beside them.' }, + { name: 'reject collisions', desc: 'same name + same namespace = refused registration, not overwrite.' }, + { name: 'buffer registrations', desc: 'triggers for absent workers wait with a bounded ttl, then flush.' }, + ], + }, + ], + bullets: ['readiness truth: a worker is ready when its registration is visible here.'], + }, + config: { + id: 'config', + kindLabel: 'engine builtin', + role: 'holds base configurations by name; the storage behind it is an adapter (fs today, secrets manager or another iii in cloud).', + sections: [ + { + heading: 'contract', + items: [ + { name: 'config_name', desc: 'compose containers reference configs by name, never by file path.' }, + { name: 'fetch-before-spawn', desc: 'the daemon pulls the base before any process exists.' }, + ], + }, + ], + note: 'names keep the compose file valid when storage moves — nothing in it points at a filesystem.', + }, + 'daemon-a': { + id: 'daemon-a', + kindLabel: 'compose daemon', + role: 'one process per machine, bound to one compose file for its lifetime. the only supervisor of its own children.', + sections: [ + { + heading: 'remote surface', + dotted: true, + items: [ + { name: 'compose::up / down', desc: 'write — start or stop the graph, dependents ordered. addressed by id=host-a.' }, + { name: 'compose::list / status / logs', desc: 'read — the addressed daemon’s compose project only.' }, + { name: 'compose::validate', desc: 'read — revalidate the bound file.' }, + ], + }, + { + heading: 'per child', + items: [ + { name: 'inject', desc: 'III_URL · III_NAMESPACE · III_CONFIG into every spawn and hook.' }, + { name: 'own', desc: 'process group / job object; sigterm → grace → sigkill.' }, + ], + }, + ], + install: 'iii compose --id host-a --file worker-compose.yaml', + }, + 'daemon-b': { + id: 'daemon-b', + kindLabel: 'compose daemon', + role: 'a second daemon on a second machine, attached to the same engine. it cannot touch host-a’s children.', + bullets: [ + 'sharing a worker across files is a namespace decision, not a cross-file dependency.', + 'down here never cascades to another daemon’s processes.', + ], + }, + database: { + id: 'database', + kindLabel: 'child worker', + role: 'a package:// binary. no run script — compose execs the resolved artifact with the standard flags.', + sections: [ + { + heading: 'start', + items: [ + { name: 'implicit', desc: 'exec artifact + --url --namespace --config.' }, + { name: 'ready', desc: 'when its registration appears on the engine.' }, + ], + }, + ], + }, + api: { + id: 'api', + kindLabel: 'child worker', + role: 'a path:// local worker. run: "pnpm dev" in the compose file — no manifest required.', + sections: [ + { + heading: 'hooks', + items: [ + { name: 'pre_start', desc: 'prisma migrate, blocking, 60s default budget.' }, + { name: 'post_run', desc: 'cleanup script, fired after the run exits, never awaited.' }, + ], + }, + ], + }, + state: { + id: 'state', + kindLabel: 'child worker', + role: 'the same published state worker another project also runs — different namespace, zero renames.', + bullets: ['function ids identical to every other state instance; the namespace disambiguates.'], + }, +} diff --git a/website/roadmap/2026-07-14-worker-compose/src/content/namespace.ts b/website/roadmap/2026-07-14-worker-compose/src/content/namespace.ts new file mode 100644 index 000000000..51fc998db --- /dev/null +++ b/website/roadmap/2026-07-14-worker-compose/src/content/namespace.ts @@ -0,0 +1,62 @@ +/* namespace — the routing story as a sequence (A5). data only. */ + +import type { SeqLane, SeqStep } from '@lib/components/diagrams/SequencePlayer' + +export const NS_LANES: SeqLane[] = [ + { id: 'caller', label: 'caller', x: 90 }, + { id: 'engine', label: 'engine', x: 390 }, + { id: 'default', label: 'state · ns default', x: 660 }, + { id: 'analytics', label: 'state · ns analytics', x: 890 }, +] + +export const NS_STEPS: SeqStep[] = [ + { + from: 'default', + to: 'engine', + label: 'register state', + title: 'first instance registers', + desc: 'started by hand, no env — it lands in the default namespace. function ids stay exactly what it registered.', + }, + { + from: 'analytics', + to: 'engine', + label: 'register state', + title: 'same name, second namespace', + desc: 'the same published worker with III_NAMESPACE=analytics injected. accepted — the namespace is a separate dimension, not a rename.', + }, + { + from: 'caller', + to: 'engine', + label: 'trigger("state::get")', + title: 'no namespace argument', + desc: 'strict rule: no namespace means the default namespace, and only it. no best-fit guessing.', + }, + { + from: 'engine', + to: 'default', + label: 'route → ns default', + title: 'default resolves', + desc: 'the call lands on the default-namespace instance. a miss would be a clear not-found listing where the id does exist.', + }, + { + from: 'caller', + to: 'engine', + label: 'trigger("state::get", namespace: "analytics")', + title: 'explicit namespace', + desc: 'the namespace argument targets that instance precisely — the only way a namespaced instance is reached.', + }, + { + from: 'engine', + to: 'analytics', + label: 'route → ns analytics', + title: 'targeted delivery', + desc: 'two instances of one worker coexist on one engine, addressable without renaming a single function.', + }, + { + from: 'caller', + to: 'engine', + label: 'register state (ns analytics)', + title: 'collision — rejected', + desc: 'a second state in the same namespace is refused at registration. today the engine warns and overwrites; this makes it a hard error.', + }, +] diff --git a/website/roadmap/2026-07-14-worker-compose/src/content/ordering.ts b/website/roadmap/2026-07-14-worker-compose/src/content/ordering.ts new file mode 100644 index 000000000..6e29d58bb --- /dev/null +++ b/website/roadmap/2026-07-14-worker-compose/src/content/ordering.ts @@ -0,0 +1,47 @@ +/* ordering — the optimistic registration buffer as a sequence (A5). data only. */ + +import type { SeqLane, SeqStep } from '@lib/components/diagrams/SequencePlayer' + +export const ORDER_LANES: SeqLane[] = [ + { id: 'api', label: 'api worker', x: 120 }, + { id: 'engine', label: 'engine', x: 460 }, + { id: 'router', label: 'http router', x: 820 }, +] + +export const ORDER_STEPS: SeqStep[] = [ + { + from: 'api', + to: 'engine', + label: 'register route (http trigger)', + title: 'the route arrives early', + desc: 'the api worker registers an http route, but the router worker is not up yet. today: a warning, and the route is dropped.', + }, + { + from: 'engine', + to: 'engine', + label: 'hold in buffer', + title: 'optimistic, not lossy', + desc: 'the engine keeps the registration in an in-memory buffer with a bounded ttl — compose exposes the timeout and retry knobs.', + }, + { + from: 'router', + to: 'engine', + label: 'router registers', + title: 'the trigger owner arrives', + desc: 'the http router worker comes up (first boot or a restart) and registers its trigger type.', + }, + { + from: 'engine', + to: 'router', + label: 'flush buffer → route live', + title: 'the buffer flushes', + desc: 'every held registration replays automatically. no worker had to know about the ordering.', + }, + { + from: 'engine', + to: 'api', + label: 'route confirmed', + title: 'order is now a performance concern', + desc: 'depends_on stays about data dependencies; registration order stops being a correctness bug.', + }, +] diff --git a/website/roadmap/2026-07-14-worker-compose/src/content/payoff.ts b/website/roadmap/2026-07-14-worker-compose/src/content/payoff.ts new file mode 100644 index 000000000..9093fe2ae --- /dev/null +++ b/website/roadmap/2026-07-14-worker-compose/src/content/payoff.ts @@ -0,0 +1,18 @@ +/* payoff — problem → answer, plus the honest costs. data only. */ + +export const SCORECARD = [ + { problem: 'duplicate names overwrite silently', answer: 'rejected registration inside a namespace — fail early, no zombies' }, + { problem: 'three conventions to find the engine', answer: 'one cli/env contract: --url · --namespace · --config, zero-arg registerWorker()' }, + { problem: 'config applies on first boot only', answer: 'daemon fetches base, merges overrides, delivers the final config at start' }, + { problem: 'startup order drops routes', answer: 'optimistic buffer with bounded ttl — order becomes performance, not correctness' }, + { problem: 'a dead dependency corrupts quietly', answer: 'cascading stop in reverse dependency order, cause in the logs' }, + { problem: 'lifecycle welded to the engine host', answer: 'one daemon per machine, all attached to one engine' }, +] as const + +export const PAYOFF_TRADEOFFS_TITLE = 'the honest costs' + +export const TRADEOFFS = [ + { title: 'the namespace surface is wide', body: 'register + trigger protocol messages, engine routing, and all three sdks change together. it is the largest single piece and it is priced in, not hidden.' }, + { title: 'scripts return to the compose file', body: 'the 2026-07-13 review removed them; this pack brings back a scoped version (three hooks, run only for local workers). needs explicit re-approval.' }, + { title: 'artifacts are per-target', body: 'a real release ships 9 platform binaries with 9 digests — reproducibility metadata is per-target from day one or it is wrong.' }, +] as const diff --git a/website/roadmap/2026-07-14-worker-compose/src/content/scripts.ts b/website/roadmap/2026-07-14-worker-compose/src/content/scripts.ts new file mode 100644 index 000000000..c84a6dd10 --- /dev/null +++ b/website/roadmap/2026-07-14-worker-compose/src/content/scripts.ts @@ -0,0 +1,77 @@ +/* scripts — the container lifecycle as a step reveal (A6). data only. */ + +import type { RevealStage } from '@lib/components/diagrams/StepReveal' + +export const SCRIPT_STAGES: RevealStage[] = [ + { + label: 'resolve config', + caption: 'base fetched from the configuration worker, config_override merged. failure here means the container never starts.', + rows: [ + { k: 'base', v: 'orders-api · fetched' }, + { k: 'override', v: 'server.port: 3000 · applied' }, + { k: 'process', v: 'none yet' }, + ], + }, + { + label: 'pre_start', + caption: 'npx prisma migrate deploy — blocking, in its own process group, 60s default budget (pre_start_timeout to change).', + rows: [ + { k: 'hook', v: 'running · blocking' }, + { k: 'budget', v: '60s default' }, + { k: 'on failure', v: 'container failed · no spawn · rollback' }, + ], + }, + { + label: 'spawn run', + caption: 'path:// worker: the compose run command. package:// binary: exec the artifact with --url --namespace --config.', + rows: [ + { k: 'pid', v: '48213 · own process group' }, + { k: 'injected', v: 'III_URL · III_NAMESPACE · III_CONFIG' }, + { k: 'status', v: 'starting' }, + ], + }, + { + label: 'ready', + tone: 'accent', + caption: 'readiness is the engine seeing the registration — not a live pid. dependents may start now.', + rows: [ + { k: 'registered', v: 'api::* · ns orders' }, + { k: 'status', v: 'ready' }, + { k: 'dependents', v: 'unblocked' }, + ], + }, + { + label: 'run exits', + tone: 'warn', + caption: 'any exit path — down, a post-ready crash, or an up rollback. exit is confirmed before anything else happens.', + rows: [ + { k: 'status', v: 'exited' }, + { k: 'cause', v: 'down · crash · rollback' }, + { k: 'registration', v: 'released after confirmed exit' }, + ], + }, + { + label: 'post_run', + caption: './cleanup-tmp.sh — fired once after the exit is confirmed, never awaited. a non-zero exit is a warning, not a blocker.', + rows: [ + { k: 'hook', v: 'fired · not awaited' }, + { k: 'on failure', v: 'warning + last_error' }, + { k: 'teardown', v: 'proceeds without waiting' }, + ], + }, +] + +/* the schema behind the claim — rendered in a SpecSheet. */ +export const SCRIPT_FIELDS = [ + { name: 'pre_start', desc: 'blocking; runs before every spawn; exit ≠ 0 or timeout fails the container.' }, + { name: 'pre_start_timeout', desc: 'default 60s; set the field only to change it; timeout kills the hook’s process group.' }, + { name: 'run', desc: 'the supervised process. path:// only — required when the worker has no manifest. compose run wins over manifest scripts.start.' }, + { name: 'post_run', desc: 'fired once after the run process exits — any exit path (down, crash, rollback); not awaited; failure = warning + last_error.' }, +] as const + +export const SCRIPT_RULES = [ + 'run on a package:// worker → validation error (binaries start implicitly).', + 'path:// with neither manifest nor run → error: add run: or create iii.worker.yaml.', + 'setup/install from the manifest are never executed by compose — sandbox contract.', + 'no stop hook in v1: teardown is sigterm → grace → sigkill; post_run covers after-exit.', +] as const diff --git a/website/roadmap/2026-07-14-worker-compose/src/content/why.ts b/website/roadmap/2026-07-14-worker-compose/src/content/why.ts new file mode 100644 index 000000000..60177ceb7 --- /dev/null +++ b/website/roadmap/2026-07-14-worker-compose/src/content/why.ts @@ -0,0 +1,34 @@ +/* why — six concrete failures in today's lifecycle, each cited. data only. */ + +export const WHY_CARDS = [ + { + title: 'duplicate names overwrite silently', + body: 'the engine warns and replaces on a name collision — a unit test codifies the overwrite. the losing process keeps running blind in the background.', + cite: 'engine/src/services.rs:106', + }, + { + title: 'three ways to find the engine', + body: 'http, state, storage and database read only a --url flag with a hardcoded default. llm-router reads III_WS_URL. shell reads III_URL. no sdk reads any of them.', + cite: 'workers/http/src/main.rs:28', + }, + { + title: 'config applies once, then never', + body: 'a worker’s --config seeds its configuration entry on first boot only; afterwards the flag is ignored. what a compose file says is not what a restarted process gets.', + cite: 'workers/http/src/main.rs:23', + }, + { + title: 'startup order drops routes', + body: 'register a route before the router worker is up and the engine prints a warning and forgets it. ordering is a correctness bug today.', + cite: '2026-07-13 review', + }, + { + title: 'lifecycle is welded to one machine', + body: 'iii worker manages processes only where the engine runs. workers cannot live where their resources are.', + cite: 'crates/iii-worker', + }, + { + title: 'a dead dependency corrupts quietly', + body: 'a queue worker keeps pushing into a crashed database. nothing stops the chain, nothing tells you.', + cite: 'lifecycle.md', + }, +] as const diff --git a/website/roadmap/2026-07-14-worker-compose/src/pages/CrashRestartPage.tsx b/website/roadmap/2026-07-14-worker-compose/src/pages/CrashRestartPage.tsx new file mode 100644 index 000000000..d0adaaf1a --- /dev/null +++ b/website/roadmap/2026-07-14-worker-compose/src/pages/CrashRestartPage.tsx @@ -0,0 +1,45 @@ +import { DurabilityTimeline } from '@lib/components/diagrams/DurabilityTimeline' +import { PageShell } from '@lib/components/PageShell' +import { SpecRow, SpecSheet } from '@lib/components/SpecSheet' +import { CRASH_STAGES } from '../content/crash-restart' + +/** + * A14 deep dive — the daemon's own crash. Children outlive it; a durable + * per-child record (pid + birth identity + group) makes restart reconciliation + * safe: verify, re-adopt the living, clean the dead, never signal a pid it + * cannot prove is the same process. + */ +export function CrashRestartPage() { + return ( + + + +
+ + + losing the engine connection never stops children; the daemon reconnects and reconciles. + + + a recorded pid is signaled only after its birth identity matches — a recycled pid is reported, never + killed. + + + stale state removed, registration released, local dependents cascaded, cause logged. + + + anything the daemon cannot prove it owns is reported for manual cleanup instead of guessed at. + + +
+ + ) +} diff --git a/website/roadmap/2026-07-14-worker-compose/src/sections/CliSection.tsx b/website/roadmap/2026-07-14-worker-compose/src/sections/CliSection.tsx new file mode 100644 index 000000000..473cf5ca4 --- /dev/null +++ b/website/roadmap/2026-07-14-worker-compose/src/sections/CliSection.tsx @@ -0,0 +1,75 @@ +import { Section } from '@lib/components/Section' +import { C, CodeBlock, K, M, S } from '@lib/components/schematic/CodeBlock' +import { Cell } from '@lib/components/schematic/Cell' +import { FnChip } from '@lib/components/schematic/FnChip' +import { CLI_NOTES, CLI_STANDARD, CLI_TODAY } from '../content/cli' + +/** + * Before/after — 41 binaries and three conventions on the left, one contract + * on the right. The daemon can only inject what workers agree to read. + */ +export function CliSection() { + return ( +
+
+
+
today
+
+ {CLI_TODAY.map((row) => ( +
+
{row.worker}
+
+ reads {row.reads} · env {row.env} · default {row.default} +
+
+ ))} +
+
+ +
+
the standard
+
+ {CLI_STANDARD.map((row) => ( +
+ {row.param} + {row.flag} + {row.env} +
+ ))} +
+
+ +
+ import {'{ registerWorker }'} from 'iii' +
+
 
+
+ {'// no url. no namespace. no config path.'} +
+
+ {'// the env contract fills all three — under compose, iii worker, or by hand.'} +
+
+ const iii = registerWorker() +
+ +
+
+
+ +
+ {CLI_NOTES.map((note, i) => ( + + {note} + + ))} +
+
+ ) +} diff --git a/website/roadmap/2026-07-14-worker-compose/src/sections/ConfigSection.tsx b/website/roadmap/2026-07-14-worker-compose/src/sections/ConfigSection.tsx new file mode 100644 index 000000000..132116a4f --- /dev/null +++ b/website/roadmap/2026-07-14-worker-compose/src/sections/ConfigSection.tsx @@ -0,0 +1,37 @@ +import { Funnel } from '@lib/components/diagrams/Funnel' +import { Section } from '@lib/components/Section' +import { SpecRow, SpecSheet } from '@lib/components/SpecSheet' +import { CONFIG_PATHS, CONFIG_REJECT, CONFIG_RULES, CONFIG_TARGET } from '../content/config' + +/** + * A8 — three config sources converge on one started process; the fourth path + * (a full config body in yaml) is the one this design rejects. + */ +export function ConfigSection() { + return ( +
+ + +
+ + {CONFIG_RULES.map((rule) => ( + + {rule.desc} + + ))} + +
+
+ ) +} diff --git a/website/roadmap/2026-07-14-worker-compose/src/sections/FailureSection.tsx b/website/roadmap/2026-07-14-worker-compose/src/sections/FailureSection.tsx new file mode 100644 index 000000000..5f272ff80 --- /dev/null +++ b/website/roadmap/2026-07-14-worker-compose/src/sections/FailureSection.tsx @@ -0,0 +1,49 @@ +import { FanOut } from '@lib/components/diagrams/FanOut' +import { Section } from '@lib/components/Section' +import { StatusPanel } from '@lib/components/schematic/StatusPanel' +import { FAILURE_HANDLERS, FAILURE_NOTES, FAILURE_SOURCE, FAILURE_TRIGGER } from '../content/failure' + +/** + * A7 — a post-ready crash fans out to an ordered local stop. The deep story + * (daemon crash + restart reconciliation) lives on the crash-restart page. + */ +export function FailureSection() { + return ( +
+ + +
+ +
+
scope
+ {FAILURE_NOTES.slice(1).map((note, i) => ( +

+ {note} +

+ ))} + + deep dive: the daemon itself crashes → + +
+
+
+ ) +} diff --git a/website/roadmap/2026-07-14-worker-compose/src/sections/Hero.tsx b/website/roadmap/2026-07-14-worker-compose/src/sections/Hero.tsx new file mode 100644 index 000000000..42430fe53 --- /dev/null +++ b/website/roadmap/2026-07-14-worker-compose/src/sections/Hero.tsx @@ -0,0 +1,108 @@ +import { Caret } from '@lib/components/schematic/Caret' +import { Cell } from '@lib/components/schematic/Cell' +import { Prompt } from '@lib/components/schematic/Prompt' +import { StatusDot } from '@lib/components/schematic/StatusDot' +import { Terminal, TerminalRow } from '@lib/components/schematic/Terminal' +import { HERO_CLAIMS, HERO_STATS } from '../content/hero' + +/** + * A1 — the hero. The win in one line, never the mechanism. The terminal shows + * the boundary in two frames: a clean up, and a collision that fails loud. + */ +export function Hero() { + return ( +
+
+
+
+ iii / tech-specs / worker-compose +
+

+ one file. +
+ one contract. +
+ zero zombies. +

+

+ a compose daemon per machine supervises the workers one file declares:{' '} + namespaced so nothing collides,{' '} + config-fed so nothing boots wrong, and{' '} + crash-cascading so nothing fails quietly. +

+ +
+ +
+ +
+ + + database ready · api ready · ns orders + + } + /> + ✗ rejected · database already registered in ns orders} + /> +
+ + +
+
+ + +
+ {HERO_STATS.map((stat) => ( +
+
+ {stat.value} +
+
+ {stat.label} +
+
+ ))} +
+
+
+ +
+
+ {HERO_CLAIMS.map((claim, i) => ( + + 0{i + 1} + {claim.title} + + } + className="border-0" + bodyClassName="max-w-[36ch]" + > + {claim.body} + + ))} +
+
+
+ ) +} diff --git a/website/roadmap/2026-07-14-worker-compose/src/sections/MapSection.tsx b/website/roadmap/2026-07-14-worker-compose/src/sections/MapSection.tsx new file mode 100644 index 000000000..da39d0c00 --- /dev/null +++ b/website/roadmap/2026-07-14-worker-compose/src/sections/MapSection.tsx @@ -0,0 +1,83 @@ +import { MapDatasheet, SystemMap } from '@lib/components/diagrams/SystemMap' +import { Section } from '@lib/components/Section' +import { StatusDot } from '@lib/components/schematic/StatusDot' +import { useEffect, useRef, useState } from 'react' +import { MAP_EDGES, MAP_INFO, MAP_NODES } from '../content/map' + +/** matches tailwind @5xl container width (64rem) */ +const PAIRED_LAYOUT_MIN_WIDTH = 1024 + +const LEGEND = [ + { swatch: , label: 'engine / daemon' }, + { swatch: , label: 'worker / builtin' }, + { swatch: , label: 'active flow' }, +] as const + +/** + * A4 — the architecture in one navigable map: the engine routes and + * arbitrates, each daemon supervises only its own children. + */ +export function MapSection() { + const [selected, setSelected] = useState('daemon-a') + const layoutRef = useRef(null) + const mapRef = useRef(null) + const [pairedLayout, setPairedLayout] = useState(false) + const [mapHeight, setMapHeight] = useState() + + useEffect(() => { + const layoutEl = layoutRef.current + const mapEl = mapRef.current + if (!layoutEl || !mapEl) return + + const sync = () => { + const paired = layoutEl.clientWidth >= PAIRED_LAYOUT_MIN_WIDTH + setPairedLayout(paired) + setMapHeight(paired ? mapEl.offsetHeight : undefined) + } + + sync() + const observer = new ResizeObserver(sync) + observer.observe(layoutEl) + observer.observe(mapEl) + return () => observer.disconnect() + }, []) + + const info = MAP_INFO[selected] ?? MAP_INFO['daemon-a'] + + return ( +
+
+ {LEGEND.map((item) => ( + + {item.swatch} + {item.label} + + ))} +
+ +
+
+
+ +
+
+
+ +
+
+
+ ) +} diff --git a/website/roadmap/2026-07-14-worker-compose/src/sections/NamespaceSection.tsx b/website/roadmap/2026-07-14-worker-compose/src/sections/NamespaceSection.tsx new file mode 100644 index 000000000..31399708a --- /dev/null +++ b/website/roadmap/2026-07-14-worker-compose/src/sections/NamespaceSection.tsx @@ -0,0 +1,42 @@ +import { SequencePlayer } from '@lib/components/diagrams/SequencePlayer' +import { Section } from '@lib/components/Section' +import { SpecRow, SpecSheet } from '@lib/components/SpecSheet' +import { NS_LANES, NS_STEPS } from '../content/namespace' + +/** + * A5 — the namespace protocol: register twice under two namespaces, route + * with and without one, and watch a same-namespace collision get rejected. + */ +export function NamespaceSection() { + return ( +
+ + +
+ + + trigger() without a namespace resolves in the default namespace, and only there; an explicit namespace + targets exactly that instance. no best-fit guessing. + + + an autogenerated namespace per compose project — two files never collide by accident. + + the default namespace; the single-user flow is unchanged. + + both declare the shared namespace explicitly; exactly one file owns the process. + + + register + trigger protocol messages, engine routing, and all three sdks change together — the widest + surface in this pack. + + +
+
+ ) +} diff --git a/website/roadmap/2026-07-14-worker-compose/src/sections/OrderingSection.tsx b/website/roadmap/2026-07-14-worker-compose/src/sections/OrderingSection.tsx new file mode 100644 index 000000000..58c2fa569 --- /dev/null +++ b/website/roadmap/2026-07-14-worker-compose/src/sections/OrderingSection.tsx @@ -0,0 +1,21 @@ +import { SequencePlayer } from '@lib/components/diagrams/SequencePlayer' +import { Section } from '@lib/components/Section' +import { ORDER_LANES, ORDER_STEPS } from '../content/ordering' + +/** + * A5 — the optimistic registration buffer: a route that arrives before its + * router waits in the engine instead of being dropped. + */ +export function OrderingSection() { + return ( +
+ +
+ ) +} diff --git a/website/roadmap/2026-07-14-worker-compose/src/sections/PayoffSection.tsx b/website/roadmap/2026-07-14-worker-compose/src/sections/PayoffSection.tsx new file mode 100644 index 000000000..cce8752c5 --- /dev/null +++ b/website/roadmap/2026-07-14-worker-compose/src/sections/PayoffSection.tsx @@ -0,0 +1,44 @@ +import { Section } from '@lib/components/Section' +import { Cell } from '@lib/components/schematic/Cell' +import { PAYOFF_TRADEOFFS_TITLE, SCORECARD, TRADEOFFS } from '../content/payoff' + +/** + * A11 — the payoff: every problem from the why section answered, plus the + * honest costs. Admitted limits read as credibility. + */ +export function PayoffSection() { + return ( +
+
+ {SCORECARD.map((row) => ( +
+ {row.problem} + {row.answer} +
+ ))} +
+ +
+
+ {PAYOFF_TRADEOFFS_TITLE} +
+
+ {TRADEOFFS.map((item) => ( + + {item.body} + + ))} +
+
+
+ ) +} diff --git a/website/roadmap/2026-07-14-worker-compose/src/sections/RunItSection.tsx b/website/roadmap/2026-07-14-worker-compose/src/sections/RunItSection.tsx new file mode 100644 index 000000000..11ac095d7 --- /dev/null +++ b/website/roadmap/2026-07-14-worker-compose/src/sections/RunItSection.tsx @@ -0,0 +1,21 @@ +import { CliPlayground } from '@lib/components/diagrams/CliPlayground' +import { Section } from '@lib/components/Section' +import { CLI_TRACKS } from '../content/cli-tracks' + +/** + * A3 — the demo, early. Four tracks: the golden path, the idempotent repeat, + * the collision that fails loud, and the ordered teardown. + */ +export function RunItSection() { + return ( +
+ +
+ ) +} diff --git a/website/roadmap/2026-07-14-worker-compose/src/sections/ScriptsSection.tsx b/website/roadmap/2026-07-14-worker-compose/src/sections/ScriptsSection.tsx new file mode 100644 index 000000000..ec44d5e2e --- /dev/null +++ b/website/roadmap/2026-07-14-worker-compose/src/sections/ScriptsSection.tsx @@ -0,0 +1,40 @@ +import { StepReveal } from '@lib/components/diagrams/StepReveal' +import { Section } from '@lib/components/Section' +import { SpecRow, SpecSheet } from '@lib/components/SpecSheet' +import { SCRIPT_FIELDS, SCRIPT_RULES, SCRIPT_STAGES } from '../content/scripts' + +/** + * A6 — the container lifecycle: prepare (blocking), supervise (the run), + * clean up (fired after exit, not awaited). One supervised process, two + * one-shot hooks. + */ +export function ScriptsSection() { + return ( +
+ + +
+ + {SCRIPT_FIELDS.map((f) => ( + + {f.desc} + + ))} + + + {SCRIPT_RULES.map((rule, i) => ( + + {rule} + + ))} + +
+
+ ) +} diff --git a/website/roadmap/2026-07-14-worker-compose/src/sections/WhySection.tsx b/website/roadmap/2026-07-14-worker-compose/src/sections/WhySection.tsx new file mode 100644 index 000000000..1e24531d9 --- /dev/null +++ b/website/roadmap/2026-07-14-worker-compose/src/sections/WhySection.tsx @@ -0,0 +1,37 @@ +import { Section } from '@lib/components/Section' +import { Cell } from '@lib/components/schematic/Cell' +import { WHY_CARDS } from '../content/why' + +/** + * A2 — the pain. Six concrete failures in today's lifecycle, each cited to + * code or to the review that surfaced it. + */ +export function WhySection() { + return ( +
+
+ {WHY_CARDS.map((card, i) => ( + + 0{i + 1} + {card.title} + + } + className="border-0" + > +

{card.body}

+

{card.cite}

+ + ))} +
+
+ ) +} diff --git a/website/roadmap/2026-07-14-worker-compose/src/spec-docs.ts b/website/roadmap/2026-07-14-worker-compose/src/spec-docs.ts new file mode 100644 index 000000000..fdb0c33b6 --- /dev/null +++ b/website/roadmap/2026-07-14-worker-compose/src/spec-docs.ts @@ -0,0 +1,12 @@ +/** + * Bundles this spec's markdown for the #/spec page. The glob MUST live + * deck-side (import.meta.glob resolves relative to the importing file); its + * literal is substituted at scaffold time because the relative depth from + * //src/ to // depends on where the base project + * lives (in iii: '../../../../tech-specs//*.md'). + */ +export const SPEC_DOCS = import.meta.glob('../../../../tech-specs/2026-07-14-worker-compose/*.md', { + query: '?raw', + import: 'default', + eager: true, +}) as Record