docs(tech-specs): worker-compose tech pack + interactive deck

tech-specs/2026-07-14-worker-compose/README.md

@@ -0,0 +1,86 @@
+---
+title: worker compose — distributed worker lifecycle
+tagline: one daemon per machine supervises the workers a compose file declares — namespaced, config-fed, crash-cascading, engine-coordinated.
+date: 2026-07-14
+tags: [compose, workers, lifecycle, dx]
+status: draft
+---
+
+# Worker Compose
+
+A standalone **compose daemon** that supervises iii workers declared in one
+`worker-compose.yaml`. The daemon is not the engine: `iii compose` starts only
+the daemon, which connects to an engine like any worker, registers the
+`compose::up/down/list/status/logs/validate` control functions, and owns
+exactly the child processes it spawned on its own machine. Several daemons —
+several machines — attach to one engine; a trigger addresses one of them by
+argument: `iii trigger compose::up id=host-a`.
+
+The load-bearing separation: **the engine routes and arbitrates; the daemon
+supervises.** Worker lifecycle today is welded to the machine the engine runs
+on (`iii worker` + iii-worker-ops). Compose breaks that weld so workers can
+run where their resources are, while the engine stays the single coordination
+point: it rejects duplicate registrations inside a namespace, holds triggers
+for workers that have not arrived yet, and answers "who is running what".
+
+## What this pack proposes
+
+Six contracts, each in its own doc:
+
+| Doc | Contract |
+| --- | --- |
+| [compose-file.md](./compose-file.md) | `worker-compose.yaml` v1 — an id-keyed `containers:` object; `depends_on` inside one file only |
+| [scripts.md](./scripts.md) | per-container `scripts:` — `pre_start` (blocking, timed), `run` (supervised), `post_run` (fires after the run exits) |
+| [namespace.md](./namespace.md) | namespace as a runtime argument — never a function-name prefix; engine rejects same-name collisions inside a namespace |
+| [configuration.md](./configuration.md) | the daemon fetches base config, applies compose overrides, hands the worker its final config at start |
+| [cli-contract.md](./cli-contract.md) | one CLI/env contract for every worker binary: `--url`, `--namespace`, `--config`; `registerWorker()` reads env |
+| [lifecycle.md](./lifecycle.md) | up/down over a validated DAG, readiness = engine registration, cascading failure, optimistic trigger buffer |
+
+## Why now
+
+Three facts from the current codebase force each major piece:
+
+1. **Duplicate registration is a silent overwrite.** The engine warns and
+   replaces on a name collision (`engine/src/services.rs:106-111` — and a unit
+   test codifies it: `registry_insert_service_duplicate_overwrites`). Running
+   two state workers means the last one wins and the first keeps running
+   blind. Namespaces + hard rejection replace that with a deterministic error.
+2. **No two worker binaries agree on how to find the engine.** `http`, `state`,
+   `storage`, `database` take only a `--url` flag with a hardcoded
+   `ws://127.0.0.1:49134` default and read no env; `llm-router` reads
+   `III_WS_URL`; `shell` and friends read `III_URL`. No SDK reads any address
+   env at all. A supervisor cannot inject a contract that does not exist —
+   the CLI/env standard creates it.
+3. **Config delivery is first-boot-only.** A worker's `--config` seeds the
+   configuration entry once; afterwards "the configuration worker is the
+   authoritative source and `--config` is ignored" (`workers/http/src/main.rs`
+   doc comment). Compose-managed config therefore cannot be a file passed at
+   spawn — the daemon has to resolve base + overrides and deliver the final
+   value through the standard contract.
+
+## Principles (decided 2026-07-13, team review)
+
+- **compose over docker**: bare-metal child processes locally; the sandbox
+  path stays the validated route to cloud. Docker is not the default runtime.
+- **fail early over silent overwrite**: same name + same namespace = rejected
+  connection, not a zombie.
+- **one file, one dependency scope**: `depends_on` never crosses compose
+  files. Sharing across files is a namespace decision, not a dependency edge.
+- **worker knowledge stays with the worker**: compose does not carry full
+  configurations, and `setup`/`install` remain the worker/sandbox contract.
+- **zero-code onboarding**: `registerWorker()` with no arguments must work —
+  the daemon injects url/namespace/config through env; developers learn
+  namespaces only when they need two of something.
+
+## Honest trade-offs
+
+- The namespace model requires protocol + all-three-SDK changes
+  (register/trigger carry a namespace; env contract) — the widest surface in
+  the pack.
+- `run` in the compose file partially reverses the 2026-07-13 "no scripts in
+  compose" decision; the counter-argument is scoped: three hooks, `run` only
+  for local workers, no setup/install override.
+- Per-target artifacts (a real release ships 9 platform binaries with 9
+  digests) mean reproducibility metadata must be per-target from day one.
+- The optimistic trigger buffer trades "fail loud on missing worker" for
+  "wait with a bounded timeout" — the timeout knob is what keeps that honest.

tech-specs/2026-07-14-worker-compose/cli-contract.md

@@ -0,0 +1,52 @@
+# the worker CLI/env contract
+
+A supervisor can only inject what workers agree to read. Today they agree on
+nothing:
+
+| Worker | Engine address | Env read |
+| --- | --- | --- |
+| http / state / storage / database / harness | `--url` flag, default `ws://127.0.0.1:49134` | none |
+| llm-router | `--url` flag | `III_WS_URL` |
+| shell / codex / devin / email / bridge / lsp / hermes | varies | `III_URL` |
+| any SDK (`register_worker`) | explicit code argument | none |
+
+(`workers/http/src/main.rs:28`, `workers/llm-router/src/main.rs:25`,
+`sdk/packages/rust/iii/src/lib.rs:96`.)
+
+## The standard
+
+Every worker binary accepts the same three parameters, each with a flag and
+an env fallback, flags winning:
+
+| Parameter | Flag | Env |
+| --- | --- | --- |
+| engine address | `--url` | `III_URL` |
+| namespace | `--namespace` | `III_NAMESPACE` |
+| configuration | `--config` | `III_CONFIG` (resolved payload or handle — pinned with the config library) |
+
+- `registerWorker()` / `register_worker()` with **no arguments** reads the
+  env contract. Explicit arguments still win — but the zero-arg form is the
+  documented default, so tutorial code contains no addresses and no
+  namespaces, and the same code runs under compose, under `iii worker`, or by
+  hand.
+- The daemon injects all three into every child (hooks included). Reserved
+  keys: a compose file cannot override them.
+- Published binary packages get the flags for free once the shared **config
+  registration library** lands (one crate/package each SDK wraps — the
+  extraction is a listed next step from the 2026-07-13 review, so the fleet
+  is not 41 hand-rolled argument parsers).
+
+## Implicit start for packages
+
+With the contract in place, a `package://` binary needs no start script:
+compose execs the resolved artifact with the standard flags. That is the
+entire start story for the 34/41 fleet workers that today ship no `scripts:`
+at all — and why `run` in scripts.md exists only for local workers.
+
+## Migration
+
+Additive: binaries keep their current flags; the standard adds env fallbacks
+and normalizes names. The compose daemon only requires the env contract —
+old workers run under compose the day they read `III_URL`/`III_NAMESPACE`/
+`III_CONFIG`, which the shared library gives them in one dependency bump.
+`iii worker` and iii-worker-ops keep working unchanged during coexistence.

tech-specs/2026-07-14-worker-compose/compose-file.md

@@ -0,0 +1,79 @@
+# worker-compose.yaml v1
+
+One file describes one project's workers. One daemon binds to one file for its
+whole lifetime; the daemon's `--id` is how remote control addresses it
+(`iii trigger compose::up id=compose-hostb`).
+
+## Canonical shape
+
+```yaml
+name: orders
+
+containers:
+  database:
+    worker: package://workers.iii.dev/database
+    version: 1.4.2
+    config_name: orders-database
+    scripts:
+      post_run: "./backup-on-exit.sh"
+
+  api:
+    worker: path://./workers/api        # no iii.worker.yaml needed — run: is enough
+    depends_on: [database]
+    config_name: orders-api
+    config_override:
+      server:
+        port: 3000
+    scripts:
+      pre_start: "npx prisma migrate deploy"
+      pre_start_timeout: 120s
+      run: "pnpm dev"
+      post_run: "./cleanup-tmp.sh"
+```
+
+## Container fields
+
+| Field | Required | Rule |
+| --- | --- | --- |
+| `worker` | yes | `package://` (registry) or `path://` (local directory) |
+| `version` | package only | exact or range, resolved into the lockfile |
+| `depends_on` | no | container ids **from the same file only** |
+| `config_name` / `config_uri` | no | where the daemon fetches base config (see configuration.md) |
+| `config_override` | no | sparse map merged over the fetched base |
+| `scripts` | no (`run` required for manifest-less `path://`) | see scripts.md |
+| `working_dir` | no | default: worker dir for `path://`, compose-file dir for `package://` |
+
+## Identity
+
+- `containers` is an **id-keyed object**; the key is the worker's lifecycle id
+  and its registered name.
+- For `path://` with a manifest, the key must equal the manifest `name`. For
+  manifest-less `path://` (compose declares `run`), the key **is** the
+  identity.
+- No key order semantics: start order comes only from the `depends_on` DAG.
+
+## Dependency scope
+
+`depends_on` resolves inside the same file, full stop. Two compose files that
+want to share one database do it by **namespace** (both point at the same
+namespace, one of them owns the process — see namespace.md), never by a
+cross-file dependency edge. This keeps ownership, rollback, and teardown
+decidable by one daemon looking at one file.
+
+## Validation (hard errors)
+
+- empty `containers`; unknown or self `depends_on`; dependency cycle (error
+  prints the full path: `api -> queue -> database -> api`);
+- unknown field anywhere (strict schema);
+- `run` on a `package://` worker; `pre_start_timeout` without `pre_start`;
+- `path://` with neither manifest nor `run`;
+- key ≠ manifest `name` when a manifest exists.
+
+`iii compose validate` runs the whole ruleset offline — no engine required.
+
+## Not in v1
+
+Cross-file `depends_on`, docker runtime keys (`ports`, `image`), full inline
+configuration bodies, hot reload, `environment`/`env_file` (children get the
+standard injected contract plus the host env they inherit; per-container env
+maps return if a concrete need appears).

tech-specs/2026-07-14-worker-compose/configuration.md

@@ -0,0 +1,71 @@
+# configuration — fetched by the daemon, finalized before spawn
+
+The compose file never carries a full configuration and the worker never does
+a startup dance (boot → connect → query → maybe restart). The **daemon**
+resolves the final config and the worker receives it ready at start.
+
+## The flow
+
+```
+configuration worker (base, by name)
+        │  fetch
+        ▼
+compose daemon ── merge ──► final config ── standard contract ──► worker start
+        ▲
+        │  sparse override
+worker-compose.yaml (config_override)
+```
+
+1. The container names its base config: `config_name: orders-api` (or a
+   `config_uri` for explicit sources). Names resolve through the
+   **configuration worker**, whose adapter decides storage — local fs today,
+   secrets manager or a separate iii instance in cloud. File paths are not
+   the contract; names are, so the same compose file works when storage
+   moves.
+2. The daemon fetches the base **before** anything else runs for that
+   container. Fetch failure = the container does not start (`up` fails rather
+   than booting on wrong defaults — an http worker on the wrong port is worse
+   than no http worker).
+3. `config_override` merges over the base: maps merge per key, arrays and
+   scalars replace, `null` is an explicit value.
+4. The finalized config reaches the worker through the standard CLI/env
+   contract (`--config` / env; see cli-contract.md).
+
+## Why the daemon delivers values (not a pointer)
+
+Today `--config` is a **first-boot seed**: it populates the configuration
+entry only when nothing is stored, and afterwards "the configuration worker
+is the authoritative source and `--config` is ignored"
+(`workers/http/src/main.rs` doc comment). If compose passed only a pointer,
+every restart after first boot would silently ignore the compose overrides.
+Delivering resolved values keeps `worker-compose.yaml` honest: what the file
+says is what the process got — while the configuration worker remains the
+authority for everything the override does not touch.
+
+## Boundaries
+
+- **env vs config is a hard line**: env is machine/deployment context
+  (engine url, namespace); config is worker behavior (ports, adapters,
+  origins). The compose file does not blur them.
+- No full config bodies in the compose file — one source of truth, no
+  "edited the YAML and nothing happened" trap.
+- Runtime config reload is out of scope here: config is fixed for the life of
+  the process (the configuration worker's watch/reload path is a separate
+  track).
+
+## Example
+
+```yaml
+containers:
+  state:
+    worker: package://workers.iii.dev/state
+    version: 0.21.x
+    config_name: orders-state
+    config_override:
+      cors:
+        allowed_origins: ["https://app.example.com"]
+```
+
+The state worker's base `orders-state` config lives with the configuration
+worker (any adapter); the compose file pins one field; the process starts
+with the merged result and never re-fetches.

tech-specs/2026-07-14-worker-compose/lifecycle.md

@@ -0,0 +1,82 @@
+# lifecycle — up, readiness, cascade, teardown
+
+The daemon owns its children; the engine is the source of truth for
+readiness. Every lifecycle answer follows from those two facts.
+
+## up
+
+1. Validate file + dependency DAG (offline ruleset — cycles print their full
+   path).
+2. Resolve every container's config (fetch base + merge overrides). Any fetch
+   failure fails `up` before a single process exists.
+3. Walk the DAG in topological order. Per container:
+   `pre_start` (blocking, timed) → spawn `run` in its own process group /
+   Job Object → wait for **readiness**.
+4. On a startup failure: capture the failing worker's logs, roll back the
+   containers **this operation** started (reverse order), leave everything
+   that was already running untouched.
+5. `up` against an already-running file is a no-op success (`changed: false`
+   per container).
+
+**Readiness = the worker's registration is visible on the engine** — not a
+live PID. A process that boots but never registers is a startup failure at
+its `startup_timeout`, and the client timeout of the invoking `iii trigger`
+must sit above the graph's budget (a 30s client default under a 60-90s
+startup budget loses the result mid-operation).
+
+## Optimistic trigger buffer
+
+Startup order breaks registrations today: a worker that registers a route
+before its trigger-type worker (e.g. the http router) is up gets a warning
+and a dropped route. The engine gains a **buffer**: trigger registrations
+that reference an absent worker wait in memory and flush when that worker
+arrives (or re-arrives after a restart). Compose exposes the knobs — how long
+a registration may wait, how many retries — so the buffer cannot silently
+hold a route forever. Ordering becomes a performance concern, not a
+correctness one; `depends_on` stays about *data* dependencies, not
+registration order.
+
+## Cascading failure
+
+If a worker crashes post-ready, its local transitive dependents stop, in
+reverse dependency order, states set to `failed`, cause in the logs. The
+alternative — a queue worker happily pushing into a dead database — corrupts
+work invisibly. Softer edges (systemd-style `wants` vs `needs`) can come
+later for optional side-processes; v1 has one edge type and it is strict.
+
+## down
+
+- `down` (bare): stops everything this daemon owns, dependents before
+  dependencies. Never anything owned by another daemon.
+- `down <id>`: the target plus its local transitive dependents.
+- Termination per child: SIGTERM to the process group → bounded grace →
+  SIGKILL (Job Object termination on windows). Claims/registrations release
+  only after the process is confirmed dead. Once an exit is confirmed the
+  child's `post_run` fires (not awaited — teardown proceeds).
+- Intentional daemon shutdown (SIGINT/SIGTERM) performs a local `down`.
+  Engine disconnect does **not** stop children: they keep serving and the
+  daemon reconciles when the engine returns.
+
+## Crash and restart
+
+State on disk per child (pid + process birth identity + group id + log
+paths). On daemon restart: verify each recorded process against its birth
+identity — reconcile the live ones, clean the dead ones, never signal a pid
+it cannot verify (pid recycling is real). A worker that dies while the daemon
+is down is detected here and cascades then.
+
+## Remote surface
+
+`compose::up / down / list / status / logs / validate` — one function family,
+registered by the daemon like any worker's; the target daemon travels as an
+argument:
+
+```bash
+iii trigger compose::up id=host-a
+iii trigger compose::down id=host-a
+iii trigger compose::status id=host-a
+```
+
+`list`/`status` answer for **the addressed daemon's compose project only**;
+the engine-wide view stays on `worker::list` (which is also being unified:
+every connected entity reports as a worker with full metadata).