Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 88 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ stays resumable). See [Usage](#usage) for every flag.

- [How it works](#how-it-works) — the loop, the two gates, the verify ladder
- [Install](#install) · [Usage](#usage) — every flag and mode
- [Phased goals](#phased-goals---phased) · [Worktrees](#worktrees---worktree) · [Sandboxing](#sandboxing) · [Per-run spend report](#per-run-spend-report) — going further
- [Phased goals](#phased-goals---phased) · [Parallel waves](#cooperative-parallel-waves---parallel-phases-experimental) · [Worktrees](#worktrees---worktree) · [Sandboxing](#sandboxing) · [Per-run spend report](#per-run-spend-report) — going further
- [Reliability](#reliability-preflight-retries-interrupts-crash-safety) — preflight, retries, Ctrl-C, crash-safety
- [Operator control](#watching-steering--extending-a-run-operator-control) — watch live, steer with `--note`, extend caps on `--resume`
- [Web UI](#web-ui-goaly-ui) — `goaly ui`: runs, live feeds, and worktrees in the browser
Expand Down Expand Up @@ -190,7 +190,10 @@ PHASE 2 · the loop (🔁 ≤ --max-iterations, default 10; bails early on STUCK
0) and can't win. Each candidate completes write-ahead, so `--resume` re-runs only the not-yet-logged
ones and re-selects deterministically — or, with `--resume-best-of-incomplete collapse`, collapses to the
best already-logged candidate and re-runs nothing. Needs a committed HEAD (it refuses to start fail-closed
on an unborn branch). See [Best-of-N parallel worker](#best-of-n-parallel-worker---candidates).
on an unborn branch). You can also request it in **natural language** — *"fix the flaky test, work with 4
subagents"* in the goal, or *"try 4 parallel attempts"* in a `--resume` note — a deterministic directive
grammar (never an LLM parse) maps it onto `--candidates`, strips the clause from the frozen goal, and
logs the interpretation loudly. See [Best-of-N parallel worker](#best-of-n-parallel-worker---candidates).
- **Compile is resilient, not one-shot.** A `COMPILE_FAILED` (a correctable authoring mistake — bad
path, transient parse miss) re-authors the verification with the error fed back as guidance, up to
`--max-compile-retries` (default 2; `0` disables), before the run fails — so one bad compile output
Expand Down Expand Up @@ -304,6 +307,53 @@ ACCEPT (a cumulative contract on the ORIGINAL goal) ──both keys──► DO
every phase). `--resume` re-enters mid-plan without repeating completed phases. `goaly runs show`
prints the frozen plan and stamps each iteration with its phase.

### Cooperative parallel waves (`--parallel-phases`, EXPERIMENTAL)

Sequential phases leave wall-clock on the table when sub-goals are **independent**. `--parallel-phases`
(opt-in, experimental) runs them as **cooperating agents**: consecutive plan phases sharing a `group`
value form a **wave** that executes concurrently, then **merges** — without weakening a single
guarantee.

```jsonc
// plan.json — phases 1+2 are one wave; phase 3 runs after the merged result
{ "phases": [
{ "goal": "implement the parser", "group": 1 },
{ "goal": "implement the formatter", "group": 1 },
{ "goal": "wire parser + formatter into the CLI" }
] }
```
```bash
goaly "build the tool" --verify-cmd "npm test" --phased --parallel-phases --autonomous \
--plan-file plan.json
```

How a wave works, and why it can't cheat:

- **Fork: every wave member is a full goaly run.** Each phase gets an isolated git worktree off the
wave-start checkpoint and runs as its own **child run** — its own compiled + frozen contract, its own
iterations, verifier ladder, and veto-only Sign-off (both keys per child), its own write-ahead log
inside the worktree — all children metered by the **one shared `--budget-tokens`** cap.
- **Merge: plumbing, not prayer.** DONE children merge in phase order with a real 3-way
`git merge-tree` against the fork point — objects only, no commits, no HEAD/branch/index movement. A
**textual conflict applies nothing** of that child.
- **Re-verify: a merge is never trusted.** After promotion, each merged child's **frozen deterministic
rungs re-run on the combined tree** — two individually-green changes can still break each other. A
red re-verify un-lands nothing silently: that phase **downgrades to the classic sequential run** on
the merged tree, under a fresh frozen contract for the same sub-goal (the bar never moves, only the
starting tree). Merge conflicts and children that never reach DONE downgrade the same way.
- **Acceptance still gates the whole.** The final cumulative acceptance contract (both keys, LLM
included) runs on the original goal exactly as in a sequential phased run — so no decomposition,
parallel or not, can green a goal whose parts pass but whole doesn't.
- **Fail-closed to sequential.** Every failure shape — a conflicted merge, a red re-verify, a crashed
child, even a missing/broken wave executor — degrades to phases running one at a time, which is
byte-for-byte today's `--phased`. The reducer stays pure: it emits one `RUN_WAVE` command and folds
one `WAVE_RAN` event; children never enter the parent's state machine.
- **Experimental limits (v1):** requires `--autonomous` (children seal their frozen contracts
concurrently — still frozen + loudly logged) and a `--plan-file` with `group` fields (the LLM planner
does not author groups yet); a crash mid-wave re-runs the whole wave on `--resume` (children live in
ephemeral worktrees); wave-child spend is reported under the parent's `harness` layer. Grouped plans
run **strictly sequentially** without the flag, and the plan's grouping is frozen into `planHash`.

## Best-of-N parallel worker (`--candidates`)

Some iterations are a coin-flip — one attempt half-finishes, another nails it. `--candidates N` (alias
Expand Down Expand Up @@ -368,6 +418,33 @@ each iteration, with --candidates N:
HEAD** — `git worktree` can't check out an unborn tree, so a `--candidates > 1` run on a HEAD-less repo
**refuses to start** (fail-closed) with a clear message; make an initial commit or use `--candidates 1`.

### Natural-language delegation — just say it

You don't have to remember the flag: a **delegation directive in the goal** maps onto the same
tournament, and mid-run the same grammar reads your **resume note**.

```bash
goaly "fix the flaky auth test, work with 4 subagents" # ⇒ --candidates 4
goaly "make the linter pass using 3 parallel attempts" # ⇒ --candidates 3
goaly "port the parser to TS, use subagents" # ⇒ --candidates 3 (documented default)
goaly --resume run-… --note "focus on the parser, try 4 parallel attempts" # raises the fan-out mid-run
```

- **Deterministic, never an LLM parse.** Detection is a small directive grammar (`src/cli/delegation.ts`)
— an LLM interpreting your config would be exactly the "LLM in control flow" goaly exists to avoid. The
grammar is deliberately narrow: only **`subagents`** (with a delegation verb — `use / spawn / work
with / delegate to …`) and **`N parallel attempts|candidates|tries`** trigger it, so goals about
app-domain parallelism (*"implement a job queue with 4 parallel workers"*, *"handle 5 parallel login
attempts"*) never match. No match ⇒ the classic single attempt — fail-closed, never a guess.
- **The directive is stripped from the goal** before the contract is compiled — the goal is frozen and
read by the judge/approver, and a leftover *"use 4 subagents"* would become an unverifiable success
criterion. The interpretation is **loudly logged** (`phrase → N`), and the explicit `--candidates` /
`--best-of` flag (or config file) **always wins**.
- **Mid-run steering rides ADR 0012.** A directive in `--resume … --note "…"` becomes a `candidates`
overlay on the `RUN_EXTENDED` marker (an operational knob, like a raised `--max-iterations` — the
frozen contract stays unreachable); the directive clause leaves the note and any remaining text still
steers the worker. Same 16 cap, same fail-closed error above it.

## Worktrees (`--worktree`)

Sometimes the run shouldn't touch your working tree at all — you keep editing while goaly builds, or
Expand Down Expand Up @@ -1179,14 +1256,18 @@ goaly --resume run-<id> --max-iterations 25 # revive FAILED at the i
goaly --resume run-<id> --budget-tokens 900000 # revive a budget abort (prior spend still counts)
goaly --resume run-<id> --stuck-no-diff false --note "try editing src/parser.ts directly"
# revive a stuck abort, with direction
goaly --resume run-<id> --candidates 4 # widen the best-of-N fan-out for what's left
goaly --resume run-<id> --note "try 4 parallel attempts" # same, said in natural language
```

Only the operational knobs are extendable (`--max-iterations`, `--budget-tokens`,
`--budget-wall-ms`, the `--stuck-*` thresholds) — the goal, verifier, and rubric are structurally
not part of an extension, so autonomy never becomes "renegotiate the bar." A DONE run refuses to
extend and points you at `--from-run`. Rule of thumb: **same goal, more room → `--resume` with
caps/note; new or refined goal → `--from-run`** (a fresh contract, compiled aware of what just
happened — see [Following up](#following-up-after-a-run-ends---from-run)).
`--budget-wall-ms`, the `--stuck-*` thresholds, `--candidates`) — the goal, verifier, and rubric are
structurally not part of an extension, so autonomy never becomes "renegotiate the bar." A DONE run
refuses to extend and points you at `--from-run`. A resume also **continues the run's own harness**
(recorded in the log) rather than silently switching to the default — session ids are
harness-specific; pass `--harness` explicitly to override. Rule of thumb: **same goal, more room →
`--resume` with caps/note; new or refined goal → `--from-run`** (a fresh contract, compiled aware of
what just happened — see [Following up](#following-up-after-a-run-ends---from-run)).

### Inspecting past runs

Expand Down
70 changes: 70 additions & 0 deletions docs/adr/0017-cooperative-parallel-waves.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# ADR 0017 — cooperative parallel waves (`--parallel-phases`, experimental)

## Status
Accepted (experimental, opt-in).

## Context

goaly had exactly one parallelism: best-of-N (`--candidates`, issue #85) — K **competing** attempts
at the SAME sub-goal, keep one, discard the rest. What it lacked was **cooperation**: running
*different, independent* sub-goals concurrently and **combining** their work. Phased decomposition
(issue #48) already produces exactly the right unit — a frozen plan of sub-goals, each executed as
its own frozen, two-key contract, finished by a cumulative acceptance contract on the original goal
— but executes strictly sequentially, leaving wall-clock on the table whenever phases touch
disjoint parts of the tree. The plan schema explicitly deferred this ("no DAG/parallelism in v1").

The tension: combining trees is where fail-closed guarantees usually die. A merge can conflict
(textually) or lie (two clean merges that break each other semantically), and any "merge agent"
that resolves conflicts with an LLM would put unverified writes on the DONE path.

## Decision

**Waves of child runs + merge-and-reverify, fail-closed to sequential.**

1. **The plan carries the grouping, frozen.** `SubGoal` gains an optional `group`; CONSECUTIVE
phases sharing a group form a wave. The grouping is canonicalized into `planHash` (groupless
plans keep their legacy hash byte-for-byte), so no transition can re-shuffle it. v1 sources
groups from `--plan-file` only.
2. **The reducer sees one command, one event.** When the current phase heads a not-yet-attempted
group and `config.parallelPhases` is on, `startPhaseCompile` emits ONE `RUN_WAVE` (per-phase
configs derived exactly as for sequential phases) and folds ONE `WAVE_RAN` — still exactly one
command per state, still pure. `PhaseCtx` gains `skip` (indices completed by a wave) and `waved`
(indices already attempted — a group can never re-fan-out), both optional so classic runs are
untouched.
3. **Every wave member is a full goaly run.** The Driver-side `WaveRunner` gives each phase an
ephemeral worktree off the wave-start checkpoint and an embedded `drive()` — its own compiled +
frozen contract, iterations, ladder, veto-only Sign-off, and write-ahead log inside the
worktree — all children on the PARENT's budget meter and interrupt probe. Worktree creation is
sequential (git lock contention); only the runs are concurrent.
4. **Merge is plumbing; the merged tree is re-verified.** DONE children merge in phase order via
`git merge-tree --write-tree --merge-base=<fork point>` (objects only; a conflicted merge
applies nothing). After promotion, each merged child's frozen DETERMINISTIC rungs re-run on the
combined tree. Judge rungs are not re-run here: each child already turned both keys in
isolation, and the final acceptance contract still gates the whole run — the merged-tree guard
is the ungameable deterministic bar in between.
5. **Every failure downgrades, nothing greens.** A merge conflict, a red re-verify, a child that
never reaches DONE, a thrown wave runner, or a missing wave seam all resolve to `unmerged`,
which the reducer turns into the CLASSIC sequential phase on the merged-so-far tree — a fresh
frozen contract for the same sub-goal. The worst case of the feature is exactly today's
`--phased`.

## Consequences

- **Invariants hold.** #1: the fan-out is Driver-side data flow (`RUN_WAVE`/`WAVE_RAN`); children
are separate pure folds over separate logs. #2: grouping frozen in `planHash`; child contracts
frozen at their own Seals. #3: two keys per child AND at acceptance. #4: a merge is never
trusted; every failure shape is a typed downgrade. #7: `WAVE_RAN` carries the post-merge
checkpoint tree (replay re-points the baseline like `PHASE_ADVANCED`).
- **Cost profile.** ~1× the sequential token cost (+ deterministic re-verification + any conflict
re-runs) for wall-clock ≈ slowest child + merge — the complement of best-of-N, which buys
quality at ~N× cost for one phase's work.
- **Experimental limits (v1), by design:** requires `--autonomous` (children seal concurrently;
an interactive gate cannot pause K children at once); a crash mid-wave re-runs the whole wave on
`--resume` (children live in ephemeral worktrees; their logs die with them); wave-child spend is
bucketed under the parent's `harness` usage layer (totals and the budget cap stay exact);
compiler-authored (git-excluded) verification files are copied from each merged child's worktree
so their frozen commands keep their inputs — colliding authored paths across children surface as
a red re-verify, never a silent overwrite that greens.
- **Deferred:** planner-authored groups (and the natural-language "split this across N subagents"
directive mapping onto them), durable child logs for fine-grained wave resume, LLM-judge re-runs
on the merged tree.
30 changes: 28 additions & 2 deletions docs/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,30 @@ <h3>Best-of-N parallel worker</h3>
above 16 is a fail-closed error). Composes with
<code>--phased</code> / <code>--delta-verify</code> / <code>--sandbox</code>; needs a committed
HEAD.</p>
<p><b>Or just say it:</b> a natural-language directive in the goal — <i>"fix the flaky test,
work with 4 subagents"</i>, <i>"… using 3 parallel attempts"</i> — maps onto
<code>--candidates</code> via a small <b>deterministic grammar</b> (never an LLM parse), and the
clause is <b>stripped from the goal</b> so it can't enter the frozen contract; the
interpretation is loudly logged and the explicit flag always wins. Mid-run, the same grammar
reads a resume note: <code>--resume &lt;id&gt; --note "try 4 parallel attempts"</code> raises
the fan-out as a <code>RUN_EXTENDED</code> candidates overlay (operator control, ADR 0012).
Domain goals like <i>"a queue with 4 parallel workers"</i> never match — no directive means the
classic single attempt.</p>
</div>
<div class="card">
<h3>Cooperative parallel waves <span style="opacity:.6">(experimental)</span></h3>
<p><code>--parallel-phases</code> (opt-in, with <code>--phased --autonomous</code>) turns
consecutive plan phases sharing a <code>"group"</code> into a concurrent <b>wave of
cooperating child runs</b> — each phase a FULL goaly run (own frozen contract, iterations,
two-key gate, write-ahead log) in an isolated worktree, on the one shared
<code>--budget-tokens</code>. DONE children then <b>merge in phase order</b> via real 3-way
<code>git merge-tree</code> plumbing (no commits), and each merged phase's frozen
<b>deterministic rungs re-run on the combined tree</b> — a merge is never trusted. A
conflict, a red re-verify, a failed child, or even a missing wave executor all
<b>downgrade fail-closed to the classic sequential phase</b>; the cumulative ACCEPTANCE
contract still gates the whole run, so parallel decomposition can't green a broken whole.
The reducer stays pure: one <code>RUN_WAVE</code> command out, one <code>WAVE_RAN</code>
event back — children never enter the parent's state machine.</p>
</div>
<div class="card">
<h3>Named worktrees</h3>
Expand Down Expand Up @@ -234,8 +258,10 @@ <h3>Watch it &amp; steer it</h3>
never disturbs the run). Ctrl-C stops cleanly between steps, then
<code>--resume &lt;id&gt; --note "…"</code> hands the worker your guidance on its next turn.
A run that hit an <em>operational</em> limit isn't a dead end — <code>--resume</code> with
<code>--max-iterations</code> / <code>--budget-tokens</code> / <code>--stuck-*</code>
continues it in place, auditable in the log. The extension schema simply <b>has no field for
<code>--max-iterations</code> / <code>--budget-tokens</code> / <code>--stuck-*</code> /
<code>--candidates</code> continues it in place, auditable in the log — and it continues the
run's <b>own recorded harness</b> (session ids are harness-specific; <code>--harness</code>
overrides explicitly). The extension schema simply <b>has no field for
the goal or verifier</b>: you can add room, never lower the frozen bar.
<a href="https://github.com/krimvp/goaly/blob/main/docs/adr/0012-operator-control.md" target="_blank" rel="noopener" style="color:var(--accent)">ADR 0012 →</a></p>
</div>
Expand Down
Loading
Loading