goaly

Run a coding agent (Claude Code, Codex, Droid, pi, or your own) in a loop until your goal is verifiably achieved — checked against a frozen success contract the agent can't weaken to pass.

The anti-reward-hacking core: "until the goal is achieved" must not collapse into "until the agent weakens its own test". goaly compiles the success contract once, freezes it, and requires two independent keys — a frozen verifier and an independent approver — before declaring a run DONE.

🌐 Interactive overview →

Quick start

npm i -g goaly                 # or, from a clone: make install

# Just give it a goal — the agent writes the check, runs, and verifies.
# A human approves the frozen contract once at Seal:
goaly "add a /health endpoint returning 200"

# Fully hands-off (-d auto-accepts the still-frozen, still-logged contract):
goaly -d "add a /health endpoint returning 200"

# Or point at a check you already have:
goaly run --goal "make the parser handle empty input" --verify-cmd "npm test"

Requires Node ≥ 20 and git. Exit codes: 0 DONE · 1 FAILED/ABORTED · 2 usage error · 130 interrupted (Ctrl-C — the run stays resumable).

How it works

COMPILE ──► SEAL ──► setup + pre-flight ──► ┌─────── loop (≤ --max-iterations) ───────┐
(author &   (freeze      (once)             │ RUN_AGENT ─► VERIFY ladder ─► SIGN-OFF  │
 freeze)     the bar)                       │     ▲             │fail          │veto  │
                                            │     └── feedback ─┴──── DECIDE ◄─┘      │
                                            └──────────────────────│─────────────────-┘
                                                     DONE · FAILED · ABORTED

• The contract is frozen at Seal. Its contractHash never changes again, and it's logged every iteration to prove the bar never moved. You approve it once — or revise it with feedback, or edit the authored files yourself and re-freeze. --autonomous skips the pause, never the freeze.
• The verifier ladder runs cheapest-and-hardest-to-game first: deterministic checks (exit codes, tests) before any LLM judge, short-circuiting on the first fail. A rung that errors is fail-closed — a malformed grader is never a green.
• Two keys for DONE: the frozen ladder passes and the independent Sign-off approver — which runs only on a green and is veto-only — doesn't veto. "Tests pass" is not "done".
• The control flow has zero LLM calls. A pure reducer owns all policy; everything stochastic hides behind narrow interfaces at four seams.
• Every run is crash-safe and resumable. A write-ahead log under .goaly/<runId>/ makes runs replayable, --resume-able, and inspectable (goaly runs list / show / watch, or goaly ui in the browser).
• Stuck detection bails early with a typed reason (no-diff, repeated failure, oscillation, harness crash, unevaluable contract, budget) instead of burning iterations.

Under --generate (the default), the compiler also authors a one-time setup command and a pre-flight proves the frozen verification can actually run — an unsound contract aborts before any worker token is spent.

Features

Everything below is documented in depth in the reference.

Feature	Flags	In short
Generated verification	--generate	The LLM authors the check + setup; pinned by hash, guarded against tampering.
Your own bar	--verify-cmd, --smoke	Any command as the deterministic rung; --smoke runs the built artifact.
Phased goals	--phased	A frozen plan of small sub-goals + cumulative acceptance on the original goal.
Best-of-N worker	--candidates N	N isolated attempts per iteration; the frozen ladder picks the winner. Or just say "use 4 subagents".
Parallel waves	--parallel-phases	Independent phases run concurrently, merge with git plumbing, re-verify. Experimental.
Worktrees	--worktree <name>	The whole run in an isolated checkout; merge back with plain git.
Adversarial review	--adversarial	Critics attack the contract before Seal; refuters attack every green before Sign-off.
Approver panels	--approver-quorum, --approver-models	Sign-off as a refute-first multi-vote panel, optionally across distinct models.
Sandboxing	--sandbox, --sandbox-net	OS-jail the agent and verifier (bwrap / firejail / container), with egress allowlists.
Operator control	--resume, --note	Watch live, steer with notes, raise caps mid-run — never the frozen bar.
Follow-ups	--from-run	A new re-verified goal that knows what the last run did.
Web UI	goaly ui	A local control center: mission dashboard, live pipeline + session inspector, worktrees, and a browser Seal review station. Localhost-only.
Spend & budgets	--budget-tokens, --cost-table	Per-layer token report (cache included); budgets survive resume.
Observability	--stream, --explain, --log-level	Live agent turns, durable transcripts, plain-language narration.
Reliability	(defaults)	Preflight, bounded retries, safe Ctrl-C, fsync'd write-ahead log.

Usage

# Choose a harness, cap iterations, set a budget; resume a crashed run by id:
goaly run --goal "..." --verify-cmd "pytest -q" --harness codex --max-iterations 8 \
          --budget-tokens 500000
goaly run --resume run-<id>

# Different models for the agent vs. the LLM steps (judge/approver/compiler):
goaly run --goal "..." --verify-cmd "npm test" --model claude-opus-4-8 --llm-model claude-sonnet-4-6

# No coding CLI at all — goaly's own agent loop on any OpenAI-compatible endpoint:
goaly run --goal "..." --verify-cmd "npm test" --harness goaly-code \
          --base-url http://localhost:11434/v1 --model qwen2.5-coder

# Inspect and follow runs (read-only), or open the browser UI:
goaly runs list
goaly runs show run-<id>
goaly ui

goaly help lists every flag. The CLI cookbook has a worked example for every mode; a config file (.goalyrc / ~/.goalyrc) keeps repeated wiring out of your invocations.

The LLM steps (compiler/judge/approver) follow the harness by default — --harness codex authors and judges on codex too, so one installed CLI is enough; --llm-provider splits them.

Add .goaly/ to your repo's .gitignore. Files authored under --generate are auto-registered in .git/info/exclude, so your git status stays clean.

Install

make install        # == npm install -g .  (bundles dist/ via the `prepare` hook)
goaly help

Or by hand: npm install && npm run build && npm install -g . — or make pack for a redistributable tarball. The default adapters shell out to the claude / codex / droid / pi CLIs; --harness goaly-code needs no CLI at all (any OpenAI-compatible endpoint, including a local keyless one like ollama).

Develop

make dev ARGS='run --goal "..." --verify-cmd "true" --harness fake --autonomous'  # run from source
make check          # typecheck + tests (the definition-of-done gate)

make help lists every task. The dev loop runs the TypeScript entry directly with tsx — no build step. See AGENTS.md for the eight invariants and conventions, and docs/adding-a-harness.md to wrap a new coding-agent CLI (one codec module + one registration line).

Embedding

The library works headless; the CLI is a thin caller:

import { drive, composeDeps, readStreamTranscript } from 'goaly';

const deps = composeDeps(config, { runId, streamTranscript: true });
await drive(deps, config, runId);
const stream = await readStreamTranscript('.goaly', runId);

DriverDeps hooks for embedders: interrupted (cooperative shutdown), sleep (retry backoff), onStreamEvent (live turn subscription), telemetry, and observer. There's also an experimental training pipeline built on the goaly-code harness.

Docs

• Reference — every flag, mode, and guarantee (start here for depth)
• DESIGN.md — what & why · ARCHITECTURE.md — how
• CONTEXT.md — the ubiquitous-language glossary (plain-language version)
• docs/adr/ — decision records · docs/adding-a-harness.md

License

MIT © krimvp