Skip to content

fix(droid): harden the codec against live droid 0.164.0 behavior#97

Merged
krimvp merged 1 commit into
mainfrom
claude/droid-harness-hardening
Jul 6, 2026
Merged

fix(droid): harden the codec against live droid 0.164.0 behavior#97
krimvp merged 1 commit into
mainfrom
claude/droid-harness-hardening

Conversation

@krimvp

@krimvp krimvp commented Jul 6, 2026

Copy link
Copy Markdown
Owner

Summary

The droid codec was verified against droid 0.153.1; the CLI self-updates and the engine now running is 0.164.0, where three of the codec's assumptions no longer hold. Every finding below was probed live against the installed binary (throwaway git sandbox, trivial prompts).

Discovery worksheet (live, droid 0.164.0)

Harness name (kebab):      droid
Binary + version:          droid 0.164.0 engine (launcher reports 0.153.1; self-updating)
Fresh invocation:          droid exec --output-format json --auto low "<prompt>"
Resume invocation:         droid exec --output-format json --auto low --fork <id> "<prompt>"
Prompt delivery:           argv
Output shape:              json-object (plain) | jsonl-stream (stream-json)
Field mapping:
  result text    <-  .result (json) | completion .finalText | last assistant message .text (stream)
  session id     <-  .session_id (latch FIRST seen; --fork mints a NEW id each continuation)
  token usage    <-  .usage (input/output + both cache buckets)
Streaming mode:            --output-format stream-json — droid-NATIVE lines, NOT the SDK envelope:
  session        <-  type=system subtype=init .session_id
  message        <-  type=message role=assistant .text   (role=user echoes the PROMPT — suppressed)
  reasoning      <-  type=reasoning .text (droid may emit the same line twice)
  tool_use       <-  type=tool_call .id/.toolName/.parameters
  tool_result    <-  type=tool_result .id/.value/.isError
  usage + done   <-  type=completion .usage/.finalText (plain-json type=result maps the same)
  reuse?         <-  custom extractor (sdkStreamExtractor matched only the system line)
Status mapping:            classifyFlatRun unchanged (bad flag → exit 1 + short msg → crashed;
                           bogus session → exit 1 + EMPTY output → crashed; is_error → truncated)
Notes / caveats:           exec -s <id> FAILS on 0.164.0 for locally-created exec sessions —
                           "MetaError: Failed to fetch session" (backend fetch), exit 1 with empty
                           stdout/stderr (error only in ~/.factory/logs). --fork <id> loads the
                           LOCAL transcript and continues under a fresh id (verified: recall works).

What changed

  • Custom stream extractor for droid's native stream-json (was: shared sdkStreamExtractor, which produced only a lone session event → dead live view and no issue-enhance: complete & robust token accounting (meter LLM steps; estimate from streamed turns when self-report is missing) #24 token estimation).
  • Line-type-aware field extractor: the stream reuses text on the user prompt echo and reasoning lines, so the old flat mapping could return the caller's own prompt as a "completed" result on a truncated stream. Now only assistant messages and the closing completion line (finalText) bear result text; a stream cut after the echo correctly classifies truncated.
  • Resume via --fork <id> instead of --session-id <id>: -s crash-loops every continuation turn on 0.164.0 (the orchestrator re-threads the failed id, step.ts:467). Fork-chaining keeps real conversation memory; goaly never assumed session-id stability.
  • interactiveResumedroid --resume <id> (documented top-level flag; --session-id exists only on exec). Note: droid's top level silently ignores unknown flags, so the old hint would boot a fresh interactive session without complaint.
  • Docs synced: docs/adding-a-harness.md (droid no longer an example of the flat/SDK reuse), README.md resume-cmd example, output.ts/stream.ts doc comments.

Live verification

  • Two-turn AgentCliHarness probe against the real binary: turn 1 streamed (session/message/usage/done through the new extractor, tokenSource: 'reported'), turn 2 fork-resumed turn 1's session and recalled the codeword.
  • Full goaly E2E in a sandbox repo, droid as both roles: goalyd run --goal ... --verify-cmd ... --harness droid --llm-provider droid --autonomousDONE in 1 iteration, deterministic rung green + droid Sign-off approver, spend report with per-category breakdown, banner prints droid --resume <id>.

Tests

  • npm run typecheck: clean for all touched files. Pre-existing failures in src/telemetry/telemetry.test.ts (5 tests + typecheck errors) exist on origin/main and are untouched here — the test file expects a different telemetry API than the committed module.
  • npx vitest run on the 7 affected files: 122/122 green; full suite: 1709 passed, 5 failed (the pre-existing telemetry ones only).
  • New regression coverage: real captured 0.164.0 stream transcript → parse + event mapping, prompt-echo/reasoning never salvaged as result text, --fork argv (fresh/resume/model/streaming), native-envelope streaming through AgentCliHarness.

🤖 Generated with Claude Code

https://claude.ai/code/session_011H2PtexDeeD3JkGTVQPycg

Live-probed Factory droid 0.164.0 (the CLI self-updates; 0.153.1 was the
last verified engine) and fixed three seams that had drifted:

- stream-json is droid-NATIVE, not the Anthropic agent-SDK envelope: map
  system/message/reasoning/tool_call/tool_result/completion lines with a
  custom StreamEventExtractor (sdkStreamExtractor yielded only `session`,
  so live streaming/token estimation saw nothing).
- the field extractor is now line-type aware: it no longer salvages the
  echoed user prompt or reasoning text as result text on a truncated
  stream, and it reads the closing completion line's `finalText`.
- session continuity via `--fork <id>`: on 0.164.0, `exec -s <id>`
  fetches the session from Factory's backend and fails for locally
  created exec sessions (silent exit 1; "Failed to fetch session" lands
  only in ~/.factory/logs) — every continuation turn would crash-loop.
  `--fork` loads the local transcript and continues under a fresh id
  (verified live: cross-turn recall works); goaly already threads
  whatever id each run reports, so continuity is a fork chain.
- interactive resume hint: `droid --resume <id>` (the documented
  top-level flag; `--session-id` exists only on `exec`).

Verified live end-to-end: a two-turn AgentCliHarness probe (streaming
events + fork memory recall) and a full goaly run with --harness droid
--llm-provider droid to DONE (both keys turned; usage reported).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011H2PtexDeeD3JkGTVQPycg
@krimvp
krimvp merged commit 03aedc7 into main Jul 6, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant