Skip to content

feat: add claude-code and codex-cli headless CLI providers#293

Open
Sangyeon Cho (csy1204) wants to merge 18 commits into
langchain-ai:mainfrom
csy1204:feat/cli-headless-providers
Open

feat: add claude-code and codex-cli headless CLI providers#293
Sangyeon Cho (csy1204) wants to merge 18 commits into
langchain-ai:mainfrom
csy1204:feat/cli-headless-providers

Conversation

@csy1204

@csy1204 Sangyeon Cho (csy1204) commented Jul 12, 2026

Copy link
Copy Markdown

What

Adds two providers — claude-code and codex-cli — that delegate the whole agent loop to locally installed headless CLIs (claude -p --output-format stream-json / codex exec --json) instead of API-backed deepagents:

  • src/constants.ts: the two providers with authMethod: "cli" (no API key; the CLI's own login is the auth), model options (claude: sonnet/opus/haiku aliases; codex: the OpenAI model list), and a run-timeout env (OPENWIKI_CLI_TIMEOUT_SECONDS, default 1800s).
  • src/agent/cli-runner/: engine adapters (argv builders + stream-event parsers mapped onto the existing OpenWikiRunEvent union, so the TUI is unchanged), a spawn/stream runner (stderr tail on failure, SIGTERM→SIGKILL timeout, auth-failure login hints, EPIPE-safe stdin), CLI-specific prompt builders (real relative paths instead of the virtual-root wording; no LangChain tool names), and a threadId ↔ CLI session store (~/.openwiki/cli-sessions.json) so chat follow-ups resume the CLI session (with fresh-session fallback when resume fails).
  • runOpenWikiAgent routes CLI providers to the runner before any API-key checks; the update-noop gate, content snapshot, and .last-update.json metadata logic are shared with the API path (metadata records claude-code/sonnet-style model ids).
  • Onboarding wizard lists both providers and skips the API-key step for them; the non-interactive startup gate exempts CLI-auth providers from the key requirement (empty-message validation still applies).
  • Safety: claude runs with --permission-mode acceptEdits plus an allowlist (read tools, read-only git, exact plan-file cleanup commands — no network tools, no broad Bash); codex runs with --sandbox workspace-write. Since headless permission models cannot express "read repo-wide, write wiki-only", repository-mode runs also do a warn-only post-run git status check (baselined against the pre-run dirty state) that reports any out-of-wiki changes.

Why

Bootstrapping repository docs (openwiki code --init/--update) is tool-call heavy, which makes the API path slow and expensive. Claude Code and Codex subscriptions already include agentic execution with native file/search/shell tools, so delegating the loop to them makes the initial analysis faster and removes per-token API cost. Existing API providers are untouched; the CLI providers are opt-in via OPENWIKI_PROVIDER only (never auto-selected by the key-based fallback chain).

How I tested it

  • Unit/integration (Vitest): 251 passing (234 pre-existing — all unchanged — plus 17 new across adapters, prompt builders, session store, runner, routing, startup gate, write guard). Parser fixtures are captured from real CLI runs (claude 2.1.207, codex 0.144.1). The runner is tested against stub CLI executables (real spawn/stdin/stream/exit paths, including a regression test that reproduced an uncaught write EPIPE crash pre-fix).
  • End-to-end against the real CLIs on a fixture repo: wiki generation with both engines (correct .last-update.json model ids), noop-skip on unchanged re-run, chat follow-up resuming the same CLI session (verified conversational carry, no fallback), and the onboarding wizard flow (provider list → model selection, no key prompt).
  • pnpm run format, pnpm run lint, pnpm test all clean.

Scope note

This is one feature (CLI-auth providers) and every hunk serves it, but it is a large surface: provider config, a new cli-runner module, prompt-builder extraction (createSystemPrompt output is byte-identical — verified before/after across all command×outputMode combinations), wizard, and the startup gate. If you'd prefer it split (e.g. provider scaffolding → runner → wizard/gate), happy to do that — flagging per the contributing guide rather than deciding unilaterally.

🤖 Generated with Claude Code

Sangyeon Cho (csy1204) and others added 17 commits July 13, 2026 17:51
@csy1204
feat: add claude-code and codex-cli provider definitions
a0c8658
@csy1204 @claude
feat: add cli-runner types and session store
fe999fd 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@csy1204 @claude
feat: add claude headless engine adapter
1c5fe3f 
Add claudeAdapter (CliEngineAdapter for engine "claude-code", cliCommand
"claude") that builds the headless stream-json argv and parses the CLI's
NDJSON stdout lines into CliParsedEvent values.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@csy1204
feat: add codex headless engine adapter
c331975
@csy1204 @claude
refactor: extract shared prompt sections and add cli prompt builder
aa68eaf 
Export OutputPromptConfig/getOutputPromptConfig and add an optional
output config parameter to createModeInstructions/createUserPrompt.
Extract createSecuritySection, createDocumentationGoalSections,
createStewardshipSections, and the OPENWIKI_CLI_REFERENCE constant out
of createSystemPrompt as pure text moves (output byte-identical). Add
src/agent/cli-runner/prompt.ts with real-path CLI prompt builders
(getCliOutputPromptConfig, createCliSystemPrompt, createCliUserPrompt)
plus tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@csy1204 @claude
fix: override remaining virtual-path fields in cli prompt config
d19f961 
The CLI output config still inherited virtual-root wording from the base
config: repository-mode rootAgentInstructions and writeBoundaryInstruction
spliced "/openwiki", "/AGENTS.md", and "/CLAUDE.md" style paths into CLI
prompts, and local-wiki writeBoundaryInstruction and
localWikiSynthesisInstruction kept leading-slash canonical wiki refs and
langchain-tool wording. Override all four with real relative-path
equivalents (the synthesis block via a verbatim-preserving slash rewrite)
and add regression tests asserting no leading-slash wiki paths remain in
either mode's system and user prompts.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@csy1204 @claude
feat: add cli runner core with streaming, timeout and session resume
cdae2b8 
Add stdinPayload to CliEngineAdapter (claude: user prompt only; codex:
system+user combined) and implement src/agent/cli-runner/index.ts:
executeCliRun spawns the headless CLI, streams stdout via readline,
buffers the last 2048 bytes of stderr, enforces a SIGTERM->SIGKILL
timeout, backfills tool_end names from tool_start, and retries once
with a fresh session when a resumed run fails. runOpenWikiCliAgent
builds prompts, resolves/saves sessions, snapshots wiki content and
writes last-update metadata (model provider/modelId). ensureCliBinaryAvailable
probes the CLI --version and surfaces an install hint on ENOENT.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@csy1204 @claude
fix: swallow cli stdin EPIPE so early exits reject instead of crashing
f2567ab 
child.on("error") only covers the ChildProcess emitter, not the stdin
stream. When the CLI exits before consuming stdin (e.g. an instant auth
failure) while a multi-KB prompt is still flushing, the write emits
EPIPE on a listener-less stream and crashes the whole process via
uncaughtException. Register a no-op stdin error handler and let the
close handler settle the run with the real exit-code/stderr failure.
Regression test drives a 1MB stdin payload into a stub that exits
immediately without reading.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@csy1204 @claude
feat: route cli providers to the headless cli runner
4eb45a9 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@csy1204
feat: support cli auth providers in the onboarding wizard
d080dbb
@csy1204 @claude
fix: exempt cli-auth providers from non-interactive api-key gate
3f5e7a2 
The non-interactive startup gate required the configured provider's API key
env var to be set for --print / non-TTY runs. CLI-auth providers
(claude-code, codex-cli) have no API key (auth is their own CLI login,
validated at run time by ensureCliBinaryAvailable), so their placeholder
key env var is never set and every non-interactive run — including the CI
`code --update --print` path — failed fast with
"OPENWIKI_CLI_AUTH_UNUSED is required for non-interactive runs".

Skip the API-key gate when providerUsesCliAuth(provider).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@csy1204 @claude
fix: scope cli-auth exemption to the api-key gate only
803d022 
The cli-auth exemption in resolveStartupCommand was an early
`return command`, which exited the whole gate function — so CLI-auth
providers in the non-interactive path also skipped the shared
empty-message validation below. A whitespace-only message
(`--update -m "   " --print`) sailed through for claude-code/codex-cli
while being rejected for api-key providers, and made cli-auth behavior
inconsistent between TTY and non-TTY modes.

Wrap only the api-key gate block in `if (!providerUsesCliAuth(provider))`
so CLI-auth providers still flow through the empty-message validation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@csy1204 @claude
feat: warn on out-of-wiki writes after repository cli runs
86372a7 
The API backend structurally blocks non-doc writes, but the CLI path only
relies on the prompt plus allowedTools/sandbox, and the post-run wiki snapshot
cannot detect source-file mutations. After a non-chat repository run, inspect
git status --porcelain (via execFile, never a shell) and emit a WARNING text
event naming any changed paths outside openwiki/. The run is never failed;
non-git working directories are skipped with a debug event.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@csy1204 @claude
fix: render the local-wiki synthesis block in the cli system prompt
756c718 
getCliOutputPromptConfig already computed a cwd-rewritten synthesis block, but
createCliSystemPrompt never rendered it, so local-wiki CLI runs lost the
knowledge-synthesis discipline that the base createSystemPrompt includes. Splice
it in after the documentation-goal section for parity. Repository mode leaves
the value empty, so the trailing separator collapses and no stray blank section
is emitted.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@csy1204 @claude
feat: add cli auth login hints and clear the sigkill timer on settle
d3e6b29 
When a CLI run exits nonzero with auth-flavored stderr (login/unauthorized/
credential/api key), append an engine-specific sign-in hint to the thrown error
("claude /login" / "codex login") so users know how to recover. Also store the
inner SIGKILL setTimeout handle and clear it in settle so a normal exit after
SIGTERM does not leave a dangling timer.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@csy1204 @claude
fix: harden session file mode and fix cli-provider wizard article
f983234 
Pass { mode: 0o600 } to writeFile when creating the CLI session cache (keeping
the follow-up chmod for the pre-existing-file case) so a freshly created cache
is never briefly world-readable. Also make getProviderArticle return "a" for
claude-code and codex-cli so the wizard reads "a Claude Code (CLI) model" and
"a Codex (CLI) model".

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@csy1204 @claude
fix: baseline the out-of-wiki write guard against pre-run dirty trees
c42d07a 
Update runs deliberately proceed on dirty trees, so a post-run-only porcelain
scan attributed pre-existing changes to the CLI agent and fired the warning on
every dirty-tree run. Capture a git status --porcelain baseline before the CLI
run (repository non-chat runs only, same skip-on-git-error semantics) and warn
only on the delta: out-of-wiki paths present after the run but not in the
baseline. findUnexpectedChanges is now (baselinePorcelain, porcelainOutput),
still pure and warn-only.

Also gate the porcelain rename split on rename/copy status letters and correct
the C-quoting comment: git only quotes control chars, quotes, backslashes, and
non-ASCII, so a literal " -> " filename stays unquoted; the status gate keeps
such paths intact on ordinary lines.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@csy1204
Sangyeon Cho (csy1204) force-pushed the feat/cli-headless-providers branch from 2888efb to c42d07a Compare July 13, 2026 08:53
audichuang added a commit to audichuang/openwiki that referenced this pull request Jul 15, 2026
@audichuang @claude
fix: persist metadata on agent-CLI failure; stop bogus default-model …
303af31 
…notice

- runAgentCliRun now wraps the vendor-CLI run in try/catch and persists
  run metadata on late failure (mirrors runOpenWikiAgentCore), so docs a
  failed run already wrote stay diffable by the next update. Neither the
  grok/claude PRs nor upstream langchain-ai#293 handled this.
- formatProviderSwitchNotice no longer claims a default model for providers
  with no preset models (bedrock, openai-compatible) — it was surfacing the
  unrelated OpenAI fallback id (gpt-5.6-terra); now prompts "Set a model
  with /model" (the fix idea from upstream langchain-ai#167, ported without its
  signature change).

Tests: failing-after-write stub asserts metadata persists on throw;
provider-switch notice covers empty-modelOptions providers.
Verified: typecheck, lint, format, pnpm test (327), build.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
audichuang added a commit to audichuang/openwiki that referenced this pull request Jul 16, 2026
@audichuang @claude
fix: per-provider agent-CLI login copy (stop telling claude-code user…
396faf5 
…s 'grok login')

Onboarding hardcoded 'Grok Build CLI login (grok login)' for ALL agent-cli
providers (credentials.tsx getCredentialSetupDetail, cli.tsx api-key error),
so claude-code users were told to run grok login. Add a loginCommand field to
AgentCliProviderConfig (claude -> 'claude', grok -> 'grok login') + a
getProviderLoginCommand helper, and route both strings through it.

Idea from upstream langchain-ai#293 (authLoginHint); implemented for our discriminated-union
config. TDD: login-command helper + claude-code-not-grok regression test.
Verified: typecheck, lint, format, pnpm test (359), build.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
audichuang added a commit to audichuang/openwiki that referenced this pull request Jul 16, 2026
@audichuang @claude
feat: persist agent-CLI sessions + fall back on stale resume (from la…
a150420 
…ngchain-ai#293)

- New session-store.ts persists thread -> {provider, sessionId} to
  ~/.openwiki/cli-sessions.json (0o600, corrupt-tolerant, provider-tagged),
  replacing the in-memory Map so chat follow-ups resume the vendor CLI
  session across separate openwiki processes. Path overridable via
  OPENWIKI_CLI_SESSIONS_PATH (keeps tests out of the real home).
- runAgentCliRun retries once with a fresh session when a --resume run fails
  (stale/expired id), instead of wedging the follow-up. Failure-metadata
  persistence refactored into a shared helper across both catch paths.

Grafted from upstream langchain-ai#293 (sessions.ts + resume fallback), adapted to our
engines/ stack and discriminated-union providers. TDD: session-store unit
tests (round-trip, provider mismatch, disk persistence, clear, corrupt-file)
+ a resume-fallback integration test.
Verified: typecheck, lint, format, pnpm test (366), build.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
audichuang added a commit to audichuang/openwiki that referenced this pull request Jul 16, 2026
@audichuang @claude
feat: warn on repository-mode agent-CLI writes outside openwiki/ (from …
5feed63 
…langchain-ai#293)

The claude-code allowlist grants unscoped Write/Edit/MultiEdit and the runner
now passes --add-dir, so the vendor CLI's cwd is no longer a hard write
boundary. Add a warn-only guard: baseline git status --porcelain before a
repository-mode run, diff after, and emit a warning listing changed paths
outside openwiki/ (never fails the run). Handles renames and C-quoted paths;
pre-existing dirty files are excluded. Skipped in local-wiki mode (writes land
at the wiki root, not under openwiki/).

Grafted from upstream langchain-ai#293 (findUnexpectedChanges), adapted to our engines/
stack. Also refreshes the now-stale claude-code allowlist comment that claimed
--add-dir is never passed. TDD: write-guard unit tests + a stray-write
integration test.
Verified: typecheck, lint, format, pnpm test, build.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
audichuang added a commit to audichuang/openwiki that referenced this pull request Jul 16, 2026
@audichuang @claude
docs: trim AGENTS.md drift after FORK-NOTES + new engine modules
ccf67b3 
Audit follow-up:
- Where-to-start 'agent-cli adapters' row enumerated 5 engine files and had
  gone stale (missing session-store.ts, write-guard.ts). Replace the
  enumeration with self-maintaining src/agent/engines/* + role hints.
- Branch-status duplicated FORK-NOTES.md (PR table, langchain-ai#293 rationale, run
  snippet, release caveat). Keep only the always-needed edit-time bits (kind
  invariant) and action-time bits (remote/push target) plus the FORK-NOTES
  pointer; the rest lives in FORK-NOTES (point, don't summarize).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
audichuang added a commit to audichuang/openwiki that referenced this pull request Jul 16, 2026
@audichuang @claude
fix: persist metadata on agent-CLI failure; stop bogus default-model …
683a231 
…notice

- runAgentCliRun now wraps the vendor-CLI run in try/catch and persists
  run metadata on late failure (mirrors runOpenWikiAgentCore), so docs a
  failed run already wrote stay diffable by the next update. Neither the
  grok/claude PRs nor upstream langchain-ai#293 handled this.
- formatProviderSwitchNotice no longer claims a default model for providers
  with no preset models (bedrock, openai-compatible) — it was surfacing the
  unrelated OpenAI fallback id (gpt-5.6-terra); now prompts "Set a model
  with /model" (the fix idea from upstream langchain-ai#167, ported without its
  signature change).

Tests: failing-after-write stub asserts metadata persists on throw;
provider-switch notice covers empty-modelOptions providers.
Verified: typecheck, lint, format, pnpm test (327), build.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
audichuang added a commit to audichuang/openwiki that referenced this pull request Jul 16, 2026
@audichuang @claude
fix: per-provider agent-CLI login copy (stop telling claude-code user…
0850026 
…s 'grok login')

Onboarding hardcoded 'Grok Build CLI login (grok login)' for ALL agent-cli
providers (credentials.tsx getCredentialSetupDetail, cli.tsx api-key error),
so claude-code users were told to run grok login. Add a loginCommand field to
AgentCliProviderConfig (claude -> 'claude', grok -> 'grok login') + a
getProviderLoginCommand helper, and route both strings through it.

Idea from upstream langchain-ai#293 (authLoginHint); implemented for our discriminated-union
config. TDD: login-command helper + claude-code-not-grok regression test.
Verified: typecheck, lint, format, pnpm test (359), build.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
audichuang added a commit to audichuang/openwiki that referenced this pull request Jul 16, 2026
@audichuang @claude
feat: persist agent-CLI sessions + fall back on stale resume (from la…
7e5ebb9 
…ngchain-ai#293)

- New session-store.ts persists thread -> {provider, sessionId} to
  ~/.openwiki/cli-sessions.json (0o600, corrupt-tolerant, provider-tagged),
  replacing the in-memory Map so chat follow-ups resume the vendor CLI
  session across separate openwiki processes. Path overridable via
  OPENWIKI_CLI_SESSIONS_PATH (keeps tests out of the real home).
- runAgentCliRun retries once with a fresh session when a --resume run fails
  (stale/expired id), instead of wedging the follow-up. Failure-metadata
  persistence refactored into a shared helper across both catch paths.

Grafted from upstream langchain-ai#293 (sessions.ts + resume fallback), adapted to our
engines/ stack and discriminated-union providers. TDD: session-store unit
tests (round-trip, provider mismatch, disk persistence, clear, corrupt-file)
+ a resume-fallback integration test.
Verified: typecheck, lint, format, pnpm test (366), build.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
audichuang added a commit to audichuang/openwiki that referenced this pull request Jul 16, 2026
@audichuang @claude
feat: warn on repository-mode agent-CLI writes outside openwiki/ (from …
86ab643 
…langchain-ai#293)

The claude-code allowlist grants unscoped Write/Edit/MultiEdit and the runner
now passes --add-dir, so the vendor CLI's cwd is no longer a hard write
boundary. Add a warn-only guard: baseline git status --porcelain before a
repository-mode run, diff after, and emit a warning listing changed paths
outside openwiki/ (never fails the run). Handles renames and C-quoted paths;
pre-existing dirty files are excluded. Skipped in local-wiki mode (writes land
at the wiki root, not under openwiki/).

Grafted from upstream langchain-ai#293 (findUnexpectedChanges), adapted to our engines/
stack. Also refreshes the now-stale claude-code allowlist comment that claimed
--add-dir is never passed. TDD: write-guard unit tests + a stray-write
integration test.
Verified: typecheck, lint, format, pnpm test, build.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
audichuang added a commit to audichuang/openwiki that referenced this pull request Jul 16, 2026
@audichuang @claude
docs: trim AGENTS.md drift after FORK-NOTES + new engine modules
1ee0494 
Audit follow-up:
- Where-to-start 'agent-cli adapters' row enumerated 5 engine files and had
  gone stale (missing session-store.ts, write-guard.ts). Replace the
  enumeration with self-maintaining src/agent/engines/* + role hints.
- Branch-status duplicated FORK-NOTES.md (PR table, langchain-ai#293 rationale, run
  snippet, release caveat). Keep only the always-needed edit-time bits (kind
  invariant) and action-time bits (remote/push target) plus the FORK-NOTES
  pointer; the rest lives in FORK-NOTES (point, don't summarize).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@csy1204