Skip to content

Native CLI OpenRouter gateway route (parity with the managed Worker) - #982

Merged
gakonst merged 1 commit into
gakonst:masterfrom
dmarzzz:openrouter-provider
Oct 13, 2026
Merged

gakonst merged 1 commit into
gakonst:masterfrom
dmarzzz:openrouter-provider

Conversation

@dmarzzz

@dmarzzz dmarzzz commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

The managed Worker already routes GLM-5.3, Astra, Kimi K3 and MiMo through OpenRouter (and Vercel) Chat Completions, under the contract in docs/AGGREGATOR_ROUTING.md. This PR brings that OpenRouter route to the native CLI and SDK and follows the same contract, so a local ncl user with an OpenRouter key gets the same models and wire behaviour as the managed route.

OPENROUTER_API_KEY=... ncl run --provider.openrouter "fix the failing test"            # glm-5.3
OPENROUTER_API_KEY=... ncl run --provider.openrouter --model kimi-k3 "..."

SDK: OpenAi::builder(OpenAiAuth::api_key(key)).chat_gateway(ChatGateway::openrouter()).

Parity with the managed route

Contract (AGGREGATOR_ROUTING.md, gateway-responses.mjs) Native route
Closed catalog with the routing catalog's OpenRouter IDs glm-5.3 (the default) → z-ai/glm-5.3, gpt-6-astra → openai/gpt-6-astra, kimi-k3 → moonshotai/kimi-k3, mimo-v2.6-pro → xiaomi/mimo-v2.6-pro; anything else is refused at startup
Fixed HTTPS endpoint, bearer auth https://openrouter.ai/api/v1/chat/completions; only a loopback base may replace it (for local test servers)
Refuse redirects a dedicated client with redirects disabled; any 3xx is a classified refusal
Never parse, quote or retain provider error bodies an HttpRejected with the status, Retry-After and a fixed classified body only; mid-stream error chunks keep no text either
Chat translation of the Cloudflare adapters tool aliases, custom tools as one string input with the verbatim hint and grammar, additional_tools flattening, full-history tool messages, the portable nanocodex-chat-reasoning-v1: / nanocodex-chat-compaction-v1: envelopes, tool-alias diagnostics, unwrapped raw custom input
OpenRouter request fields reasoning.effort, provider.require_parameters, parallel_tool_calls: false omitted, max_tokens for Kimi/MiMo, MiMo's local forced-choice handling, no stream_options
GLM is text-only the same image notice in tool results, and a terminal 400 unsupported_content for direct images

How it plugs in. The new module crates/nanocodex-oai-api/src/transport/chat.rs sits behind the standard HTTPS transport:

  • ResponsesHttp::send hands it the encoded Responses Lite body when a gateway is configured, and it re-encodes the Chat reply as Responses events;
  • request encoding, retries, the event decoder, Code Mode and tools are unchanged, and exec still returns as a custom_tool_call;
  • text and reasoning stream live, while tool calls are released only after the completed reply validates, as the streaming adapter does;
  • chat_gateway() selects stateless HTTPS with full replay, as model(Glm53 | Kimi | Mimo) already does;
  • it's native only, and the wasm build is unchanged.

Decisions

  • No new configuration surface beyond --provider.openrouter, its key flag and the SDK builder method. The base URL is fixed (loopback is allowed for tests only).
  • Astra is included because the managed catalog routes it through OpenRouter. It's covered by the fake-endpoint journey only, not a real run, because of its price.
  • Malformed model output (an unknown tool alias or invalid tool JSON) is a retryable transport error, so the turn resamples instead of failing. The JS adapters classify "invalid provider stream" as non-retryable; matching that is a one-line change if you prefer it.
  • Unit tests: only the compaction envelope round trip remains, because the CLI can't fill a context window cheaply enough to trigger compaction. Everything else is covered by the shipped-CLI journeys, per AGENTS.md.

Stacked follow-ups

Kept out of this PR so it stays parity-only:

Tests

Shipped-CLI journeys (bin/nanocodex/tests/openrouter_provider.rs). Only Chat Completions inference is synthetic:

  1. Exec round trip: the first request's shape (bearer key, wire ID, provider.require_parameters, no Responses input or additional_tools, exec as a function tool with a single string input); a fragmented streamed tool call runs as a custom exec call that writes a file; the second request replays the call and its output; final answer and run.completed.
  2. Rejections and redirects: a 401 whose body echoes the key is reported only by class and status, with no body text and no key; a 307 is refused and the redirect target is never contacted.
  3. Fail-closed aliases: an unregistered tool alias (tool_999) fails the attempt (normalize_tool_alias_unregistered_wire); the retry's unwrapped raw cell runs exactly as written.
  4. Catalog wire IDs: Kimi, MiMo and Astra send the routing catalog's IDs with require_parameters and no parallel_tool_calls.
  5. Refusals: a missing key, an unserved model (sol) and a foreign base URL are all refused before anything is sent.

Real OpenRouter runs and full transcripts are in the evidence comment below.

Local CI: see the evidence comment for each CI step's result, run locally with Rust 1.97.

Closes #981

🤖 Generated with Claude Code

`ncl --provider.openrouter` (OPENROUTER_API_KEY) and
`OpenAiBuilder::chat_gateway(ChatGateway::openrouter())` bring the managed
Worker's OpenRouter route (docs/AGGREGATOR_ROUTING.md) to the native CLI and
SDK, following its contract: the closed catalog (GLM-5.3, GPT-6 Astra, Kimi K3,
MiMo) with the routing catalog's wire IDs, a fixed HTTPS endpoint with refused
redirects, classified provider errors without bodies, and the Cloudflare
adapters' Chat Completions translation (tool aliases, custom tools as one
string `input`, full-history tool messages, portable reasoning and compaction
envelopes, single-call mode, `provider.require_parameters`, `max_tokens`,
MiMo's local forced choice, the GLM image notice).

The native HTTPS transport keeps the Responses Lite request until the wire and
re-encodes the Chat reply as Responses events, so `exec` still returns as a
`custom_tool_call`.

Closes gakonst#981

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dmarzzz
dmarzzz force-pushed the openrouter-provider branch from e6f21e3 to f698376 Compare October 11, 2026 22:33
@dmarzzz dmarzzz changed the title Serve GLM-5.3, Kimi K3 and MiMo through OpenRouter from the native CLI Native CLI OpenRouter gateway route (parity with the managed Worker) Oct 11, 2026
@dmarzzz

dmarzzz commented Oct 11, 2026

Copy link
Copy Markdown
Contributor Author

Evidence (base #982 at f69837619)

All runs were on macOS arm64 with Rust 1.97, the toolchain CI uses. Evidence files stay in the ignored output/ directory and are not committed.

1. Shipped-CLI journeys (synthetic Chat Completions endpoint only)

cargo nextest run --workspace --locked --profile ci     # these lines are from the full workspace run
PASS nanocodex-bin::openrouter_provider openrouter_turn_runs_exec_as_a_custom_tool_and_replays_it
PASS nanocodex-bin::openrouter_provider openrouter_rejections_and_redirects_are_classified_without_bodies
PASS nanocodex-bin::openrouter_provider openrouter_unknown_tool_alias_fails_closed_and_raw_custom_input_runs
PASS nanocodex-bin::openrouter_provider openrouter_sends_the_managed_catalog_wire_ids
PASS nanocodex-bin::openrouter_provider openrouter_refuses_missing_keys_and_unserved_models
PASS nanocodex-oai-api transport::chat::tests::compaction_round_trips_through_the_portable_summary_envelope
Journey Input Expected Observed
Exec round trip ncl run --provider.openrouter --api-base-url http://127.0.0.1:<port>/api/v1 --model glm-5.3 "write the marker"; the endpoint streams a fragmented tool_calls reply for the exec alias the request carries z-ai/glm-5.3, provider.require_parameters, no Responses input or additional_tools, and exec as a function tool with a single string input; the cell runs; turn 2 replays tool_calls + tool; run.completed as expected; marker.txt contains ok, and the tool.call arguments are the raw cell
Rejection a 401 whose body echoes Bearer sk-or-v1-… and an Authorization header classified error with status 401; no body text and no key in stdout/stderr "the gateway rejected the request" (401); the key and body text are absent
Redirect a 307 to a second local server refused; the target is never contacted "the gateway answered with a redirect, which is refused"; the second server recorded 0 requests
Fail-closed alias turn 1 returns tool_999; the retry returns the exec cell unwrapped (no {"input": …}) the attempt fails as normalize_tool_alias_unregistered_wire; the retry's raw cell runs verbatim as expected
Catalog IDs --model kimi-k3, mimo-v2.6-pro, gpt-6-astra moonshotai/kimi-k3, xiaomi/mimo-v2.6-pro, openai/gpt-6-astra; require_parameters; no parallel_tool_calls as expected
Refusals no key; --model sol; --api-base-url https://gateway.example/api/v1 refused at startup; 0 requests as expected

2. Real OpenRouter run (GLM-5.3)

cd <ws> && CODEX_HOME=<tmp> ncl run --link-homes false --provider.openrouter --model glm-5.3 --cwd <ws> --subagents false \
  "Work in strictly separate steps, one tool call per step … Step 1: write buggy.py … mean(xs) returns sum(xs)/len(xs)-1 … Step 4: run it again and confirm it prints 4.0."

Expected: a correct buggy.py with exec round-tripping through multiple model calls. Observed: exit 0, and python3 buggy.py prints 4.0.

tool.call   exec          "const r = await tools.apply_patch(\"*** Begin Patch\\n*** Add File: buggy.py\\n+def mean(xs):\\n+    return s
tool.call   apply_patch   "*** Begin Patch\n*** Add File: buggy.py\n+def mean(xs):\n+    return sum(xs) / len(xs) - 1\n+\n+\n+print(mean
tool.result apply_patch   completed
tool.result exec          completed
tool.call   exec          "const r = await tools.exec_command({ cmd: \"python3 buggy.py\", workdir: \"<ws>
tool.call   exec_command  "{'cmd': 'python3 buggy.py', 'workdir': '<ws>
tool.result exec_command  completed
tool.result exec          completed
tool.call   exec          "const r = await tools.exec_command({ cmd: \"python3 buggy.py\", workdir: \"<ws>
tool.call   exec_command  "{'cmd': 'python3 buggy.py', 'workdir': '<ws>
tool.result exec_command  completed
tool.result exec          completed
tool.call   exec          "const r = await tools.exec_command({ cmd: \"python3 buggy.py\", workdir: \"<ws>
tool.call   exec_command  "{'cmd': 'python3 buggy.py', 'workdir': '<ws>
tool.result exec_command  completed
tool.result exec          completed
tool.call   exec          "const r = await tools.apply_patch(\"*** Begin Patch\\n*** Update File: buggy.py\\n@@\\n def mean(xs):\\n-    
tool.call   apply_patch   "*** Begin Patch\n*** Update File: buggy.py\n@@\n def mean(xs):\n-    return sum(xs) / len(xs) - 1\n+    retur
tool.result apply_patch   completed
tool.result exec          completed
tool.call   exec          "const r = await tools.exec_command({ cmd: \"python3 buggy.py\", workdir: \"<ws>
tool.call   exec_command  "{'cmd': 'python3 buggy.py', 'workdir': '<ws>
tool.result exec_command  completed
tool.result exec          completed
attempt failures: none
run.completed: {"status": "completed", "model": "@cf/zai-org/glm-5.3", "transport": "responses_https_sse", "duration_ms": 8454, "model_calls": 7, "tool_calls": 12, "response_retries": 0, "cost_status": "estimated_from_usage", "cost_usd": "0.04584172"}

OpenRouter billed $0.0309 for this run, measured as the change in key usage. The local rate-table estimate was $0.0458; #983 replaces it with the reported charge.

3. Local CI (.github/workflows/ci.yml steps, Rust 1.97)

Step Result
cargo build --locked --workspace --tests --bin nanocodex-hand pass
cargo nextest run --workspace --locked --profile ci 3167 passed, 6 failed, 81 skipped. The 6 failures (3 update::store tests, nanocodex2_background_hand late_computer_provider…, nanocodex2_tui_lifecycle stalled_tmux_hint…, and nanocodex-hand-daemon native_hand…reconnect…) also fail on unmodified master on this macOS host, so they're pre-existing and environment-specific. None touch this change
cargo test --workspace --locked --doc pass
node scripts/check-fast.mjs (fmt + clippy on affected crates) fmt passes; clippy on all affected library crates (--all-features -D warnings) passes; nanocodex-bin clippy passes. nanocodex-hand-daemon clippy fails with macOS-only dead-code errors in Linux Hand files (linux_hand_install.rs, screen_native.rs) that this PR doesn't touch; CI runs it on Ubuntu
cargo deny check pass
scripts/check-experimental-boundary.sh, check-crate-boundaries.sh, check-rustls-provider.sh, release-crates.sh check pass
typos pass
rust-extra contracts (cargo check of the independently usable crates) pass
cargo doc --workspace --all-features -D warnings fails on macOS in netlink-sys (a Linux-only dependency under --all-features), not in this change. Scoped cargo doc -p nanocodex-oai-api -p nanocodex-bin --all-features -D warnings passes
WASM (js/nanocodex-vite/scripts/build-js-package.sh --release) pass

🤖 Generated with Claude Code

@gakonst
gakonst merged commit dad94bb into gakonst:master Oct 13, 2026
gakonst added a commit that referenced this pull request Oct 13, 2026
OpenRouter gateway: account reported charges and fail stalled streams (stacked on #982)
gakonst added a commit that referenced this pull request Oct 13, 2026
OpenRouter gateway: wire-model passthrough for cheap family variants (stacked on #982)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OpenRouter Responses API: additional_tools rejected and custom exec tool returned as function_call

2 participants