Skip to content

feat(providers): add Ollama & LM Studio local providers and explicit model attribution - #204

Merged
antonbabenko merged 3 commits into
antonbabenko:masterfrom
ivankorn:feat/local-providers-and-model-names
Oct 15, 2026
Merged

antonbabenko merged 3 commits into
antonbabenko:masterfrom
ivankorn:feat/local-providers-and-model-names

Conversation

@ivankorn

@ivankorn ivankorn commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This PR adds first-class support for local inference runtimes (Ollama and LM Studio) alongside OpenRouter and built-in providers, and introduces explicit, collision-free delegate attribution (<provider>:<alias>).

Following reviewer feedback on PR #204, this PR has been rebased onto master (v3.30.0), temporal grounding has been dropped (addressed in #205), host skills have been split to a separate PR, and the blocking items from both review rounds (B1-B16, then section 4 below) have been resolved.


Key Changes

1. Dedicated Local Transports & Model Record Providers

  • Local Providers (ollama, lmstudio):
    • models records accept "provider": "openrouter" | "ollama" | "lmstudio".
    • Added dedicated ollama (http://localhost:11434/v1) and lmstudio (http://localhost:1234/v1) OpenAICompatibleProvider instances in server/mcp/index.js.
    • Added ollama and lmstudio blocks with default endpoints to config/config.default.json and updated config/config.schema.json.
    • Dropped temporary llmstudio alias in favor of canonical lmstudio.
  • Keyless Transport Security (B13a, B13b, B14):
    • Local providers are keyless by default (apiKeyEnv: ""), sending no Authorization header and ensuring OPENROUTER_API_KEY is never leaked to local or custom endpoints.
    • The standalone OpenRouter bridge filters delegates to provider === "openrouter" only and rejects non-OpenRouter models with model-not-allowed.
    • Custom apiBase endpoints on OpenRouter delegates do not inherit OPENROUTER_API_KEY unless explicitly configured with apiKeyEnv.
  • Model Tag Syntax (LP3):
    • Slugs in models.<alias>.model accept colons, dots, and slashes (e.g. nemotron-3-ultra:cloud, llama3.3:70b, glm-5.3:cloud), while aliases stay restricted to ^[a-z0-9-]+$.

2. Collision-Free Delegate Attribution (B1, B16)

  • <provider>:<alias> Attribution:
    • In core/registry.js, formatDelegateName(m) formats configured delegates as <provider>:<alias> (e.g., ollama:nemotron-local, lmstudio:deepseek-local, openrouter:claude-arb).
    • Keying by alias rather than model slug guarantees that two records sharing a model slug (e.g. across two endpoints) remain independent in the dedup cache, circuit-breaker streaks, and arbiter tracking.
  • Built-in Gemini Preservation (B1):
    • The built-in Gemini adapter retains its canonical name gemini in startStdio, panel, ask-all, etc., reporting the active model in the model result field.
  • Dropped google Provider from models (B3, B4, B15):
    • Custom Gemini models are configured exclusively via providers.gemini.model, eliminating the risk of arbitrary model IDs being sent to agy, misrouting arbiters to OpenRouter, or bypassing providers.gemini.enabled: false.

3. Reviewer Feedback & Hardening (B2, B5, B6, B7, B8, B9)

  • Null Guard in resolveModels (B2): Restored if (!isObject(m)) null guard so entries like models: { x: null } land cleanly in invalidModels instead of throwing.
  • Config Validation Reporting (B5): Preserved invalidModels in validateConfig regardless of OpenRouter enabled state so malformed local records are always reported.
  • Panel Dispatch Enforcement (B6): Removed matchModel fallback in ask-one to strictly dispatch from active panel members (selectForAskAll).
  • Auto-Arbiter Preference (B7): Kept auto-arbiter preference restricted to openrouter:* so stopped local daemons are not auto-selected as arbiters. Explicit { model: "<id>" } arbiters dispatch cleanly to their configured provider.
  • Analytics Levers (B8): core/analyze.js keys configLevers strictly by alias, identifies local models as kind: "local", and suggests models.<alias>.askAll without skewing OpenRouter outlier counts or generating invalid OpenRouter compare links.
  • Timeout Inheritance (B9): pinAlias falls back to providers.<prov>.timeout when delegate.timeout is unset.
  • No DelegationRequest Widening: Avoided widening DelegationRequest with transport details; endpoints and auth are encapsulated in provider instances.

4. Standalone Bridge Auth and Sessions (review round 2)

  • Per-record models.<id>.apiBase / apiKeyEnv are honored only by the standalone OpenRouter bridge. The unified server ignores them and emits a warning; apiKeyEnv must be a non-empty string.
  • Unknown providers fail closed: a delegate whose provider is not registered is reported in unavailable[] and never falls back to OpenRouter. An explicit arbiter on an unregistered provider warns and falls back to auto-selection.
  • Standalone key rule (resolveEffectiveAuth): record apiKeyEnv set -> that env var, or keyless if empty, never the provider key; no record apiBase, or the same base as the provider after trailing slashes are stripped -> provider key; any other base -> keyless. SETUP.md and TECHNICAL.md carry the migration note.
  • openrouter-reply reuses the session's stored endpoint and key env-var name (never the secret), so config edits do not change a running thread; a different alias or model re-resolves from config.

Configuration Example

{
  "version": 1,
  "providers": {
    "gemini": {
      "enabled": true,
      "model": "gemini-3.8-flash-high"
    },
    "ollama": {
      "enabled": true,
      "apiBase": "http://localhost:11434/v1"
    },
    "lmstudio": {
      "enabled": true,
      "apiBase": "http://localhost:1234/v1"
    },
    "openrouter": {
      "enabled": true,
      "apiKeyEnv": "OPENROUTER_API_KEY"
    }
  },
  "models": {
    "nemotron-local": {
      "provider": "ollama",
      "model": "nemotron-3-ultra:cloud",
      "askAll": true,
      "consensus": true
    },
    "deepseek-local": {
      "provider": "lmstudio",
      "model": "deepseek-r1-distill-qwen-14b",
      "askAll": true,
      "consensus": false
    },
    "claude-arb": {
      "provider": "openrouter",
      "model": "anthropic/claude-3.7-sonnet",
      "askAll": true,
      "consensus": true
    }
  }
}

Dispatched delegates resolve explicitly as:

  • gemini (runtime model reported in model)
  • ollama:nemotron-local
  • lmstudio:deepseek-local
  • openrouter:claude-arb

Verification & Tests

  • test/local-providers.test.js: 17 targeted regression tests, including:
    • LP1: validateConfig accepts ollama and lmstudio providers in models.
    • LP-B2: models: { x: null } lands in invalidModels without throwing.
    • LP-B5: Invalid local record reported in invalidModels when OpenRouter is disabled.
    • LP-B1-B16: Gemini keeps name gemini, and formatDelegateName uses <provider>:<alias>.
    • LP-B13a-B14: Standalone OpenRouter bridge rejects non-OpenRouter models and guards apiKey.
    • LP-B13b: Keyless local endpoints send no Authorization header and do not leak OPENROUTER_API_KEY.
    • LP-B4-B6: pinAlias pins to target provider and ask-one dispatches from active panel.
    • LP-B8: configLevers maps by alias only (not slug), and leverFor identifies local models.
    • LP-B9: providers.ollama.timeout applies when delegate.timeout is unset.
    • LP3: validateConfig accepts colons, dots, and slashes in model slugs.
    • LP-R1-LP-R8: fail-closed provider lookup, arbiter fallback, unified-server warning, standalone auth rules 1-3, reply session reuse.
    • LP-R9: local delegate timeout and model survive the dispatch settings overlay.
    • LP-R10: standalone openrouter-list lists only openrouter records.
  • Gates: npm run check (typecheck + every suite, 1207 tests), npm run sync:check and npm run test:contracts (19) pass on the head commit.
  • Line Length: All markdown documentation adheres strictly to $\le 80$ columns.

Maintainer follow-up (4bed433, b587c51)

Fixes found in the final review, pushed to this branch:

  • core/settings.js resolves any <provider>:<alias> delegate name. Before, only openrouter: names resolved, so ollama:/lmstudio: delegates were dispatched with the 180s default timeout regardless of models.<id>.timeout or providers.<p>.timeout, and reported an empty model in provenance (LP-R9).
  • Standalone openrouter-list lists only openrouter records; the openrouter tool cannot call local aliases (LP-R10).
  • Removed duplicate warning loops in server/mcp/index.js (validateConfig already emits the warning).
  • Reverted an unrelated core/analyze.js change that pointed codex high-error rows at providers.codex.reasoningEffort.
  • TECHNICAL.md: list scope, invalidModels with a disabled provider, accepted provider values. config.schema.json: threat model for per-record apiBase / apiKeyEnv.

Cross-model review (/consensus, code-reviewer, converged in 2 rounds; Gemini, Grok and an OpenRouter delegate approved, Codex timed out). Accepted: schema threat model (fixed in b587c51). Dismissed, with reasons:

  • <provider>:<alias> split ambiguous for aliases with : - record ids must match ^[a-z0-9-]+$, so the first-colon split is unambiguous.
  • resolveArbiter({ model }) could pin a disabled record - it searches the enabled-filtered model list and checks providerEnabled.
  • Same-alias replies reuse the stored endpoint without a re-check - that is the requested session-isolation behavior.
  • Docs outside the change set - README, SETUP, TECHNICAL, tool guide, schema and default config are all updated; sync:check passes.
  • Missing provider block vs enabled: false unclear - a missing block counts as enabled, same as other providers.
  • Arbiter { model } key unclear - unchanged existing contract: it names the models record id.
  • Startup-only apiBase undocumented - documented in TECHNICAL.md and SETUP.md.

@ivankorn ivankorn changed the title feat(providers): add first-class ollama & lmstudio providers, explicit model naming, and google model pinning feat(providers): support ollama & lmstudio, transparent model naming, and Antigravity slash commands Sep 22, 2026
@ivankorn ivankorn changed the title feat(providers): support ollama & lmstudio, transparent model naming, and Antigravity slash commands feat(providers): add Ollama & LM Studio support, transparent model naming, and mandatory temporal grounding Sep 22, 2026

@antonbabenko antonbabenko left a comment •

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this. Native Ollama / LM Studio support with honest ollama: / lmstudio: labels is something we want. This review was cross-checked by a multi-model panel (GPT, Gemini, Grok, DeepSeek, Kimi) over 6 rounds; every item below was verified against the diff. Requesting changes before merge.

Naming rule the items below assume: the Gemini built-in keeps the name gemini; every model record is named <provider>:<alias> (the unique config id), never by model slug.

Blocking

B13 (security): OPENROUTER_API_KEY is sent to non-OpenRouter endpoints, on two paths.

  • (a) The standalone bridge server/openrouter/index.js resolves any record from the shared models list, which now includes ollama / lmstudio / google records, and always sends process.env[or.apiKeyEnv] to delegate.apiBase || or.apiBase. A local record with its own apiBase receives the OpenRouter key; one without apiBase sends a local slug to OpenRouter.
  • (b) Unified server: on master pinAlias did not forward a per-record apiBase. This PR forwards it, and openai-compatible.js falls back to the provider apiKeyEnv, so any record (including provider: "openrouter") with a custom apiBase and no apiKeyEnv now receives OPENROUTER_API_KEY.
  • Please require an explicit credential choice for any non-default endpoint, and keep non-openrouter records out of the standalone bridge.

B14: Same root cause. openrouter-list and the openrouter tool list and serve local and google records as OpenRouter delegates.

B15: isModelEnabled in validateConfig checks providersRaw[m.provider]. For google records that is providers.google, which does not exist, so providers.gemini.enabled: false does not suppress them.

B16: formatDelegateName uses the model slug, not the alias. Two records with the same slug (for example two Ollama endpoints) get the same provider name, so the dedup cache, circuit-breaker streaks and arbiter exclusion treat them as one.

B1: startStdio names the Gemini built-in google:${geminiCfg.model}. config.default.json ships model: "auto-gemini-3", so every existing install's gemini voice is renamed to google:auto-gemini-3. Lookups still work through the new alias, but opinion source, debug-log / analyze rows, session history and the command docs all change, and history splits across the upgrade. Keep the built-in name gemini and report the model in the model field.

B2: resolveModels dropped the if (!isObject(m)) guard. models: { x: null } now throws a TypeError at load instead of landing in invalidModels. Please restore it and add a test.

B3: antigravity.js now honors req.model, so google records pass arbitrary ids to agy. agy rejects non-shipped pins with model-not-allowed (see CLAUDE.md design decision #9). Either drop google records or show with a dispatch test that per-call ids work.

B4: The object form of resolveArbiter sends every record through pinAlias(orProvider, ...), so a google arbiter is dispatched to OpenRouter.

B5: Local records ride the openrouter transport object. It is always constructed, so dispatch still works with OpenRouter disabled, but invalidModels is reported only when OpenRouter is enabled, so a bad local record is dropped without a word. Please report invalid local records regardless of OpenRouter state. A dedicated local transport would also close B13(b).

B6: On master, ask-one only dispatches from the selectForAskAll set. The new matchModel fallback makes askAll: false, over-cap and expert-ineligible records callable, while the error text still says "not in the active panel". If this is intended, document it and honor the enabled flags; otherwise remove the fallback.

B7: Local health() always returns { ok: true }, and resolveArbiter now auto-prefers ollama: / lmstudio:, so a stopped local daemon can be picked as the arbiter. Please add a reachability probe that fails closed, or leave local names out of the auto-arbiter preference.

B8: In core/analyze.js, leverFor labels ollama: / lmstudio: rows as kind openrouter with the slug as alias, so analyze can recommend nonexistent models.<slug>.* settings. configLevers also does byAlias.set(m.model), which lets a slug overwrite another record's alias. Remove the slug keys, resolve rows by alias, and add a regression test (fixing B16 alone does not fix this).

B9: providers.ollama.timeout / providers.lmstudio.timeout are documented and resolved but never applied: pinAlias reads only delegate.timeout, so local calls inherit the OpenRouter provider ceiling. Wire it through or remove it from the schema and docs.

B12 (tests): The new tests cover validation and naming only. Please add tests for:

  • no OPENROUTER_API_KEY reaching a custom or local apiBase, on both paths
  • local apiBase routing, and a keyless call sending no Authorization header (callOpenRouter already omits it for an empty key, so this is a regression guard)
  • an invalid local record with OpenRouter disabled (B5)
  • B15, B16 (duplicate slugs), B1 naming, B2 null guard, B4 / B6 routing, B8, B9

Follow-ups (not blocking this PR)

  • Temporal grounding: merged in #205 (now on master), please drop it from this PR and rebase. We agree with the problem (delegates rejecting newer model or tool names as "hallucinated") and fixed it differently. Grok and OpenRouter run with no tools, so a date -u plus live-search rule cannot reach them. #205 instead stamps today's UTC date and a short "unrecognized means [unverified], never non-existent" rule into every delegate prompt in code. Looking up the facts stays with the host (a "Time-sensitive questions" note in AGENTS.md). Please remove plugins/deliberation/skills/temporal-grounding/, the "Mandatory Temporal Grounding" blocks in AGENTS.md / POWER.md / plugins/deliberation/rules/AGENTS.md, and the AWS / Terraform tool names.
  • Split the PR. The new ask-all / consensus host skills are not assessed in this review. Please open them as a separate PR.
  • Drop the llmstudio alias. It is new in this PR, so nothing depends on it yet.
  • config/config.default.json needs entries for the new provider keys (the repo's doc-upkeep rule), even though the schema and pinAlias already carry runtime defaults.
  • Hardening: MCP tool arguments never reach req.apiBase / req.apiKeyEnv today. Prefer resolving them from config inside the provider over widening DelegationRequest.

A narrower PR that adds a dedicated local provider keyed by record alias would very likely be accepted.

antonbabenko added a commit that referenced this pull request Sep 24, 2026
…gate prompt (#205)

* feat: stamp the current UTC date and a no-denial rule into every delegate prompt

Delegates trained months ago call newer model or tool names hallucinated and
file false critical issues. core/grounding.js adds today's UTC date plus a
one-sentence rule (unrecognized means [unverified], never non-existent) in all
four prompt builders, so every path gets it, standalone bridges included.
Looking facts up stays with the host: AGENTS.md and the ask-all/consensus
commands tell it to verify and inline them with an as-of date and source.

Replaces the temporal-grounding skill proposed in #204.

* fix: say "tools you actually have" so the grounding note does not contradict tool-less delegates

* docs: add the time-sensitive facts step to every /ask-* command and the delegation template
@ivankorn
ivankorn force-pushed the feat/local-providers-and-model-names branch from 223b09b to c8bd45b Compare September 27, 2026 11:36
@ivankorn ivankorn changed the title feat(providers): add Ollama & LM Studio support, transparent model naming, and mandatory temporal grounding feat(providers): add Ollama & LM Studio local providers and explicit model attribution Sep 27, 2026
@ivankorn

Copy link
Copy Markdown
Contributor Author

Thanks for the thorough review and clear architectural feedback, @antonbabenko!

I have addressed all the blocking points (B1–B16) and follow-up items, rebased onto master (v3.18.0), and force-pushed the branch.

Resolution Summary

  1. Rebase & Temporal Grounding (Follow-up & feat: stamp the current UTC date and a no-denial rule into every delegate prompt #205):
  2. PR Scope Split (Follow-up):
    • Dropped the host skills (plugins/deliberation/skills/ask-all/, plugins/deliberation/skills/consensus/) from this PR to keep it focused strictly on local providers and attribution.
  3. Drop llmstudio Alias (Follow-up):
    • Dropped the llmstudio alias; only lmstudio and ollama are accepted.
  4. Config Defaults (Follow-up):
    • Added entries for ollama (http://localhost:11434/v1) and lmstudio (http://localhost:1234/v1) in config/config.default.json and updated config/config.schema.json.
  5. Drop google from models (B3, B4, B15):
    • Dropped google from models. The built-in Gemini adapter continues to be configured via providers.gemini.model. This cleanly prevents agy model-not-allowed errors, arbiter misrouting to OpenRouter, and providers.gemini.enabled: false bypasses.
  6. Built-in Gemini Naming & Attribution (B1 & B16):
    • Built-in Gemini remains named gemini across startStdio, panel, ask-all, etc., reporting its runtime model in the model property (B1).
    • Configured delegates in models are formatted as <provider>:<alias> via formatDelegateName in core/registry.js (e.g. ollama:nemotron-local, lmstudio:deepseek-local, openrouter:claude-arb). Keying by alias rather than model slug prevents collisions in dedup cache, circuit-breaker streaks, and arbiter tracking when two endpoints share a model slug (B16).
  7. Transport Hardening & No Token Leakage (B13a, B13b, B14, Follow-up):
    • Did not widen DelegationRequest with apiBase/apiKeyEnv. Dedicated local provider instances (ollama, lmstudio) are instantiated with apiKeyEnv: "".
    • Keyless local endpoints send no Authorization header and never read or send OPENROUTER_API_KEY (B13b).
    • The standalone OpenRouter bridge only lists (openrouter-list) and resolves models with provider === "openrouter" (B14), rejecting non-OpenRouter models with model-not-allowed (B13a). Custom apiBase endpoints on OpenRouter delegates do not inherit OPENROUTER_API_KEY unless explicitly configured with apiKeyEnv.
  8. Null Guard in resolveModels (B2):
    • Restored if (!isObject(m)) null guard in resolveModels so entries like models: { x: null } land in invalidModels instead of throwing.
  9. Config Validation Reporting (B5):
    • validateConfig preserves invalidModels regardless of OpenRouter enabled state so malformed local records are always reported.
  10. Panel Dispatch Enforcement (B6):
    • Removed matchModel fallback from ask-one, strictly dispatching from the active panel set (selectForAskAll).
  11. Auto-Arbiter Preference (B7):
    • Auto-arbiter preference remains restricted to openrouter:*, so stopped local daemons cannot be auto-selected as arbiters. Explicit { model: "<id>" } arbiters dispatch cleanly to their configured provider.
  12. Analytics Levers (B8):
    • In core/analyze.js, configLevers keys strictly by alias (not model slug), leverFor identifies local models as kind: "local", and models.<alias>.askAll suggestions work without skewing slowOpenRouterCount or building OpenRouter compare links.
  13. Timeout Inheritance (B9):
    • pinAlias falls back to providers.<prov>.timeout when delegate.timeout is unset.
  14. Automated Regression Tests (B12):
    • Added 10 tests in test/local-providers.test.js covering LP1, LP3, LP-B1, LP-B2, LP-B4, LP-B5, LP-B6, LP-B8, LP-B9, LP-B13a, LP-B13b, LP-B14, LP-B16.
  15. Verification:
    • npm test: all 894 tests pass (0 failures).
    • npm run typecheck: passes with 0 errors.
    • node scripts/sync-hosts.js --check: 30 host files up to date with 0 drift.
    • All documentation lines wrapped to $\le 80$ characters.

@ivankorn

Copy link
Copy Markdown
Contributor Author

As requested in the follow-up review, the native host skills have been opened as a separate PR: #211

@antonbabenko antonbabenko left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the rebase and for working through B1-B16. The temporal grounding is gone, the skills moved out, llmstudio is dropped, and the default config has the new provider blocks. On the PR branch, npm test passes 894/894, and tsc and sync-hosts --check are clean.

We ran the branch through a multi-round cross-model review (GPT, Gemini, Grok, and two OpenRouter models). Every reviewer agreed on the blockers below. Most of them are about which endpoint an API key is sent to.

"Provider key" below means process.env[or.apiKeyEnv] (OPENROUTER_API_KEY by default).

Blockers

1. Per-record apiBase / apiKeyEnv are only half wired.
The schema now advertises models.<id>.apiBase and models.<id>.apiKeyEnv ("for this record only"). The unified server's pinAlias (core/registry.js) forwards neither of them. It does forward delegate.apiKey, but resolveModels never sets that field, so that path is dead. The PR also deletes the NOTE that said per-model apiBase only applies to standalone /ask-openrouter.

Smallest fix: keep both record fields standalone-bridge-only.

  • Say so in the schema descriptions and the docs.
  • Have the unified server emit a config warning at load for any record that sets them.
  • Remove the dead apiKey forwarding and restore the NOTE.
  • Validate models.<id>.apiKeyEnv as a non-empty string in resolveModels. A bad value becomes an invalidModels entry.

Wiring these fields into the unified server can come in a later PR.

2. The provider lookup must fail closed.
pinDelegates and resolveArbiter use byName.get(prov) || byName.get("openrouter"). If a provider is ever not registered, its model slug and prompt go to OpenRouter with the provider key. Validation prevents this today, but the fallback is the wrong default.

  • Never dispatch through another provider's transport.
  • If a delegate's provider is not registered, do not dispatch it; report it in unavailable.
  • If an explicit arbiter { model } has an unregistered provider, do not dispatch it. Warn and keep master's auto selection from the registered pool. Return an error without dispatch only when that pool is empty.

3. Standalone bridge key rule: good direction, needs to be complete and documented.
The bridge only serves provider: "openrouter" records, which the PR already enforces with model-not-allowed. Keep that. Evaluate the key in this order:

  1. apiKeyEnv is set on the record: use that env var. If the var is unset or empty, send no key. Never fall through to the provider key.
  2. The record has no apiBase, or its apiBase equals or.apiBase after stripping trailing slashes: use the provider key.
  3. Otherwise, send no key.

This changes behavior for existing users: a record with its own apiBase used to get OPENROUTER_API_KEY. Please add a migration note to SETUP.md and TECHNICAL.md. Authenticated proxies must set models.<id>.apiKeyEnv; keyless proxies leave it unset.

4. The standalone reply path sends proxy sessions to OpenRouter.
openrouter-reply rebuilds the delegate as { model: priorSession.model }. A reply to a proxy session therefore goes to or.apiBase with the provider key. Fix:

  • When a session starts, store the alias, the effective apiBase, and the NAME of the env var chosen by rule 3. Never store the secret.
  • Replies reuse that stored copy and read the env value at call time. Editing the config mid-session then does not change where replies go.
  • A reply with no alias/model, or with the session's own alias/model, keeps the session.
  • A reply that names a different alias (compared by alias, not model slug) or a different raw model is resolved again from the config. Raw models use or.apiBase and the provider key.

5. Tests for these paths.
The 10 new tests do not reach the defects above. Please add tests for:

  • An unknown-provider delegate is not dispatched and appears in unavailable.
  • An unknown-provider explicit arbiter is not dispatched, warns, auto-selects from the pool, and errors when the pool is empty.
  • The unified-server warning for ignored record fields, and apiKeyEnv validation.
  • Rules 1-3, including an unset named var, a trailing slash, and a non-default or.apiKeyEnv.
  • A reply on a session that uses the provider key sends that key to or.apiBase.
  • A reply on a proxy session reaches the proxy with its own key or no key, never the provider key.
  • A reply after a config edit keeps the stored values.
  • A reply naming a different alias or raw model is resolved again.
  • OpenRouter delegates are still named openrouter:<alias>.

Small, same PR

  • Drop the "NONE" sentinel from the openai-compatible.js health check and from TECHNICAL.md. An empty or unset apiKeyEnv field already means keyless.
  • config.schema.json: ollamaProvider.apiKeyEnv defaults to "OLLAMA_API_KEY", but the code default is keyless. Make it keyless, like lmstudioProvider.
  • One doc line: providers.ollama / providers.lmstudio apiBase is read at server start, so changing it needs a restart. providers.openrouter works the same way today.

Follow-ups (not blocking, we will track them)

  • Hot reload of apiBase for all HTTP providers.
  • A cheap daemon check for local providers (e.g. GET /v1/models) so a stopped Ollama or LM Studio shows up as unavailable. Today the retry and the circuit breaker limit the cost.
  • core/analyze.js: derive the ollama: / lmstudio: prefixes from one list.
  • Optional per-record apiBase / apiKeyEnv in the unified server.

Related PRs

  • #211 (host skills): we will review it after this PR merges and #211 is rebased on top.
  • #212 (temporal-grounding skill): closing as superseded by #205. The decision there was no skill: the date is stamped in code, and the lookup guidance lives in AGENTS.md and the commands.

…model attribution

- Add first-class Ollama and LM Studio local providers backed by OpenAI-compatible HTTP bridge
- Support explicit model naming/attribution across fan-out, arbitration, analysis, and dashboard
- Fail-closed provider dispatch: reject cross-transport fallback, mark unregistered delegates unavailable
- Safe auth resolution on standalone OpenRouter bridge with keyless defaults and session isolation
- Validate models.<id>.apiKeyEnv as non-empty string and warn on standalone-only keys in unified server
- Add comprehensive regression tests in test/local-providers.test.js covering all review concerns
- Update schemas, documentation (SETUP.md, TECHNICAL.md, README.md, tool-guide.md), and host skills
@ivankorn
ivankorn force-pushed the feat/local-providers-and-model-names branch from c8bd45b to a37b4f0 Compare October 14, 2026 15:54
@ivankorn

Copy link
Copy Markdown
Contributor Author

Thanks for the thorough review, @antonbabenko! All feedback items have been addressed and rebased cleanly on top of master (v3.30.0):

1. Per-record apiBase / apiKeyEnv scoped to standalone bridge

  • Updated config.schema.json, SETUP.md, and TECHNICAL.md to clearly mark models.<id>.apiBase and models.<id>.apiKeyEnv as standalone-bridge-only.
  • The unified server emits a startup warning when any model record defines apiBase or apiKeyEnv.
  • Removed dead apiKey forwarding from pinAlias in core/registry.js and restored the standalone NOTE.
  • Validated models.<id>.apiKeyEnv as a non-empty string in resolveModels; invalid values are rejected into invalidModels.

2. Fail-closed provider lookup

  • Provider lookup in core/registry.js (pinDelegates) and server/mcp/index.js (resolveArbiter) fails closed without falling back to OpenRouter. Unregistered provider delegates are reported in unavailable[].
  • Explicit arbiter { model } targeting an unregistered provider logs a warning and falls back to auto-selection from the registered provider pool; returns an error without dispatch if the pool is empty.

3. Standalone bridge auth rules 1–3 & migration note

  • Implemented resolveEffectiveAuth(delegate, or) in server/openrouter/index.js following the exact 3-step precedence:
    1. Record apiKeyEnv set: uses that env var (if unset or empty, sends keyless request; never falls through).
    2. Record has no apiBase or equals or.apiBase (trailing slashes stripped): uses provider key (OPENROUTER_API_KEY).
    3. Otherwise: sends keyless request (never leaking provider key).
  • Documented the migration note in SETUP.md and TECHNICAL.md (custom apiBase endpoints needing authentication must set apiKeyEnv).

4. Standalone reply path session isolation

  • sessions.set stores alias, effective apiBase, and apiKeyEnvVar name; secrets are never held in memory.
  • openrouter-reply reuses the stored endpoint and reads the env var at call time. Editing config mid-session does not alter active threads.
  • Replies with same/no alias keep the session; switching alias or raw model re-resolves from config.

5. Regression test suite

  • Added test/local-providers.test.js covering all 9 scenarios requested in the review (all 15 unit tests pass).
  • Full suite verified: all 1,207 tests, 19 multi-host contracts, typecheck, and sync checks pass (npm run check && npm run sync:check && npm run test:contracts).

Small items

  • Removed "NONE" sentinel from openai-compatible.js and TECHNICAL.md.
  • Keyless default on ollamaProvider.apiKeyEnv in config.schema.json (matching lmstudioProvider).
  • Documented in SETUP.md and TECHNICAL.md that providers.ollama and providers.lmstudio apiBase is read at startup.

…st to openrouter

- core/settings.js resolves any <provider>:<alias> name, so ollama/lmstudio
  delegates keep models.<id>.timeout and providers.<p>.timeout at dispatch
  instead of the 180s default, and report their model in provenance
- standalone openrouter-list returns only openrouter records; the openrouter
  tool rejects local aliases
- drop duplicate unified-server warning loops (validateConfig already emits it)
- revert the unrelated codex configKey in analyze high-error recommendations
- TECHNICAL.md: list scope, invalidModels with a disabled provider, provider rule
- tests: LP-R9 (timeout through the settings overlay), LP-R10 (list scope)
@antonbabenko

Copy link
Copy Markdown
Owner

Thanks @ivankorn, this round addressed every item from the last review. I checked each one against the code: per-record apiBase / apiKeyEnv are now standalone-only with the warning, provider lookup fails closed, resolveEffectiveAuth follows the three key rules, and replies reuse the stored endpoint and key env-var name.

I pushed two small commits on top so this can merge:

  • 4bed433 fix(providers):
    • core/settings.js only resolved openrouter:<alias> names. callProvider applies the resolved settings before pinAlias.ask, so every ollama: / lmstudio: delegate ran with the 180s default timeout, ignoring models.<id>.timeout and providers.<p>.timeout, and showed an empty model in provenance. It now resolves any <provider>:<alias> name. LP-B9 missed this because it calls ask() directly; the new LP-R9 goes through the settings overlay.
    • Standalone openrouter-list now lists only openrouter records, since the openrouter tool rejects local aliases (LP-R10).
    • Removed the two warning loops in server/mcp/index.js: validateConfig already adds that warning to consensusWarnings.
    • Reverted the core/analyze.js line that pointed codex high-error rows at providers.codex.reasoningEffort (unrelated to this PR).
    • TECHNICAL.md: list scope, invalidModels when a provider is disabled, accepted provider values.
  • b587c51 docs(config): threat-model text for per-record apiBase / apiKeyEnv in config.schema.json, raised by the cross-model review.

npm run check, npm run sync:check and npm run test:contracts pass. The /consensus code review converged in 2 rounds; the dismissed points and their reasons are in the PR description.

@antonbabenko
antonbabenko merged commit e8f55c9 into antonbabenko:master Oct 15, 2026
1 check passed
@github-actions

Copy link
Copy Markdown
Contributor

This PR is included in v3.31.0 🎉

Claude Code

/plugin marketplace update antonbabenko
/reload-plugins

If the update still shows the old version, the marketplace re-pin is a few minutes behind; run it again shortly.

Codex, Kiro, Cursor, OpenCode plugins: update deliberation from your host's plugin manager.

MCP server (any host): @antonbabenko/deliberation-mcp@3.31.0 is published. Restart your host to pick it up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants