Skip to content

feat: add OrcaRouter as a named LLM provider (OpenAI-compatible gateway) - #141

Open
XiaoHuo888-hue wants to merge 1 commit into
vstorm-co:mainfrom
XiaoHuo888-hue:feat/orcarouter-provider
Open

feat: add OrcaRouter as a named LLM provider (OpenAI-compatible gateway)#141
XiaoHuo888-hue wants to merge 1 commit into
vstorm-co:mainfrom
XiaoHuo888-hue:feat/orcarouter-provider

Conversation

@XiaoHuo888-hue

Copy link
Copy Markdown

Summary

Adds OrcaRouter as a model provider. OrcaRouter is an OpenAI-compatible model routing gateway that exposes 150+ models from OpenAI, Anthropic, Google, DeepSeek, Qwen, MiniMax, xAI and others behind a single endpoint and API key. It also provides gateway-level security controls for AI agents.

I'm an engineer on the OrcaRouter team.

Changes

  • Generator (fastapi_gen/): new LLMProviderType.ORCAROUTER enum value and use_orcarouter cookiecutter flag (single-provider or all). OrcaRouter is accepted only for pydantic_ai / pydantic_deep (same constraint as OpenRouter). RAG embeddings auto-derive to sentence-transformers because OrcaRouter exposes chat/completions but not an embeddings endpoint.
  • Interactive wizard: prompt_llm_provider offers "OrcaRouter (model routing)" for PydanticAI / PydanticDeep; --llm-provider orcarouter added to the CLI.
  • Generated backend wiring:
    • core/config.pyORCAROUTER_API_KEY, default AI_MODEL=anthropic/claude-sonnet-4.6, and an AI_AVAILABLE_MODELS list (catalog-verified IDs).
    • agents/assistant.py — OrcaRouter dispatch through OpenAIResponsesModel + OpenAIProvider(base_url="https://api.orcarouter.ai/v1"), mirroring how OpenRouter is wired.
    • agents/pydantic_deep_assistant.py — same model-object wiring for the PydanticDeep agent (a model string would resolve to the OpenAI endpoint instead).
    • api/routes/v1/health.py — OrcaRouter included in the readiness LLM key map.
    • backend/pyproject.toml — OrcaRouter uses pydantic-ai-slim[openai,duckduckgo,web-fetch] (OpenAI-compatible transport).
  • Generated docs/config: ORCAROUTER_API_KEY in .env.example, ENV_VARS.md, kubernetes/secret.yaml, MANUAL_STEPS.md, and docs/configuration.md.
  • Repo docs: provider tables in README.md, AGENTS.md, CLAUDE.md, docs/index.md, docs/guides/configuration.md, docs/rag.md, and template/VARIABLES.md.
  • Tests: OrcaRouter config validation (framework constraint, context flags), wizard prompt option, RAG embedding derivation, and a template-integration matrix case.

Testing

  • New tests added for the new functionality
  • pytest tests/test_config.py tests/test_prompts.py tests/test_rag_config.py — 205 passed, 0 failures
  • pytest tests/test_rag_integration.py::TestRAGWithEmbeddingProviders — 4 passed
  • ruff check fastapi_gen tests and ruff check on three freshly generated OrcaRouter projects (pydantic_ai, pydantic_deep, all-providers) — clean
  • Live (real key), through the same OpenAIResponsesModel + OpenAIProvider(base_url=https://api.orcarouter.ai/v1) code path:
    • anthropic/claude-sonnet-4.6 (default) → 200 PONG
    • orcarouter/auto → 200 PONG
    • openai/gpt-4o → 200 PONG
    • wrong key → 401
  • test_template_integration.py matrix — the [orcarouter] instances generate and render cleanly; the ruff/ty checks in that module invoke uvx/uv, which are unavailable on this Windows box. The failure count there is 61 on this branch vs 59 on pristine main — the +2 are exactly the added orcarouter parametrization, failing for the same uvx/uv not-found reason as every other matrix config on main. No config that passes on main fails on this branch.
  • Type checking (ty check) — requires uv, not run locally

Related Issues

N/A

Notes for Reviewers

OrcaRouter is OpenAI-compatible, so the transport is OpenAIResponsesModel pointed at https://api.orcarouter.ai/v1 rather than a dedicated model class. Model IDs in AI_AVAILABLE_MODELS were checked against the live /v1/models catalog (189 models). Embeddings are intentionally not wired: the OrcaRouter /embeddings endpoint returns 403, so RAG defaults to local sentence-transformers for this provider.

Add OrcaRouter (OpenAI-compatible model routing gateway) as a named LLM
provider alongside OpenRouter. Wires the generator, interactive wizard,
generated backend (pydantic_ai + pydantic_deep), RAG embedding
auto-derivation, env/docs, and tests.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: XiaoHuo888 <sjh00112233@outlook.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Triage

Development

Successfully merging this pull request may close these issues.

2 participants