diff --git a/AGENTS.md b/AGENTS.md index 8786142c..af056deb 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -57,7 +57,7 @@ Jinja2 conditionals: `{%- if cookiecutter.enable_rag %}...{%- endif %}` ## Key Features - **5 AI Frameworks**: PydanticAI, PydanticDeep, LangChain, LangGraph, DeepAgents -- **4 LLM Providers**: OpenAI, Anthropic, Google Gemini, OpenRouter +- **5 LLM Providers**: OpenAI, Anthropic, Google Gemini, OpenRouter, OrcaRouter - **RAG**: 4 vector stores (Milvus, Qdrant, ChromaDB, pgvector), 4 embedding providers, reranking, image description - **Document Sources**: Local files (CLI), API upload, Google Drive (service account), S3/MinIO - **Sync Sources**: Configurable connectors (Google Drive, S3) with scheduled sync diff --git a/CLAUDE.md b/CLAUDE.md index 9b86ea05..4455b17c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -92,7 +92,7 @@ template/ ## Key Design Decisions - 5 AI frameworks: PydanticAI, PydanticDeep, LangChain, LangGraph, DeepAgents -- 4 LLM providers: OpenAI, Anthropic, Google Gemini, OpenRouter +- 5 LLM providers: OpenAI, Anthropic, Google Gemini, OpenRouter, OrcaRouter - 4 vector store backends: Milvus, Qdrant, ChromaDB, pgvector - 4 embedding providers: OpenAI, Voyage, Gemini (multimodal), SentenceTransformers - RAG document sources: local files (CLI), Google Drive, S3/MinIO diff --git a/README.md b/README.md index 9f34c5ea..85b544a4 100644 --- a/README.md +++ b/README.md @@ -714,7 +714,7 @@ They also ship a ready-to-use **`.claude/` toolkit** that adapts to the options ### 🤖 AI/LLM First - **5 AI Frameworks** - [PydanticAI](https://ai.pydantic.dev), [PydanticDeep](https://github.com/vstorm-co/pydantic-deep), [LangChain](https://python.langchain.com), [LangGraph](https://langchain-ai.github.io/langgraph/), [DeepAgents](https://github.com/vstorm-co/pydantic-deepagents) -- **4 LLM Providers** - OpenAI, Anthropic, Google Gemini, OpenRouter +- **5 LLM Providers** - OpenAI, Anthropic, Google Gemini, OpenRouter, OrcaRouter - **RAG** - Document ingestion, vector search, reranking (Milvus, Qdrant, ChromaDB, pgvector) - **WebSocket Streaming** - Real-time responses with full event access - **Rich Chat UI** - Specialized tool-call cards (web search, knowledge base, Python, charts, skills), live subagent feed, citation sources panel, plan/task checklist, reasoning view, and in-chat file previews @@ -754,7 +754,7 @@ They also ship a ready-to-use **`.claude/` toolkit** that adapts to the options | Category | Integrations | |----------|-------------| | **AI Frameworks** | PydanticAI, PydanticDeep, LangChain, LangGraph, DeepAgents | -| **LLM Providers** | OpenAI, Anthropic, Google Gemini, OpenRouter | +| **LLM Providers** | OpenAI, Anthropic, Google Gemini, OpenRouter, OrcaRouter | | **RAG / Vector Stores** | Milvus, Qdrant, ChromaDB, pgvector | | **RAG Sources** | Local files, API upload, Google Drive, S3/MinIO, Sync Sources (per-org UI, scheduled) | | **Embeddings** | OpenAI, Voyage, Gemini (multimodal), SentenceTransformers | @@ -786,7 +786,7 @@ They also ship a ready-to-use **`.claude/` toolkit** that adapts to the options │ │ PydanticAI · LangChain · LangGraph · DeepAgents │ │ │ │ ──────────────────────────────────────────────────────────── │ │ │ │ Tools: datetime · web_search (Tavily) · search_knowledge_base │ │ -│ │ Providers: OpenAI · Anthropic · Gemini · OpenRouter │ │ +│ │ Providers: OpenAI · Anthropic · Gemini · OpenRouter · OrcaRouter │ │ │ └─────────────────────────────────────────────────────────────────┘ │ │ │ │ ┌─────────────────────────────────────────────────────────────────┐ │ @@ -890,7 +890,7 @@ See [Architecture Documentation](https://github.com/vstorm-co/full-stack-ai-agen ## 🤖 AI Agent -Choose from **5 AI frameworks** and **4 LLM providers** when generating your project: +Choose from **5 AI frameworks** and **5 LLM providers** when generating your project: ```bash # PydanticAI with OpenAI (default) diff --git a/docs/guides/configuration.md b/docs/guides/configuration.md index 604b3ae9..e381a3ae 100644 --- a/docs/guides/configuration.md +++ b/docs/guides/configuration.md @@ -10,7 +10,7 @@ All available options when generating a project. | `--orm` | `sqlalchemy`, `sqlmodel` | ORM choice (SQLModel for simplified syntax) | | `--oauth-google` | flag | Enable Google OAuth2 login | | `--ai-framework` | `pydantic_ai`, `pydantic_deep`, `langchain`, `langgraph`, `deepagents` | AI agent framework | -| `--llm-provider` | `openai`, `anthropic`, `google`, `openrouter` | LLM provider | +| `--llm-provider` | `openai`, `anthropic`, `google`, `openrouter`, `orcarouter` | LLM provider | | `--task-queue` | `none`, `celery`, `taskiq`, `arq` | Background task queue (Redis-backed). **Prefect** is available via the interactive wizard. | | `--frontend` | `none`, `nextjs` | Frontend framework | @@ -31,8 +31,8 @@ fastapi-fullstack create my_app --minimal | Framework | Providers | Description | |-----------|-----------|-------------| -| `pydantic_ai` | OpenAI, Anthropic, Google, OpenRouter | Type-safe agents with WebSearch/WebFetch built-in | -| `pydantic_deep` | OpenAI, Anthropic, Google | Deep coding assistant (filesystem tools, Docker/Daytona sandbox) | +| `pydantic_ai` | OpenAI, Anthropic, Google, OpenRouter, OrcaRouter | Type-safe agents with WebSearch/WebFetch built-in | +| `pydantic_deep` | OpenAI, Anthropic, Google, OpenRouter, OrcaRouter | Deep coding assistant (filesystem tools, Docker/Daytona sandbox) | | `langchain` | OpenAI, Anthropic, Google | Comprehensive chain-based agents | | `langgraph` | OpenAI, Anthropic, Google | Graph-based ReAct agents | | `deepagents` | OpenAI, Anthropic, Google | Agentic framework with subagent delegation | diff --git a/docs/index.md b/docs/index.md index c0658adc..70ec28cb 100644 --- a/docs/index.md +++ b/docs/index.md @@ -48,7 +48,7 @@ See the [Quick Start guide](guides/quick-start.md) for details and the [Installa | Framework | Streaming | Observability | Providers | |-----------|:---------:|:-------------:|:---------:| -| **PydanticAI** | WebSocket | Logfire | OpenAI, Anthropic, OpenRouter | +| **PydanticAI** | WebSocket | Logfire | OpenAI, Anthropic, OpenRouter, OrcaRouter | | **LangChain** | WebSocket | LangSmith | OpenAI, Anthropic | | **LangGraph** | WebSocket | LangSmith | OpenAI, Anthropic | diff --git a/docs/rag.md b/docs/rag.md index ab24fbd9..edbf8ad0 100644 --- a/docs/rag.md +++ b/docs/rag.md @@ -97,7 +97,7 @@ CROSS_ENCODER_MODEL=... # Model name (default: cross-encoder/ms-marco-MiniLM- | Option | Values | Description | |--------|--------|-------------| | `enable_rag` | bool | Enable RAG functionality | -| `embedding_provider` | auto-derived | Embedding model provider (auto-derived from LLM provider: OpenAI→openai, Anthropic→voyage, OpenRouter→sentence_transformers) | +| `embedding_provider` | auto-derived | Embedding model provider (auto-derived from LLM provider: OpenAI→openai, Anthropic→voyage, Google→gemini, OpenRouter→openai, OrcaRouter→sentence_transformers) | | `pdf_parser` | `pymupdf`, `llamaparse` | PDF parsing method (set via `--pdf-parser` CLI flag) | | `enable_reranker` | bool | Enable reranking (set via `--reranker` CLI flag: none/cohere/cross_encoder) | diff --git a/fastapi_gen/cli.py b/fastapi_gen/cli.py index 1ac8139d..4c766b51 100644 --- a/fastapi_gen/cli.py +++ b/fastapi_gen/cli.py @@ -395,11 +395,11 @@ def new(output: Path | None, no_input: bool, name: str | None, minimal: bool) -> ) @click.option( "--llm-provider", - type=click.Choice(["openai", "anthropic", "google", "openrouter", "all"]), + type=click.Choice(["openai", "anthropic", "google", "openrouter", "orcarouter", "all"]), default="openai", help=( "LLM provider (default: openai). 'all' installs every SDK and lets users " - "pick the model at runtime. openrouter requires pydantic_ai." + "pick the model at runtime. openrouter/orcarouter requires pydantic_ai." ), ) @click.option("--redis", is_flag=True, help="Enable Redis") @@ -1362,6 +1362,7 @@ def templates() -> None: console.print(" --llm-provider anthropic Anthropic (claude-opus-4-7)") console.print(" --llm-provider google Google Gemini (gemini-2.5-flash)") console.print(" --llm-provider openrouter OpenRouter (pydantic_ai only)") + console.print(" --llm-provider orcarouter OrcaRouter (pydantic_ai only)") console.print( " --websockets Enable WebSocket support (real-time chat streaming)" ) diff --git a/fastapi_gen/config.py b/fastapi_gen/config.py index fe356b06..23da2a4c 100644 --- a/fastapi_gen/config.py +++ b/fastapi_gen/config.py @@ -96,6 +96,7 @@ class LLMProviderType(StrEnum): ANTHROPIC = "anthropic" GOOGLE = "google" OPENROUTER = "openrouter" + ORCAROUTER = "orcarouter" ALL = "all" @@ -450,7 +451,7 @@ def validate_option_combinations(self) -> "ProjectConfig": raise ValueError("Caching requires Redis to be enabled") if ( self.ai_framework != AIFrameworkType.NONE - and self.llm_provider == LLMProviderType.OPENROUTER + and self.llm_provider in (LLMProviderType.OPENROUTER, LLMProviderType.ORCAROUTER) and self.ai_framework not in ( AIFrameworkType.PYDANTIC_AI, @@ -458,7 +459,7 @@ def validate_option_combinations(self) -> "ProjectConfig": ) ): raise ValueError( - f"OpenRouter is only supported with PydanticAI or PydanticDeep, " + f"OpenRouter/OrcaRouter is only supported with PydanticAI or PydanticDeep, " f"not {self.ai_framework.value}" ) if ( @@ -809,6 +810,8 @@ def to_cookiecutter_context(self) -> dict[str, Any]: "use_google": self.llm_provider in (LLMProviderType.GOOGLE, LLMProviderType.ALL), "use_openrouter": self.llm_provider in (LLMProviderType.OPENROUTER, LLMProviderType.ALL), + "use_orcarouter": self.llm_provider + in (LLMProviderType.ORCAROUTER, LLMProviderType.ALL), "use_all_providers": self.llm_provider == LLMProviderType.ALL, # Legacy fixed values (always enabled, not user-configurable) # AI @@ -900,6 +903,8 @@ def to_cookiecutter_context(self) -> dict[str, Any]: if self.llm_provider == LLMProviderType.ANTHROPIC else EmbeddingProviderType.GEMINI.value if self.llm_provider == LLMProviderType.GOOGLE + else EmbeddingProviderType.SENTENCE_TRANSFORMERS.value + if self.llm_provider == LLMProviderType.ORCAROUTER else EmbeddingProviderType.OPENAI.value ), "use_openai_embeddings": self.rag_features.enable_rag @@ -907,12 +912,14 @@ def to_cookiecutter_context(self) -> dict[str, Any]: not in ( LLMProviderType.ANTHROPIC, LLMProviderType.GOOGLE, + LLMProviderType.ORCAROUTER, ), "use_voyage_embeddings": self.rag_features.enable_rag and self.llm_provider == LLMProviderType.ANTHROPIC, "use_gemini_embeddings": self.rag_features.enable_rag and self.llm_provider == LLMProviderType.GOOGLE, - "use_sentence_transformers": False, + "use_sentence_transformers": self.rag_features.enable_rag + and self.llm_provider == LLMProviderType.ORCAROUTER, "enable_reranker": self.rag_features.enable_rag and self.rag_features.reranker_type != RerankerType.NONE, "use_cohere_reranker": self.rag_features.enable_rag diff --git a/fastapi_gen/prompts.py b/fastapi_gen/prompts.py index 66d31b01..2a157be1 100644 --- a/fastapi_gen/prompts.py +++ b/fastapi_gen/prompts.py @@ -786,11 +786,14 @@ def prompt_llm_provider(ai_framework: AIFrameworkType) -> LLMProviderType: questionary.Choice("Google Gemini (gemini-2.5-flash)", value=LLMProviderType.GOOGLE), ] - # OpenRouter available for PydanticAI and PydanticDeep (both use pydantic-ai) + # OpenRouter and OrcaRouter available for PydanticAI and PydanticDeep (both use pydantic-ai) if ai_framework in (AIFrameworkType.PYDANTIC_AI, AIFrameworkType.PYDANTIC_DEEP): choices.append( questionary.Choice("OpenRouter (multi-provider)", value=LLMProviderType.OPENROUTER) ) + choices.append( + questionary.Choice("OrcaRouter (model routing)", value=LLMProviderType.ORCAROUTER) + ) return cast( LLMProviderType, diff --git a/template/VARIABLES.md b/template/VARIABLES.md index 49387164..277b0494 100644 --- a/template/VARIABLES.md +++ b/template/VARIABLES.md @@ -217,7 +217,7 @@ These variables are set automatically by the generator. | `use_qdrant` | bool | `false` | Qdrant vector database is selected | Computed from `vector_store` | | `use_chromadb` | bool | `false` | ChromaDB vector database is selected (embedded mode) | Computed from `vector_store` | | `use_pgvector` | bool | `false` | pgvector (PostgreSQL extension) is selected | Computed from `vector_store`, requires PostgreSQL | -| `embedding_provider` | enum | auto-derived | Embedding model provider. Auto-derived from LLM provider: OpenAI→openai, Anthropic→voyage, OpenRouter→sentence_transformers | Auto-derived from `llm_provider` | +| `embedding_provider` | enum | auto-derived | Embedding model provider. Auto-derived from LLM provider: OpenAI→openai, Anthropic→voyage, Google→gemini, OpenRouter→openai, OrcaRouter→sentence_transformers | Auto-derived from `llm_provider` | | `use_openai_embeddings` | bool | `false` | OpenAI embeddings are selected | Computed from `llm_provider` | | `use_voyage_embeddings` | bool | `false` | Voyage AI embeddings are selected | Computed from `llm_provider` | | `use_gemini_embeddings` | bool | `false` | Google Gemini multimodal embeddings are selected | Computed from `llm_provider` | @@ -238,7 +238,7 @@ These variables are set automatically by the generator. **Notes:** - RAG requires a vector database (Milvus, Qdrant, ChromaDB, or pgvector) -- Embedding provider is auto-derived from LLM provider (OpenAI→openai, Anthropic→voyage, Google→gemini, OpenRouter→sentence_transformers) +- Embedding provider is auto-derived from LLM provider (OpenAI→openai, Anthropic→voyage, Google→gemini, OpenRouter→openai, OrcaRouter→sentence_transformers) - Reranker is enabled via `--reranker` CLI flag (cohere, cross_encoder) - Cohere and Cross-Encoder rerankers improve search result relevance - LlamaParse requires an API key; PyMuPDF is free and local (with tables, OCR fallback) @@ -275,11 +275,12 @@ These variables are set automatically by the generator. | `use_deepagents` | bool | `false` | DeepAgents (agentic coding, LangChain) is selected | Computed from `ai_framework` | | `use_pydantic_deep` | bool | `false` | PydanticDeep (deep agentic coding, Docker sandbox) is selected | Computed from `ai_framework` | | `sandbox_backend` | enum | `"state"` | Agent sandbox environment for DeepAgents/PydanticDeep. Values: `state`, `daytona` | Only used when `use_deepagents` or `use_pydantic_deep` is true | -| `llm_provider` | enum | `"openai"` | LLM provider. Values: `openai`, `anthropic`, `google`, `openrouter` | - | +| `llm_provider` | enum | `"openai"` | LLM provider. Values: `openai`, `anthropic`, `google`, `openrouter`, `orcarouter` | - | | `use_openai` | bool | `true` | OpenAI is selected | Computed from `llm_provider` | | `use_anthropic` | bool | `false` | Anthropic is selected | Computed from `llm_provider` | | `use_google` | bool | `false` | Google Gemini is selected | Computed from `llm_provider` | | `use_openrouter` | bool | `false` | OpenRouter is selected | Computed from `llm_provider` | +| `use_orcarouter` | bool | `false` | OrcaRouter is selected | Computed from `llm_provider` | | `enable_langsmith` | bool | `false` | Enable LangSmith observability (tracing, prompt management) | Requires LangChain, LangGraph, or DeepAgents | | `enable_web_search` | bool | `false` | Web search. PydanticAI/PydanticDeep use the model-native WebSearch capability; LangChain/LangGraph/DeepAgents use a Tavily-backed tool (needs `TAVILY_API_KEY`) | Requires an AI framework | | `enable_web_fetch` | bool | `false` | Web fetch. PydanticAI/PydanticDeep use the model-native WebFetch capability; LangChain/LangGraph/DeepAgents use the portable `fetch_url` tool | Requires an AI framework | @@ -298,7 +299,7 @@ These variables are set automatically by the generator. - PydanticAI uses `iter()` for full event streaming over WebSocket - LangGraph implements a ReAct (Reasoning + Acting) agent pattern with graph-based architecture - DeepAgents provides an agentic coding assistant with built-in filesystem tools (ls, read_file, write_file, edit_file, glob, grep) and task management -- OpenRouter with LangChain, LangGraph, or DeepAgents is not supported +- OpenRouter and OrcaRouter with LangChain, LangGraph, or DeepAgents is not supported --- diff --git a/template/cookiecutter.json b/template/cookiecutter.json index fed55d96..5e45c587 100644 --- a/template/cookiecutter.json +++ b/template/cookiecutter.json @@ -81,6 +81,7 @@ "use_anthropic": false, "use_google": false, "use_openrouter": false, + "use_orcarouter": false, "use_all_providers": false, "use_ai": true, "enable_conversation_persistence": true, diff --git a/template/{{cookiecutter.project_slug}}/ENV_VARS.md b/template/{{cookiecutter.project_slug}}/ENV_VARS.md index a863f2ae..f80b049a 100644 --- a/template/{{cookiecutter.project_slug}}/ENV_VARS.md +++ b/template/{{cookiecutter.project_slug}}/ENV_VARS.md @@ -54,6 +54,9 @@ group is for and which are required vs optional. {%- if cookiecutter.use_openrouter %} | `OPENROUTER_API_KEY` | **required** | — | From openrouter.ai | {%- endif %} +{%- if cookiecutter.use_orcarouter %} +| `ORCAROUTER_API_KEY` | **required** | — | From orcarouter.ai | +{%- endif %} {%- if cookiecutter.enable_logfire %} | `LOGFIRE_TOKEN` | optional | — | When set, ships traces to Logfire (logfire.pydantic.dev) | {%- endif %} diff --git a/template/{{cookiecutter.project_slug}}/MANUAL_STEPS.md b/template/{{cookiecutter.project_slug}}/MANUAL_STEPS.md index 547576dc..86ce29d0 100644 --- a/template/{{cookiecutter.project_slug}}/MANUAL_STEPS.md +++ b/template/{{cookiecutter.project_slug}}/MANUAL_STEPS.md @@ -60,6 +60,14 @@ These are used to sign JWTs and authenticate service-to-service calls. Rotate at - [ ] Set `OPENROUTER_API_KEY` in `.env`. {%- endif %} +{%- if cookiecutter.use_orcarouter %} + +## OrcaRouter + +- [ ] Create API key at https://www.orcarouter.ai. +- [ ] Set `ORCAROUTER_API_KEY` in `.env`. +{%- endif %} + {%- if cookiecutter.enable_oauth_google %} ## Google OAuth diff --git a/template/{{cookiecutter.project_slug}}/backend/.env.example b/template/{{cookiecutter.project_slug}}/backend/.env.example index 99678cb1..ebd6a8be 100644 --- a/template/{{cookiecutter.project_slug}}/backend/.env.example +++ b/template/{{cookiecutter.project_slug}}/backend/.env.example @@ -208,9 +208,12 @@ GOOGLE_API_KEY= {%- if cookiecutter.use_openrouter %} OPENROUTER_API_KEY= {%- endif %} +{%- if cookiecutter.use_orcarouter %} +ORCAROUTER_API_KEY= +{%- endif %} {%- if cookiecutter.use_all_providers %} # Multi-provider deployment — model name must include the provider prefix: -# openai/, anthropic/, google/, openrouter// +# openai/, anthropic/, google/, openrouter//, orcarouter// AI_MODEL=openai/gpt-5.5 {%- elif cookiecutter.use_openai %} AI_MODEL=gpt-5.5 @@ -220,6 +223,8 @@ AI_MODEL=claude-opus-4-7 AI_MODEL=gemini-2.5-flash {%- elif cookiecutter.use_openrouter %} AI_MODEL=anthropic/claude-opus-4-7 +{%- elif cookiecutter.use_orcarouter %} +AI_MODEL=anthropic/claude-sonnet-4.6 {%- endif %} AI_TEMPERATURE=0.7 {%- if cookiecutter.use_pydantic_ai %} diff --git a/template/{{cookiecutter.project_slug}}/backend/app/agents/assistant.py b/template/{{cookiecutter.project_slug}}/backend/app/agents/assistant.py index e0e6e9c9..85ec4133 100644 --- a/template/{{cookiecutter.project_slug}}/backend/app/agents/assistant.py +++ b/template/{{cookiecutter.project_slug}}/backend/app/agents/assistant.py @@ -39,6 +39,10 @@ from pydantic_ai.models.openrouter import OpenRouterModel from pydantic_ai.providers.openrouter import OpenRouterProvider {%- endif %} +{%- if cookiecutter.use_orcarouter and not cookiecutter.use_openai %} +from pydantic_ai.models.openai import OpenAIResponsesModel +from pydantic_ai.providers.openai import OpenAIProvider +{%- endif %} from pydantic_ai.settings import ModelSettings from app.agents.prompts import DEFAULT_SYSTEM_PROMPT @@ -89,6 +93,7 @@ def _build_model(model_name: str) -> "OpenAIResponsesModel | AnthropicModel | Go - anthropic/claude-* → Anthropic - google/gemini-* → Google - openrouter// → OpenRouter + - orcarouter// → OrcaRouter (OpenAI-compatible) - bare names (no slash) → fall back to OpenAI for backwards compat. """ name = model_name or settings.AI_MODEL @@ -109,6 +114,14 @@ def _build_model(model_name: str) -> "OpenAIResponsesModel | AnthropicModel | Go return OpenRouterModel( rest, provider=OpenRouterProvider(api_key=settings.OPENROUTER_API_KEY) ) + if prefix == "orcarouter": + return OpenAIResponsesModel( + rest, + provider=OpenAIProvider( + api_key=settings.ORCAROUTER_API_KEY, + base_url="https://api.orcarouter.ai/v1", + ), + ) # Bare model name — best-effort sniff by family. if lowered.startswith(("claude-", "claude/")): return AnthropicModel(name.removeprefix("claude/")) @@ -147,6 +160,17 @@ def _build_model(model_name: str) -> OpenRouterModel: model_name or settings.AI_MODEL, provider=OpenRouterProvider(api_key=settings.OPENROUTER_API_KEY), ) +{%- elif cookiecutter.use_orcarouter %} + + +def _build_model(model_name: str) -> OpenAIResponsesModel: + return OpenAIResponsesModel( + model_name or settings.AI_MODEL, + provider=OpenAIProvider( + api_key=settings.ORCAROUTER_API_KEY, + base_url="https://api.orcarouter.ai/v1", + ), + ) {%- endif %} diff --git a/template/{{cookiecutter.project_slug}}/backend/app/agents/pydantic_deep_assistant.py b/template/{{cookiecutter.project_slug}}/backend/app/agents/pydantic_deep_assistant.py index f456698f..92cc3a6f 100644 --- a/template/{{cookiecutter.project_slug}}/backend/app/agents/pydantic_deep_assistant.py +++ b/template/{{cookiecutter.project_slug}}/backend/app/agents/pydantic_deep_assistant.py @@ -56,6 +56,7 @@ "anthropic": "anthropic", "google": "google-gla", # Google AI (Gemini) via GOOGLE_API_KEY "openrouter": "openrouter", + "orcarouter": "openai-responses", } @@ -181,7 +182,26 @@ def _get_system_prompt(self) -> str: def _build_agent_and_deps(self) -> tuple[Agent[DeepAgentDeps, str], DeepAgentDeps]: """Instantiate the pydantic-deep agent and its dependencies.""" backend = self._backend_override if self._backend_override is not None else self._create_backend() - model_str = self._get_model_string() +{%- if cookiecutter.use_orcarouter %} + # OrcaRouter is OpenAI-compatible with a custom base URL, so build the + # model object directly — a model string would resolve to OPENAI_API_KEY + # and api.openai.com instead of the OrcaRouter endpoint. + from pydantic_ai.models.openai import OpenAIResponsesModel + from pydantic_ai.providers.openai import OpenAIProvider + + if settings.LLM_PROVIDER == "orcarouter" or self.model_name.startswith("orcarouter/"): + model = OpenAIResponsesModel( + self.model_name.removeprefix("orcarouter/"), + provider=OpenAIProvider( + api_key=settings.ORCAROUTER_API_KEY, + base_url="https://api.orcarouter.ai/v1", + ), + ) + else: + model = self._get_model_string() +{%- else %} + model = self._get_model_string() +{%- endif %} history_path = ( self._history_messages_path if self._history_messages_path is not None @@ -190,7 +210,7 @@ def _build_agent_and_deps(self) -> tuple[Agent[DeepAgentDeps, str], DeepAgentDep logger.info( "Creating PydanticDeep agent — model=%s backend=%s conversation=%s", - model_str, + model, type(backend).__name__, self.conversation_id, ) @@ -208,7 +228,7 @@ def _build_agent_and_deps(self) -> tuple[Agent[DeepAgentDeps, str], DeepAgentDep {%- endif %} agent = create_deep_agent( - model=model_str, + model=model, backend=backend, instructions=self._get_system_prompt(), # Per-conversation history persistence diff --git a/template/{{cookiecutter.project_slug}}/backend/app/api/routes/v1/health.py b/template/{{cookiecutter.project_slug}}/backend/app/api/routes/v1/health.py index f82aa8eb..d908eb15 100644 --- a/template/{{cookiecutter.project_slug}}/backend/app/api/routes/v1/health.py +++ b/template/{{cookiecutter.project_slug}}/backend/app/api/routes/v1/health.py @@ -185,6 +185,7 @@ async def readiness_probe( "anthropic": "ANTHROPIC_API_KEY", "google": "GOOGLE_API_KEY", "openrouter": "OPENROUTER_API_KEY", + "orcarouter": "ORCAROUTER_API_KEY", }.get(llm_provider) api_key = getattr(settings, key_field, None) if key_field else None checks["llm"] = { diff --git a/template/{{cookiecutter.project_slug}}/backend/app/core/config.py b/template/{{cookiecutter.project_slug}}/backend/app/core/config.py index fc8c925e..c280497c 100644 --- a/template/{{cookiecutter.project_slug}}/backend/app/core/config.py +++ b/template/{{cookiecutter.project_slug}}/backend/app/core/config.py @@ -310,10 +310,14 @@ def REDIS_URL(self) -> str: {%- if cookiecutter.use_openrouter %} OPENROUTER_API_KEY: str = "" {%- endif %} +{%- if cookiecutter.use_orcarouter %} + ORCAROUTER_API_KEY: str = "" +{%- endif %} {%- if cookiecutter.use_all_providers %} # Multi-provider: model can come from any installed SDK. Prefix with the # provider name (`openai/gpt-5.5`, `anthropic/claude-opus-4-7`, - # `google/gemini-2.5-flash`, `openrouter/anthropic/claude-opus-4-7`) + # `google/gemini-2.5-flash`, `openrouter/anthropic/claude-opus-4-7`, + # `orcarouter/anthropic/claude-sonnet-4.6`) # so the dispatcher in agents/assistant.py routes to the right backend. AI_MODEL: str = "openai/gpt-5.5" {%- elif cookiecutter.use_openai %} @@ -324,6 +328,8 @@ def REDIS_URL(self) -> str: AI_MODEL: str = "gemini-2.5-flash" {%- elif cookiecutter.use_openrouter %} AI_MODEL: str = "anthropic/claude-opus-4-7" +{%- elif cookiecutter.use_orcarouter %} + AI_MODEL: str = "anthropic/claude-sonnet-4.6" {%- endif %} AI_TEMPERATURE: float = 0.7 AI_THINKING_ENABLED: bool = False @@ -343,6 +349,9 @@ def REDIS_URL(self) -> str: # OpenRouter (proxies many providers) "openrouter/anthropic/claude-opus-4-7", "openrouter/deepseek/deepseek-r1", + # OrcaRouter (model routing, OpenAI-compatible) + "orcarouter/anthropic/claude-sonnet-4.6", + "orcarouter/openai/gpt-5.5", ] {%- elif cookiecutter.use_openai %} AI_AVAILABLE_MODELS: list[str] = [ @@ -384,6 +393,15 @@ def REDIS_URL(self) -> str: "google/gemini-2.5-flash", "deepseek/deepseek-r1", ] +{%- elif cookiecutter.use_orcarouter %} + AI_AVAILABLE_MODELS: list[str] = [ + "anthropic/claude-sonnet-4.6", + "anthropic/claude-opus-4.7", + "openai/gpt-5.5", + "openai/gpt-5-mini", + "google/gemini-2.5-flash", + "deepseek/deepseek-v4-pro", + ] {%- endif %} AI_FRAMEWORK: str = "{{ cookiecutter.ai_framework }}" LLM_PROVIDER: str = "{{ cookiecutter.llm_provider }}" diff --git a/template/{{cookiecutter.project_slug}}/backend/pyproject.toml b/template/{{cookiecutter.project_slug}}/backend/pyproject.toml index 5a81586f..e711b957 100644 --- a/template/{{cookiecutter.project_slug}}/backend/pyproject.toml +++ b/template/{{cookiecutter.project_slug}}/backend/pyproject.toml @@ -123,6 +123,8 @@ dependencies = [ "pydantic-ai-slim[google,duckduckgo,web-fetch]>=1.80.0", {%- elif cookiecutter.use_openrouter %} "pydantic-ai-slim[openrouter,duckduckgo,web-fetch]>=1.80.0", +{%- elif cookiecutter.use_orcarouter %} + "pydantic-ai-slim[openai,duckduckgo,web-fetch]>=1.80.0", {%- endif %} {%- if cookiecutter.enable_mcp_client %} # MCP client transports + toolset support (Settings → Integrations). diff --git a/template/{{cookiecutter.project_slug}}/backend/tests/test_agents.py b/template/{{cookiecutter.project_slug}}/backend/tests/test_agents.py index 4d162d79..d3622071 100644 --- a/template/{{cookiecutter.project_slug}}/backend/tests/test_agents.py +++ b/template/{{cookiecutter.project_slug}}/backend/tests/test_agents.py @@ -70,7 +70,7 @@ def test_init_with_custom_values(self): # ``_build_model`` is the single per-provider model factory in # assistant.py, so patching it keeps these tests provider-agnostic - # (openai/anthropic/google/openrouter/all) and avoids needing real API keys. + # (openai/anthropic/google/openrouter/orcarouter/all) and avoids needing real API keys. @patch("app.agents.assistant._build_model") def test_agent_property_creates_agent(self, mock_build_model): """Test agent property creates agent on first access.""" diff --git a/template/{{cookiecutter.project_slug}}/docs/configuration.md b/template/{{cookiecutter.project_slug}}/docs/configuration.md index e172af23..ff17193b 100644 --- a/template/{{cookiecutter.project_slug}}/docs/configuration.md +++ b/template/{{cookiecutter.project_slug}}/docs/configuration.md @@ -133,6 +133,10 @@ Computed properties: | `OPENROUTER_API_KEY` | (empty) | OpenRouter API key | | `AI_MODEL` | `anthropic/claude-opus-4-7` | Default LLM model for chat | {%- endif %} +{%- if cookiecutter.use_orcarouter %} +| `ORCAROUTER_API_KEY` | (empty) | OrcaRouter API key | +| `AI_MODEL` | `anthropic/claude-sonnet-4.6` | Default LLM model for chat | +{%- endif %} | `AI_TEMPERATURE` | `0.7` | LLM temperature (0.0 = deterministic, 1.0 = creative) | | `AI_AVAILABLE_MODELS` | (auto-configured) | JSON list of models shown in the UI model selector | | `AI_FRAMEWORK` | `{{ cookiecutter.ai_framework }}` | AI framework (informational) | @@ -489,3 +493,6 @@ Before deploying to production, ensure these variables are properly set: {%- if cookiecutter.use_openrouter %} 8. `OPENROUTER_API_KEY` -- Your production API key {%- endif %} +{%- if cookiecutter.use_orcarouter %} +8. `ORCAROUTER_API_KEY` -- Your production API key +{%- endif %} diff --git a/template/{{cookiecutter.project_slug}}/kubernetes/secret.yaml b/template/{{cookiecutter.project_slug}}/kubernetes/secret.yaml index 0811df30..bfc1d90d 100644 --- a/template/{{cookiecutter.project_slug}}/kubernetes/secret.yaml +++ b/template/{{cookiecutter.project_slug}}/kubernetes/secret.yaml @@ -46,6 +46,9 @@ stringData: {%- if cookiecutter.use_openrouter %} OPENROUTER_API_KEY: "" {%- endif %} +{%- if cookiecutter.use_orcarouter %} + ORCAROUTER_API_KEY: "" +{%- endif %} {%- if cookiecutter.enable_oauth_google %} # OAuth diff --git a/tests/test_config.py b/tests/test_config.py index e5087ced..282ed755 100644 --- a/tests/test_config.py +++ b/tests/test_config.py @@ -512,7 +512,7 @@ def test_openrouter_with_langchain_raises_validation_error(self) -> None: ai_framework=AIFrameworkType.LANGCHAIN, background_tasks=BackgroundTaskType.NONE, ) - assert "OpenRouter is only supported with PydanticAI" in str(exc_info.value) + assert "OpenRouter/OrcaRouter is only supported with PydanticAI or PydanticDeep" in str(exc_info.value) def test_openrouter_with_langgraph_raises_validation_error(self) -> None: """Test that OpenRouter + LangGraph combination is rejected.""" @@ -523,7 +523,7 @@ def test_openrouter_with_langgraph_raises_validation_error(self) -> None: ai_framework=AIFrameworkType.LANGGRAPH, background_tasks=BackgroundTaskType.NONE, ) - assert "OpenRouter is only supported with PydanticAI" in str(exc_info.value) + assert "OpenRouter/OrcaRouter is only supported with PydanticAI or PydanticDeep" in str(exc_info.value) def test_openrouter_with_pydanticai_is_valid(self) -> None: """Test that OpenRouter + PydanticAI combination is accepted.""" @@ -545,7 +545,7 @@ def test_openrouter_with_deepagents_raises_validation_error(self) -> None: ai_framework=AIFrameworkType.DEEPAGENTS, background_tasks=BackgroundTaskType.NONE, ) - assert "OpenRouter is only supported with PydanticAI" in str(exc_info.value) + assert "OpenRouter/OrcaRouter is only supported with PydanticAI or PydanticDeep" in str(exc_info.value) def test_deepagents_with_openai_is_valid(self) -> None: """Test that DeepAgents + OpenAI combination is accepted.""" @@ -609,6 +609,74 @@ def test_openrouter_with_pydantic_deep_is_valid(self) -> None: assert config.llm_provider == LLMProviderType.OPENROUTER assert config.ai_framework == AIFrameworkType.PYDANTIC_DEEP + def test_orcarouter_with_langchain_raises_validation_error(self) -> None: + """Test that OrcaRouter + LangChain combination is rejected.""" + with pytest.raises(ValidationError) as exc_info: + ProjectConfig( + project_name="test", + llm_provider=LLMProviderType.ORCAROUTER, + ai_framework=AIFrameworkType.LANGCHAIN, + background_tasks=BackgroundTaskType.NONE, + ) + assert "OpenRouter/OrcaRouter is only supported with PydanticAI or PydanticDeep" in str(exc_info.value) + + def test_orcarouter_with_langgraph_raises_validation_error(self) -> None: + """Test that OrcaRouter + LangGraph combination is rejected.""" + with pytest.raises(ValidationError) as exc_info: + ProjectConfig( + project_name="test", + llm_provider=LLMProviderType.ORCAROUTER, + ai_framework=AIFrameworkType.LANGGRAPH, + background_tasks=BackgroundTaskType.NONE, + ) + assert "OpenRouter/OrcaRouter is only supported with PydanticAI or PydanticDeep" in str(exc_info.value) + + def test_orcarouter_with_deepagents_raises_validation_error(self) -> None: + """Test that OrcaRouter + DeepAgents combination is rejected.""" + with pytest.raises(ValidationError) as exc_info: + ProjectConfig( + project_name="test", + llm_provider=LLMProviderType.ORCAROUTER, + ai_framework=AIFrameworkType.DEEPAGENTS, + background_tasks=BackgroundTaskType.NONE, + ) + assert "OpenRouter/OrcaRouter is only supported with PydanticAI or PydanticDeep" in str(exc_info.value) + + def test_orcarouter_with_pydanticai_is_valid(self) -> None: + """Test that OrcaRouter + PydanticAI combination is accepted.""" + config = ProjectConfig( + project_name="test", + llm_provider=LLMProviderType.ORCAROUTER, + ai_framework=AIFrameworkType.PYDANTIC_AI, + background_tasks=BackgroundTaskType.NONE, + ) + assert config.llm_provider == LLMProviderType.ORCAROUTER + assert config.ai_framework == AIFrameworkType.PYDANTIC_AI + + def test_orcarouter_with_pydantic_deep_is_valid(self) -> None: + """Test that OrcaRouter + PydanticDeep combination is accepted.""" + config = ProjectConfig( + project_name="test", + llm_provider=LLMProviderType.ORCAROUTER, + ai_framework=AIFrameworkType.PYDANTIC_DEEP, + background_tasks=BackgroundTaskType.NONE, + ) + assert config.llm_provider == LLMProviderType.ORCAROUTER + assert config.ai_framework == AIFrameworkType.PYDANTIC_DEEP + + def test_orcarouter_context_flags(self) -> None: + """Test that OrcaRouter sets correct context flags.""" + config = ProjectConfig( + project_name="test", + llm_provider=LLMProviderType.ORCAROUTER, + ai_framework=AIFrameworkType.PYDANTIC_AI, + background_tasks=BackgroundTaskType.NONE, + ) + context = config.to_cookiecutter_context() + + assert context["use_orcarouter"] is True + assert context["use_openrouter"] is False + def test_langsmith_with_pydantic_deep_raises_error(self) -> None: """Test that LangSmith + PydanticDeep is rejected (pydantic-deep uses Logfire).""" with pytest.raises(ValidationError) as exc_info: diff --git a/tests/test_prompts.py b/tests/test_prompts.py index b6a2a106..d27fd9ef 100644 --- a/tests/test_prompts.py +++ b/tests/test_prompts.py @@ -901,14 +901,41 @@ def test_openrouter_option_added_for_pydanticai(self, mock_questionary: MagicMoc prompt_llm_provider(AIFrameworkType.PYDANTIC_AI) - # Check that select was called with 4 choices (OpenAI, Anthropic, Google, OpenRouter) + # Check that select was called with 5 choices (OpenAI, Anthropic, Google, OpenRouter, OrcaRouter) select_call = mock_questionary.select.call_args choices = select_call[1]["choices"] - assert len(choices) == 4 + assert len(choices) == 5 @patch("fastapi_gen.prompts.questionary") - def test_openrouter_option_not_added_for_langchain(self, mock_questionary: MagicMock) -> None: - """Test OpenRouter option is NOT added when using LangChain.""" + def test_returns_orcarouter_for_pydanticai(self, mock_questionary: MagicMock) -> None: + """Test OrcaRouter provider is returned for PydanticAI.""" + mock_select = MagicMock() + mock_select.ask.return_value = LLMProviderType.ORCAROUTER + mock_questionary.select.return_value = mock_select + mock_questionary.Choice = MagicMock() + + result = prompt_llm_provider(AIFrameworkType.PYDANTIC_AI) + + assert result == LLMProviderType.ORCAROUTER + + @patch("fastapi_gen.prompts.questionary") + def test_orcarouter_option_added_for_pydanticai(self, mock_questionary: MagicMock) -> None: + """Test OrcaRouter option is added when using PydanticAI.""" + mock_select = MagicMock() + mock_select.ask.return_value = LLMProviderType.OPENAI + mock_questionary.select.return_value = mock_select + mock_questionary.Choice = MagicMock() + + prompt_llm_provider(AIFrameworkType.PYDANTIC_AI) + + # Check that select was called with 5 choices (OpenAI, Anthropic, Google, OpenRouter, OrcaRouter) + select_call = mock_questionary.select.call_args + choices = select_call[1]["choices"] + assert len(choices) == 5 + + @patch("fastapi_gen.prompts.questionary") + def test_orcarouter_option_not_added_for_langchain(self, mock_questionary: MagicMock) -> None: + """Test OrcaRouter option is NOT added when using LangChain.""" mock_select = MagicMock() mock_select.ask.return_value = LLMProviderType.OPENAI mock_questionary.select.return_value = mock_select diff --git a/tests/test_rag_config.py b/tests/test_rag_config.py index 93e91b30..296f82af 100644 --- a/tests/test_rag_config.py +++ b/tests/test_rag_config.py @@ -101,6 +101,21 @@ def test_openrouter_derives_openai_embeddings(self) -> None: == EmbeddingProviderType.OPENAI ) + def test_orcarouter_derives_sentence_transformers_embeddings(self) -> None: + """Test that OrcaRouter LLM provider derives SentenceTransformers embeddings.""" + config = ProjectConfig( + project_name="test", + llm_provider=LLMProviderType.ORCAROUTER, + rag_features=RAGFeatures(enable_rag=True), + background_tasks=BackgroundTaskType.CELERY, + enable_redis=True, + enable_docker=True, + ) + assert ( + config.to_cookiecutter_context()["embedding_provider"] + == EmbeddingProviderType.SENTENCE_TRANSFORMERS + ) + def test_openai_derives_openai_embeddings(self) -> None: """Test that OpenAI LLM provider derives OpenAI embeddings.""" config = ProjectConfig( @@ -225,6 +240,22 @@ def test_openrouter_derives_openai_embeddings_context_flags(self) -> None: assert context["use_openai_embeddings"] is True assert context["use_voyage_embeddings"] is False + def test_orcarouter_derives_sentence_transformers_embeddings_context_flags(self) -> None: + """Test OrcaRouter LLM provider derives SentenceTransformers (not OpenAI).""" + config = ProjectConfig( + project_name="test", + llm_provider=LLMProviderType.ORCAROUTER, + rag_features=RAGFeatures(enable_rag=True), + background_tasks=BackgroundTaskType.CELERY, + enable_redis=True, + enable_docker=True, + ) + context = config.to_cookiecutter_context() + + assert context["use_sentence_transformers"] is True + assert context["use_openai_embeddings"] is False + assert context["use_voyage_embeddings"] is False + def test_reranker_enabled_context_flags(self) -> None: """Test reranker enabled sets correct context flags.""" config = ProjectConfig( diff --git a/tests/test_rag_integration.py b/tests/test_rag_integration.py index dc2ee03f..3b77b633 100644 --- a/tests/test_rag_integration.py +++ b/tests/test_rag_integration.py @@ -301,14 +301,32 @@ def test_rag_with_voyage_embeddings(self, tmp_path) -> None: content = rag_config.read_text() assert "voyage" in content.lower() - def test_rag_with_sentence_transformers(self, tmp_path) -> None: - """Test RAG with SentenceTransformers (auto-derived from OpenRouter).""" + def test_rag_with_openrouter_openai_embeddings(self, tmp_path) -> None: + """Test RAG with OpenAI-compatible embeddings (auto-derived from OpenRouter).""" config = ProjectConfig( - project_name="rag_st_emb", + project_name="rag_or_emb", database=DatabaseType.POSTGRESQL, background_tasks=BackgroundTaskType.CELERY, enable_redis=True, - llm_provider=LLMProviderType.OPENROUTER, # Derives SentenceTransformers + llm_provider=LLMProviderType.OPENROUTER, # Derives OpenAI-compatible embeddings + rag_features=RAGFeatures(enable_rag=True), + enable_docker=True, + ) + project = generate_project(config, tmp_path) + + # Verify config uses OpenAI-compatible embeddings + rag_config = project / "backend" / "app" / "services" / "rag" / "config.py" + content = rag_config.read_text() + assert "text-embedding" in content.lower() + + def test_rag_with_orcarouter_sentence_transformers(self, tmp_path) -> None: + """Test RAG with SentenceTransformers (auto-derived from OrcaRouter).""" + config = ProjectConfig( + project_name="rag_orca_st", + database=DatabaseType.POSTGRESQL, + background_tasks=BackgroundTaskType.CELERY, + enable_redis=True, + llm_provider=LLMProviderType.ORCAROUTER, # Derives SentenceTransformers rag_features=RAGFeatures(enable_rag=True), enable_docker=True, ) diff --git a/tests/test_template_integration.py b/tests/test_template_integration.py index 1e08efe4..c32aad51 100644 --- a/tests/test_template_integration.py +++ b/tests/test_template_integration.py @@ -238,6 +238,12 @@ def test_full_project_valid_python_syntax(self, generated_project_full: Path) -> llm_provider=LLMProviderType.OPENROUTER, background_tasks=BackgroundTaskType.NONE, ), + "orcarouter": dict( + database=DatabaseType.POSTGRESQL, + ai_framework=AIFrameworkType.PYDANTIC_AI, + llm_provider=LLMProviderType.ORCAROUTER, + background_tasks=BackgroundTaskType.NONE, + ), "anthropic": dict( database=DatabaseType.POSTGRESQL, ai_framework=AIFrameworkType.PYDANTIC_AI,