refactor(inference): remove managed inference routes (#3195)

* refactor(inference): remove managed inference routes

Closes #3172

Remove the inference route control plane, inference.local data path, built-in router crate, and SDK surface. Move inference workloads to explicitly imported provider profiles and native endpoints, with migration cleanup and updated tests and documentation.

Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>

* fix(policy): preserve alternate upstream isolation

Restore the provider policy activation guard so legacy OpenAI and Anthropic providers configured for alternate base URLs do not grant egress to the built-in public vendor endpoints.

Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>

---------

Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>

Author: johntmyers

.agents/skills/sync-agent-infra/SKILL.md

@@ -45,7 +45,7 @@ Use this map when product behavior, commands, or development workflows change. I
 | Sandbox policy schema, presets, or enforcement behavior | `generate-sandbox-policy`, `openshell-cli` |
 | Supervisor middleware policy, registrations, runtime, or failure behavior | `generate-sandbox-policy`, `openshell-cli`, `debug-openshell-cluster` |
 | Gateway deployment, Helm, runtime drivers, or health checks | `debug-openshell-cluster`, `helm-dev-environment` |
-| Inference routing, providers, or `inference.local` behavior | `debug-inference`, `openshell-cli` |
+| Inference providers, native model endpoints, or migration from `inference.local` | `debug-inference`, `openshell-cli`, `generate-sandbox-policy` |
 | TUI architecture, navigation, data fetching, or UX | `tui-development` |
 | Release artifacts or post-publish smoke coverage | `test-release-canary` |
 | GitHub Actions workflows, required checks, or CI diagnostics | `watch-github-actions`; also `test-release-canary` for release smoke coverage |

AGENTS.md

@@ -39,8 +39,7 @@ These pipelines connect skills into end-to-end workflows. Individual skill files
 | `crates/openshell-conformance-cli/` | Conformance CLI | Distributable `list` and `run` entrypoint for gateway conformance |
 | `crates/openshell-server/` | Gateway server | Control-plane API, sandbox lifecycle, auth boundary |
 | `crates/openshell-sandbox/` | Sandbox runtime | Container supervision, policy-enforced egress routing |
-| `crates/openshell-policy/` | Policy engine | Filesystem, network, process, and inference constraints |
-| `crates/openshell-router/` | Privacy router | Privacy-aware LLM routing |
+| `crates/openshell-policy/` | Policy engine | Filesystem, network, and process constraints |
 | `crates/openshell-bootstrap/` | Gateway metadata | Gateway registration metadata, auth token storage, mTLS bundle storage |
 | `crates/openshell-gateway-interceptors/` | Gateway interceptors | Intercepts and transforms configured gRPC requests at the gateway routing boundary |
 | `crates/openshell-ocsf/` | OCSF logging | OCSF v1.8.0 event types, builders, shorthand/JSONL formatters, tracing layers |
@@ -57,15 +56,14 @@ These pipelines connect skills into end-to-end workflows. Individual skill files
 | `crates/openshell-driver-db-credstore/` | Database credential driver | In-process `CredentialDriver` backend for gateway database credential storage |
 | `crates/openshell-driver-kubernetes/` | Kubernetes compute driver | In-process `ComputeDriver` backend for K8s sandbox pods |
 | `crates/openshell-driver-docker/` | Docker compute driver | In-process `ComputeDriver` backend for local Docker sandbox containers |
-| `crates/openshell-driver-mxc/` | MXC compute driver | Windows in-process `ComputeDriver` backend for MXC sandbox execution |
 | `crates/openshell-driver-podman/` | Podman compute driver | In-process `ComputeDriver` backend for local Podman sandbox containers |
 | `crates/openshell-driver-vm/` | VM compute driver | Standalone libkrun-backed `ComputeDriver` subprocess (embeds its own rootfs + runtime) |
 | `crates/openshell-driver-mxc/` | Microsoft MXC compute driver | In-process Windows AppContainer and isolation-session compute backend |
 | `crates/openshell-prover/` | Policy prover | Policy verification and proof generation |
 | `crates/openshell-server-macros/` | Server macros | Compile-time helpers for gateway RPC authorization |
 | `crates/openshell-supervisor-middleware/` | Middleware runtime | Generic middleware registry, remote service integration, and chain execution |
 | `crates/openshell-supervisor-middleware-builtins/` | Built-in middleware | First-party in-process middleware implementations |
-| `crates/openshell-supervisor-network/` | Network supervisor | Proxying, L7 enforcement, policy evaluation, and inference routing |
+| `crates/openshell-supervisor-network/` | Network supervisor | Proxying, L7 enforcement, policy evaluation, and provider credential injection |
 | `crates/openshell-supervisor-process/` | Process supervisor | Process lifecycle, namespace, and bypass monitoring |
 | `crates/openshell-vfio/` | VFIO support | PCI and GPU passthrough preparation and lifecycle |
 | `python/openshell/` | Python SDK | Python bindings and CLI packaging |
@@ -116,7 +114,7 @@ Use an OCSF builder + `ocsf_emit!()` for events that represent **observable sand
 - SSH authentication (accepted, denied, nonce replay)
 - Process lifecycle (start, exit, timeout, signal failure)
 - Security findings (unsafe policy, unavailable controls, replay attacks)
-- Configuration changes (policy load/reload, TLS setup, inference routes, settings)
+- Configuration changes (policy load/reload, TLS setup, provider attachments, settings)
 - Application lifecycle (supervisor start, SSH server ready)
 
 ### When to use plain tracing
@@ -138,7 +136,7 @@ Use `info!()`, `debug!()`, `warn!()` for **internal operational plumbing** that
 | SSH sessions | `SshActivityBuilder` | Authentication, channel operations |
 | Process start/stop | `ProcessActivityBuilder` | Entrypoint lifecycle, signal failures |
 | Security alerts | `DetectionFindingBuilder` | Nonce replay, bypass detection, unsafe policy. Dual-emit with the domain event. |
-| Policy/config changes | `ConfigStateChangeBuilder` | Policy load, Landlock apply, TLS setup, inference routes, settings |
+| Policy/config changes | `ConfigStateChangeBuilder` | Policy load, Landlock apply, TLS setup, provider attachments, settings |
 | Supervisor lifecycle | `AppLifecycleBuilder` | Sandbox start, SSH server ready/failed |
 
 ### Severity guidelines

CONTRIBUTING.md

@@ -80,7 +80,7 @@ Public skills live in `skills/` and work without an OpenShell source checkout. I
 | --- | --- |
 | `openshell-cli` | CLI usage, sandbox lifecycle, provider management, and BYOC workflows |
 | `debug-openshell-cluster` | Diagnose gateway deployment and health issues |
-| `debug-inference` | Diagnose managed, system, local, and direct external inference issues |
+| `debug-inference` | Diagnose attached-provider inference, native endpoints, and migration from `inference.local` |
 | `generate-sandbox-policy` | Generate YAML sandbox policies from requirements or API documentation |
 
 Public skills use `openshell --help` for installed command syntax and published OpenShell documentation for product concepts and configuration. They must not depend on repository-relative source or documentation files.

Cargo.lock

@@ -108,15 +108,15 @@ bash examples/sandbox-policy-quickstart/demo.sh
 OpenShell isolates each sandbox in its own container with policy-enforced egress routing. A lightweight gateway coordinates sandbox lifecycle, and every outbound connection is intercepted by the policy engine, which does one of three things:
 
 - **Allows** — the destination and binary match a policy block.
-- **Routes for inference** — strips caller credentials, injects backend credentials, and forwards to the managed model.
+- **Binds credentials to endpoints** — injects provider credentials only after policy admits a request to a profile-authorized endpoint.
 - **Denies** — blocks the request and logs it.
 
 | Component          | Role                                                                                         |
 | ------------------ | -------------------------------------------------------------------------------------------- |
 | **Gateway**        | Control-plane API that coordinates sandbox lifecycle and acts as the auth boundary.          |
 | **Sandbox**        | Isolated runtime with container supervision and policy-enforced egress routing.              |
 | **Policy Engine**  | Enforces filesystem, network, and process constraints from application layer down to kernel. |
-| **Privacy Router** | Privacy-aware LLM routing that keeps sensitive context on sandbox compute.                   |
+| **Provider Access** | Profile-defined endpoints, binary policy, and endpoint-bound credential injection for model APIs and other services. |
 
 OpenShell runs a gateway control plane that manages sandbox lifecycle through a configured compute driver. Supported compute platforms include Docker, Podman, MicroVM, and Kubernetes.
 
@@ -129,14 +129,16 @@ OpenShell applies defense in depth across four policy domains:
 | Filesystem | Prevents reads/writes outside allowed paths.        | Locked at sandbox creation. |
 | Network    | Blocks unauthorized outbound connections.           | Hot-reloadable at runtime.  |
 | Process    | Blocks privilege escalation and dangerous syscalls. | Locked at sandbox creation. |
-| Inference  | Reroutes model API calls to controlled backends.    | Hot-reloadable at runtime.  |
+| Providers  | Grants endpoint-bound credentials and network access. | Hot-reloadable at runtime. |
 
-Policies are declarative YAML files. Static sections (filesystem, process) are locked at creation; dynamic sections (network, inference) can be hot-reloaded on a running sandbox with `openshell policy set`.
+Policies are declarative YAML files. Static sections (filesystem, process) are locked at creation; network policy and provider attachments can be updated on a running sandbox.
 
 ## Providers
 
 Agents need credentials — API keys, tokens, service accounts. OpenShell manages these as **providers**: named credential bundles that are injected into sandboxes at creation. The CLI auto-discovers credentials for recognized agents (Claude, Codex, OpenCode, Copilot) from your shell environment, or you can create providers explicitly with `openshell provider create`. Credentials never leak into the sandbox filesystem; they are injected as environment variables at runtime.
 
+Inference access uses the same provider workflow. Attach an inference-capable provider to a sandbox, call the provider's native endpoint, and select the model in the client. Provider profiles contribute the endpoint policy and bind credential placeholders to the authorized destination.
+
 ## GPU Support (Experimental)
 
 > **Experimental** — GPU passthrough works on supported hosts but is under active development. Expect rough edges and breaking changes.
@@ -159,8 +161,8 @@ Docker-backed GPU sandboxes auto-select CDI when available and otherwise fall ba
 | [OpenCode](https://opencode.ai/)                              | [`base`](https://github.com/NVIDIA/OpenShell-Community/tree/main/sandboxes/base) | Works out of the box. Provider uses `OPENAI_API_KEY` or `OPENROUTER_API_KEY`. |
 | [Codex](https://developers.openai.com/codex)                  | [`base`](https://github.com/NVIDIA/OpenShell-Community/tree/main/sandboxes/base) | Works out of the box. Provider uses `OPENAI_API_KEY`.                         |
 | [GitHub Copilot CLI](https://docs.github.com/en/copilot/github-copilot-in-the-cli) | [`base`](https://github.com/NVIDIA/OpenShell-Community/tree/main/sandboxes/base) | Works out of the box. Provider uses `GITHUB_TOKEN` or `COPILOT_GITHUB_TOKEN`. |
-| [OpenClaw](https://openclaw.ai/)                 | [NemoClaw](https://github.com/NVIDIA/NemoClaw)                                   | Run OpenClaw more securely inside NVIDIA OpenShell with managed inference using NemoClaw.       |
-| [Hermes Agent](https://github.com/NousResearch/hermes-agent)   | [NemoClaw](https://github.com/NVIDIA/NemoClaw)                                   | Run Hermes Agent more securely inside NVIDIA OpenShell with managed inference using NemoClaw.   |
+| [OpenClaw](https://openclaw.ai/)                 | [NemoClaw](https://github.com/NVIDIA/NemoClaw)                                   | Run OpenClaw more securely inside NVIDIA OpenShell with the NemoClaw blueprint.       |
+| [Hermes Agent](https://github.com/NousResearch/hermes-agent)   | [NemoClaw](https://github.com/NVIDIA/NemoClaw)                                   | Run Hermes Agent more securely inside NVIDIA OpenShell with the NemoClaw blueprint.   |
 | [Ollama](https://ollama.com/)                                 | [Community](https://github.com/NVIDIA/OpenShell-Community)                       | Launch with `openshell sandbox create --from ollama`.                         |
 | [Pi](https://pi.dev/)                                 | [Community](https://github.com/NVIDIA/OpenShell-Community)                       | Launch with `openshell sandbox create --from pi`.                         |
 
@@ -172,9 +174,9 @@ Docker-backed GPU sandboxes auto-select CDI when available and otherwise fall ba
 | `openshell sandbox connect [name]`                         | SSH into a running sandbox.                     |
 | `openshell sandbox list`                                   | List all sandboxes.                             |
 | `openshell provider create --type [type] --from-existing`  | Create a credential provider from env vars.     |
+| `openshell sandbox provider attach <sandbox> <provider>`   | Attach a provider to a running sandbox.         |
 | `openshell policy set <name> --policy file.yaml`           | Apply or update a policy on a running sandbox.  |
 | `openshell policy get <name>`                              | Show the active policy.                         |
-| `openshell inference set --provider <p> --model <m>`       | Configure the `inference.local` endpoint.       |
 | `openshell logs [name] --tail`                             | Stream sandbox logs.                            |
 | `openshell term`                                           | Launch the real-time terminal UI for debugging. |
 

README.md

@@ -136,8 +136,6 @@ def test_multiply(sandbox):
 |---|---|---|
 | `sandbox_client` | session | gRPC client connected to the active gateway |
 | `sandbox` | function | Factory returning a `Sandbox` context manager |
-| `inference_client` | session | Client for managing inference routes |
-| `mock_inference_route` | session | Creates a mock OpenAI-protocol route for tests |
 
 ### Rust CLI E2E (`e2e/rust/`)