feat: Infrahub Marketplace integration for the MCP server

[minitriga]

PRD: Infrahub Marketplace integration for the MCP server

Problem Statement

When an AI agent works with Infrahub through the MCP server today, it can inspect and modify the schema of the connected instance, but it has no way to discover schemas that already exist. The Infrahub Marketplace (marketplace.infrahub.app) hosts a growing catalog of published, versioned, community- and OpsMill-authored schema bundles — but an agent can only reach them if a human leaves the conversation, browses the site, downloads YAML, and loads it by hand. The knowledge ("there's a published dcim schema you can start from") lives on the other side of a wall the agent can't cross.

Solution Overview

The MCP server gains a marketplace-aware capability: an agent can search the catalog, read a schema's or collection's details and its YAML, and — when explicitly enabled — install a chosen schema into the connected Infrahub instance on a safe session branch for human approval. Discovery and fetch are read-only and anonymous (the marketplace's public API serves only public, published content). Installation reuses the server's existing branch-safe write conventions, so nothing lands on the default branch without a human in the loop. For the user, "find me a starting schema for DCIM and set it up" becomes a single conversational flow instead of a context switch to a browser.

User Stories

1. As an infrastructure engineer, I want to search the marketplace by keyword, tag, or namespace from my AI assistant, so that I can find relevant published schemas without leaving my workflow.
2. As an engineer, I want to see a marketplace schema's description, author, version, and stats, so that I can judge whether it fits before pulling it in.
3. As an engineer, I want to fetch the actual schema YAML of a specific schema (latest or a pinned version), so that I can review exactly what I'd be adopting.
4. As an engineer, I want to browse and fetch a Collection (a curated bundle of schemas), so that I can adopt a themed "starter pack" in one step.
5. As an engineer, I want the agent to install a marketplace schema into my connected Infrahub on a session branch, so that I can review the change as a Proposed Change before it touches the default branch.
6. As a platform admin, I want marketplace access to be a config flag I can turn off, so that I control whether agents reach an external service at all.
7. As a platform admin, I want install to be blocked whenever the server is in read-only mode, so that the marketplace can't become a backdoor around write controls.
8. As an operator, I want marketplace calls that fail (network error, unknown ref, service down) to return clear, actionable messages, so that I can tell "not found" from "can't reach the marketplace."
9. As a security reviewer, I want marketplace requests to carry no Infrahub credentials and to honour the SDK's proxy/TLS settings, so that reaching an external service doesn't leak internal secrets.
10. As an auditor, I want install actions recorded through the same audit path as other writes, so that adopting external schemas is traceable.

User Journeys (prioritised)

P1 — Discover and read a marketplace schema

• Journey: Agent searches the catalog, picks a result, and retrieves its details and YAML.
• Acceptance: Given the marketplace has a public published schema matching "dcim", When the agent calls marketplace_search("dcim") then marketplace_get_schema("opsmill/dcim"), Then it receives the ranked catalog entry list and, on the second call, the schema's metadata plus its decompressed YAML.

P2 — Install a marketplace schema into the connected Infrahub

• Journey: Agent fetches a schema's YAML and loads it into Infrahub on a session branch for review.
• Acceptance: Given the server is not in read-only mode and marketplace access is enabled, When the agent calls marketplace_install("opsmill/dcim"), Then the schema is loaded via the SDK onto the session branch and the response reports the branch and what was applied — the default branch is untouched.

P3 — Adopt a Collection bundle

• Journey: Agent browses collections and fetches an assembled bundle of its member schemas.
• Acceptance: Given a public collection opsmill/starter, When the agent calls marketplace_get_collection("opsmill/starter"), Then it receives the collection's ordered member schemas as a valid multi-document YAML stream.

Functional Requirements

• FR-001: The server MUST expose a read-only tool to search the marketplace catalog by free-text query, with optional tag and namespace filters and pagination.
• FR-002: The server MUST expose a read-only tool to retrieve a schema's catalog metadata and its YAML payload, resolving namespace/name and accepting an optional pinned version.
• FR-003: The server MUST expose a read-only tool to retrieve a collection's metadata and its assembled member-schema YAML.
• FR-004: Marketplace read tools MUST be tagged for retrieval and marked readOnlyHint, and MUST return only public, published content.
• FR-005: The server MUST expose a tool tagged "write" that installs a marketplace schema into the connected Infrahub via the Infrahub SDK, isolated to a session branch.
• FR-006: The install tool MUST be blocked by the existing read-only enforcement when the server is in read-only mode.
• FR-007: Marketplace access MUST be gated by a ServerConfig flag; when disabled, marketplace tools MUST NOT be registered/callable.
• FR-008: The marketplace base URL MUST be configurable via ServerConfig, defaulting to https://marketplace.infrahub.app.
• FR-009: Marketplace HTTP requests MUST NOT include Infrahub credentials and MUST honour the SDK's proxy/TLS configuration.
• FR-010: The server MUST distinguish and clearly report "not found", "invalid ref", and "cannot reach marketplace" conditions without leaking internal detail.
• FR-011: When a namespace/name resolves to both a schema and a collection, the server MUST resolve deterministically (schema wins) and surface the ambiguity, matching infrahubctl behaviour.

Key Entities

• Schema (marketplace, external): A published catalog entry keyed by (namespace, name) — the thing an agent searches for and installs. Maps onto the connected instance's schema on install.
• Schema Version (marketplace, external): The immutable, semver'd YAML payload actually downloaded and loaded.
• Collection (marketplace, external): An ordered bundle of Schemas; installing one fans out to its member schemas.
• Session branch (existing): The isolation boundary an install lands on — reused unchanged from the write tools.
• ServerConfig (existing, extended): Gains marketplace_url and a marketplace-enabled flag, validated at startup.
• MarketplaceClient (new): A thin async client over the public /api/v1 API, mirroring infrahubctl marketplace's URL scheme and schema-vs-collection auto-detect.

Edge Cases

• Marketplace unreachable or 5xx → surfaced as a distinct "cannot reach" error, not "not found".
• namespace/name matches both a schema and a collection → schema wins, ambiguity noted.
• Pinned version doesn't exist though the schema does → "no such version" (not generic not-found).
• Collection bundle exceeds the marketplace's size cap → the marketplace returns 413; the server relays a clear "too large" message.
• Install attempted while read-only → blocked before any HTTP call to the marketplace.
• Marketplace access disabled by config → tools absent; agent gets no partial capability.
• Downloaded YAML fails Infrahub's own schema validation on load → install fails on the branch with the SDK's validation error, default branch untouched.

Success Criteria

• SC-001: An engineer can go from "find a DCIM schema" to reading its YAML entirely within the assistant, with no browser step.
• SC-002: An engineer can adopt a published schema into their Infrahub for review in under a minute, and it never appears on the default branch without approval.
• SC-003: With marketplace access disabled, no marketplace capability is reachable and no external request is made.
• SC-004: A failed marketplace interaction tells the operator whether the problem is the ref, the content, or the service — in one message.

Implementation Decisions

• Modules to build / modify (from module sketch, revised after SDK inspection):
  • MarketplaceClient (service, new): async httpx client over /api/v1 — search, schema/collection detail, versioned download (+gzip decompress), ref resolve/auto-detect. Mirrors infrahubctl marketplace URL/semantics; the SDK offers no importable marketplace client, only the CLI + a marketplace_url config default, so this is net-new but not reinvented convention.
  • tools/marketplace.py (MCP tool, new): read tools (marketplace_search, marketplace_get_schema, marketplace_get_collection) + the "write"-tagged marketplace_install.
  • resources/marketplace.py (MCP resource, new, optional): infrahub://marketplace/… mirrors of discovery for resource-capable clients.
  • config.py (config, extend): marketplace_url default + marketplace-enabled flag.
• API / interface surface: new MCP tools above; optional new resources. No changes to existing tools.
• Error handling: a small set of marketplace error categories (invalid-ref, not-found, network/unreachable, too-large) mapped to MCP-standard errors.
• Data / persistence: none (marketplace state is external; install writes go through the SDK to Infrahub).
• SDK / CLI surface: none added here — this consumes the marketplace API directly and the SDK for the Infrahub-side load.

Testing Decisions

• What makes a good test here. Test observable behaviour — given a mocked marketplace response, the right tool output or error category — not httpx internals.
• Unit tests: MarketplaceClient (mock httpx: pagination, gzip, ref parsing, auto-detect, error mapping) and the marketplace_install path (read-only guard, session-branch routing, validation-failure handling). The SDK exposes no importable marketplace client, so this client is ours to unit-test.
• Integration / contract tests: marketplace tools registered and gated correctly by the config flag; read-only mode blocks install.
• E2E scenario: search → get_schema → install onto a session branch against a testcontainers Infrahub (building on spec 001-infrahub-testcontainers), asserting the default branch is untouched.
• Prior art: tests/ mirror of tools/write.py and existing schema-tool tests; SDK's own ctl/marketplace.py for URL/semantics reference.

Constitution Alignment

• II. Infrahub SDK Integration: Marketplace calls are to a separate external service, so direct httpx is correct — but the install-side load into Infrahub MUST go through the SDK. This boundary is called out explicitly so "never raw HTTP" isn't read as forbidding the marketplace client.
• III. Branch-Safe by Default: Install is "write"-tagged, session-branch isolated, and blocked by ReadOnlyMiddleware in read-only mode.
• IV. Type Safety & Config at Boundary: New config validated at startup via pydantic-settings; client returns typed models.
• VI. Security & Input Boundaries: No Infrahub credentials on marketplace requests; refs validated before use; external errors sanitised.

Governance Gates Crossed

• Database schema or migration change
• API / public interface change (new MCP tools/resources)
• New dependency (none — httpx already present via the SDK)
• CI/CD workflow change
• Authentication / authorization change (new external-service call path + a new write tool; needs sign-off per AGENTS.md "Ask First")

Assumptions

• The marketplace public API stays stable at /api/v1 with anonymous read access to public/published content (consistent with infrahubctl marketplace).
• Installing a schema means loading its YAML into the connected Infrahub via the SDK's schema-load path onto a session branch; turning the branch into a Proposed Change follows the existing session conventions.
• Agents adopting external schemas is desirable default behaviour when enabled; admins who disagree disable the flag.

Out of Scope

• Publishing, forking, upvoting, reviewing, or any authenticated/write operation against the marketplace.
• Marketplace user accounts, API tokens, or namespace management from the MCP server.
• Automatic dependency resolution across schemas (the marketplace tracks dependency edges; v1 installs what's asked for).
• Turning the session branch into a Proposed Change automatically (left to existing session flow / the user).

Open Questions

• [NEEDS CLARIFICATION: Should marketplace_install load only the named schema, or also follow the marketplace's tracked dependencies? (v1 assumes named-only.)]
• [NEEDS CLARIFICATION: Should marketplace access default on or off for a fresh install, given it introduces an outbound external call?]

Further Notes

• Related specs: specs/001-infrahub-testcontainers (E2E harness this would build on).
• Related ADRs: 0004-tag-based-read-only-mode, 0005-lazy-session-branch-creation, 0006-config-validation-at-boundary.
• Source of this PRD: request to integrate marketplace.infrahub.app into the MCP server, grounded in the marketplace's public API / opsmill/infrahub-marketplace source and the SDK's ctl/marketplace.py.