Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions apps/web/src/artifacts/artifactRef.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,8 @@ describe("artifactRefFromProps", () => {
expect(artifactRefFromProps(undefined)).toBeNull();
// An unknown kind is dropped rather than echoed into the chip.
expect(artifactRefFromProps({ artifact_id: "a", version: 1, kind: "<b>x</b>" })?.kind).toBe("");
// A chart (ADR 0116) is a known kind, so an untitled one still says what it is.
expect(artifactRefFromProps({ artifact_id: "a", version: 1, kind: "vega-lite" })?.kind).toBe("vega-lite");
});

it("names an untitled artifact by its kind", () => {
Expand Down
2 changes: 1 addition & 1 deletion apps/web/src/artifacts/artifactRef.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ export const ARTIFACT_REF_COMPONENT = "artifact-ref";
/** The artifact plugin's panel — `plugin:<id>:<view>` from its manifest. */
export const ARTIFACT_VIEW_KEY = "plugin:artifact:artifact";

const KINDS = new Set(["html", "svg", "mermaid", "react", "markdown", "file"]);
const KINDS = new Set(["html", "svg", "mermaid", "react", "markdown", "vega-lite", "file"]);

export type ArtifactRef = { id: string; version: number; title: string; kind: string };

Expand Down
9 changes: 9 additions & 0 deletions changelog.d/4025.added.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
- **Charts as Vega-Lite specs, and a plugin-services seam, for the local-first Data Analyst (ADR 0116) (#4025).**
The Artifact panel adds a `vega-lite` kind: a small JSON spec with its rows inline is drawn by vendored, SRI-pinned
Vega / Vega-Lite / vega-embed, in the console's own chart palette (dark or light, redrawn on a theme switch). It
runs under a no-network CSP: no `eval`, a loader that refuses every fetch, and inline data only. A chart is a query
plus a few hundred bytes of spec instead of a hand-written component. In the end-to-end run, "what were my best
weekdays last quarter? chart it" went from send to chart in about 18 s, against about 50 s for the component path.
Plugins also gain a call seam: `registry.register_service(name, fn)` offers a callable, and
`graph.sdk.service("<plugin>.<name>")` resolves it, or returns `None` when the provider is off. The first service
is `artifact.show`, which the new Data Analyst plugin (`protoLabsAI/data-plugin`) uses to put its charts in the panel.
170 changes: 170 additions & 0 deletions docs/adr/0116-local-first-data-analyst-duckdb-and-vega-lite-charts.md

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions docs/adr/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,3 +123,4 @@ decision, numbered, never deleted (supersede instead).
| [0112](./0112-console-code-pane.md) | Console code pane (**amended 2026-09-25: an opt-in toolset, `filesystem.code_pane`, off by default**) — a read-only **navigator** surface the agent points into, motivated by evidence that watching an agent work lowers comprehension (arXiv 2607.26375; Anthropic's AI-assistance coding-skills study). (D1) core `code` surface, File \| Diff tabs + a Recent trail, opened on the dock NOT holding chat; (D2) `GET /api/fs/file` — the `read_file` fence, `\n`-delimited lines with endings preserved, 2 MB / 20k lines / 2k chars-per-line caps with paging, binary → metadata; (D3) `GET /api/fs/diff` — hardened git (no ext diff / textconv / filter driver / fsmonitor / hook / submodule recursion; all `GIT_*` env scrubbed), scoped to the project root, untracked files as Python-built patches, 1 MB cap, 10 s → 504; (D4) `tools/fs_secrets.py` secret-like-name deny list for display surfaces (403 before existence; `read_file` unchanged — a follow-up); (D5) `show_code` fs tool → a strictly validated `code-ref` component; (D6) console: chip, live-only auto-open on desktop, never on mobile, opt-in debounced follow mode with pin, pane-by-default links; highlighting library decided in the console PR. Amends 0035/0051/0056/0062/0086. | Proposed |
| [0113](./0113-agent-pairing-for-remote-fleet-members.md) | **Agent pairing for remote fleet members.** A hub pairs with a remote the way a phone does (ADR 0087): the remote shows a code (Settings ▸ Devices ▸ Pair an agent, or `protoagent pair`), and the hub claims it through the existing unauthenticated `/api/pairing/claim`, storing the minted per-hub device token (revocable on the remote) as the remote's bearer. (D2) agent codes are 10-char Crockford base32 (~50 bits), 5-min TTL, single-use, under the shared 5-miss lockout, which resets on each new code; the kind is fixed when the code is minted. (D4) a delegate to a remote routes through the hub's loopback proxy (fleet token → hub → paired token), so there is one token and one registry. (D5) an authenticated probe reports `auth: ok\|rejected`. (D6) remote WebSockets are re-enabled, and the hub never lends a stored credential: it only swaps a hub-verified operator `?token=`, and passes tickets through. (D8) the `settings.devices` flag is removed after a desktop-app test. (D9) mDNS advertises only on a reachable bind. (D10) a credential goes over plain http only to loopback or a tailnet address, or with an explicit opt-in. Rejects hub-requests/remote-approves and HTTPS discovery. Amends 0042 §I and 0087. | Proposed |
| [0115](./0115-gateway-inflight-limiter.md) | **Model calls share a bounded, priority-ordered in-flight lane per endpoint and model (off by default).** Nothing caps how many chat-model calls are in flight at once (`graph/` has no semaphore; the per-fan-out caps don't know about each other), so under load every caller times out and retries together (the #209 review-gate incident). A new in-process limiter in `graph/llm_limiter.py` keyed by `(base_url, model)` gives each lane a bounded queue: `model.max_inflight` (default **0 = off**), `model.inflight_queue_timeout` (300 s, separate from `request_timeout`), `model.inflight_interactive_reserve` (slots only `interactive` callers take). Waiters are served strict-priority (interactive > default > bulk) with 60 s aging, acquire OUTSIDE the stream-stall guard so wait time never counts as backend latency, and a queue timeout raises `GatewayQueueTimeout` (not retried, never mislabelled a stall). Per-process only (ACP coders / clawpatch / other instances stay out, D7); gateway-side limits remain the cross-process backstop. Adds queue-depth/wait Prometheus metrics + `GET /api/telemetry/llm-lanes` + `sdk.llm_lanes()`. Builds on 0006, keyed per 0106, boundary per 0025. Refs #3760. | Proposed |
| [0116](./0116-local-first-data-analyst-duckdb-and-vega-lite-charts.md) | **A local-first data analyst: DuckDB over files in place, charts as Vega-Lite specs, and a plugin-service call seam.** (D1) a `data` plugin in its own repo (`protoLabsAI/data-plugin`) — `data_connect/sources/schema/query/profile/chart/export` + two skills; the archetype bundle comes later. (D2) embedded DuckDB views over CSV/Parquet/JSON in place; SQLite/XLSX snapshotted to Parquet (their extensions would `INSTALL` from the network). (D3) read-only enforced BY THE ENGINE — a fresh in-memory connection per query with exact-file `allowed_paths`, `enable_external_access=false`, a private temp dir, extension autoload/autoinstall off, `lock_configuration` — plus a single-`SELECT` statement guard and row/time/memory caps. (D4) the scope is the operator-only `data_dirs` (`spawns: true`), campaign's upload fence (symlinks resolved first, credential names, agent home, hardlinks, too-broad roots), re-validated per query. (D5) a core `vega-lite` artifact kind: vendored SRI-pinned vega/vega-lite/vega-embed, a nonce CSP with no network, the CSP-safe expression interpreter (no eval), a deny-everything loader (+ `usermeta.embedOptions` stripped), inline-data checks at write time, themed from `--pl-color-chart-*` with live re-theme. (D6) plugin services — `registry.register_service` / `sdk.service(name)`, rebound wholesale on reload; first service `artifact.show`. (D7) exports only into the agent workspace. | Proposed |
26 changes: 26 additions & 0 deletions docs/reference/plugin-registry-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ Host services (agent invoke + event bus) a surface/route can use — the server
| [`register_mcp_server()`](#registry-register-mcp-server) | Contribute a **managed MCP server** the agent connects to ([ADR 0019](/adr/0019-plugin-config-settings-secrets)) |
| [`register_middleware()`](#registry-register-middleware) | Add a plugin-contributed LangGraph `AgentMiddleware` ([ADR 0032](/adr/0032-pluggable-middleware)) |
| [`register_router()`](#registry-register-router) | Mount a FastAPI `APIRouter` on the server ([ADR 0018](/adr/0018-plugin-surfaces-routes-subagents)) |
| [`register_service()`](#registry-register-service) | Offer `fn` to OTHER plugins as the service `<plugin_id>.<name>` ([ADR 0116](/adr/0116-local-first-data-analyst-duckdb-and-vega-lite-charts)) |
| [`register_setup_step()`](#registry-register-setup-step) | Register a SETUP STEP: the server-side half of a `plugin_setup` setup-gap action |
| [`register_skill_dir()`](#registry-register-skill-dir) | Add a directory of `SKILL.md` skills bundled with the plugin |
| [`register_subagent()`](#registry-register-subagent) | Add a `SubagentConfig` to `SUBAGENT_REGISTRY` ([ADR 0018](/adr/0018-plugin-surfaces-routes-subagents)) |
Expand Down Expand Up @@ -351,6 +352,31 @@ plugin-views.md, [ADR 0026](/adr/0026-plugin-contributed-console-surfaces)); a p
[#1732](https://github.com/protoLabsAI/protoAgent/issues/1732)). The default-deny auth middleware guards all non-public paths
regardless of prefix.

### `registry.register_service` {#registry-register-service}

```python
registry.register_service(name: str, fn, description: str = '') -> None
```

Offer `fn` to OTHER plugins as the service `<plugin_id>.<name>` ([ADR 0116](/adr/0116-local-first-data-analyst-duckdb-and-vega-lite-charts)).

The cross-plugin CALL seam: where the event bus (`emit`) is fire-and-forget, a
service returns a result. A consumer resolves it at call time with
`graph.sdk.service("<plugin_id>.<name>")` and gets your callable — or `None` when
your plugin is disabled, so it must degrade (a service is an optional capability). It
never imports your plugin, so you can refactor freely behind the name. The artifact
plugin's `artifact.show` is the reference: the data plugin creates charts through it.

The name is namespaced to this plugin: `"show"` registers `"<plugin_id>.show"`, and
a name already under this plugin's namespace is kept as-is (a plugin may only provide
under its own). The bare part is lowercase `[a-z][a-z0-9_]*`. Treat the callable's
signature and return shape as a PUBLIC API — document them in its docstring, keep them
backward compatible, and return refusals as data rather than raising for an expected
"no". It runs in the CALLER's thread (often a tool body; sync or async is your call,
say which). `description` is a one-line summary for status surfaces. Live while the
plugin is loaded — a reload that disables it stops the name resolving. Guard with
`getattr(registry, "register_service", None)` on hosts older than this seam.

### `registry.register_setup_step` {#registry-register-setup-step}

```python
Expand Down
28 changes: 28 additions & 0 deletions docs/reference/plugin-sdk-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ the module's own.

## Contents

**Plugin services (the plugin↔plugin call seam, ADR 0116)** — [`service()`](#sdk-service)

**Agent + model access (the plugin↔agent channel, ADR 0043)** — [`complete()`](#sdk-complete), [`config()`](#sdk-config), [`gateway_client()`](#sdk-gateway-client), [`run_subagent()`](#sdk-run-subagent), [`subagent_types()`](#sdk-subagent-types), [`turn_stop_reason()`](#sdk-turn-stop-reason)

**Model in-flight priority (the plugin↔limiter channel, ADR 0115 D6)** — [`llm_lanes()`](#sdk-llm-lanes), [`llm_priority()`](#sdk-llm-priority)
Expand All @@ -47,6 +49,32 @@ the module's own.

**Managed Python runtime (ADR 0094 — the provisioned child interpreter)** — [`managed_python_exe()`](#sdk-managed-python-exe)

## Plugin services (the plugin↔plugin call seam, ADR 0116)

### `sdk.service` {#sdk-service}

```python
sdk.service(name: str) -> Callable | None
```

The callable another plugin offers as `name` (`"<plugin_id>.<name>"`), or `None`.

The way one plugin CALLS another without importing it: the provider registers it with
`registry.register_service` and you resolve it here, at call time — never at
`register()` time, since plugins load in no guaranteed order and a reload can swap the
provider in or out. `None` means no loaded plugin provides it (disabled, not installed,
or a core older than the provider needs): degrade with an actionable message rather than
fail. Guard the lookup itself with `getattr(sdk, "service", None)` on cores older than
this seam. The callable's signature and return shape are the PROVIDER's documented API —
read its docstring (`help(sdk.service("artifact.show"))`). The reference provider is the
artifact plugin's `artifact.show`::

show = sdk.service("artifact.show")
if show is None:
return "The Artifact plugin is off — enable it to see charts."
r = show(kind="vega-lite", code=spec_json, title="Sales by weekday")
return r["message"] + r["ref"] # the artifact-ref chip tail goes LAST

## Agent + model access (the plugin↔agent channel, ADR 0043)

### `sdk.config` {#sdk-config}
Expand Down
70 changes: 70 additions & 0 deletions graph/plugin_services.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
"""Plugin services (ADR 0116) — a callable one plugin offers and another calls BY NAME.

The cross-plugin call seam. Before it, a plugin could reach another only through the event bus
(ADR 0039), which is fire-and-forget: no return value, and no answer to "is that plugin even
on?". A plugin that needs a RESULT from another — the data plugin creating a chart in the
Artifact panel and reporting the artifact it made — had nothing to call except the other
plugin's internals, which is exactly the coupling the plugin contract forbids.

So a plugin registers a service (``registry.register_service("show", fn)`` → ``artifact.show``)
and a consumer resolves it at CALL time (``graph.sdk.service("artifact.show")``), getting the
callable or ``None``. ``None`` is the normal "that plugin is disabled / not installed / on an
older core" answer, and every consumer must handle it — a service is an optional capability,
never a hard dependency (a hard one is a manifest ``requires_plugins`` concern, not this).

The live mapping is replaced WHOLESALE at build and on every plugin reload, like the verifier
and work-provider registries (#1752: a stale registry is worse than an empty one — a disabled
plugin's service must stop resolving the moment it's unloaded, not at the next restart). The
operator-MCP process applies the same mapping (``server/operator_mcp.py``), so a plugin tool
running there resolves services exactly as it would in the main process.
"""

from __future__ import annotations

import logging
import re
from collections.abc import Callable, Mapping

log = logging.getLogger(__name__)

# ``<plugin_id>.<name>`` — the provider's id is the namespace (``register_service`` prefixes it),
# so one plugin can't register a name under another's. Same id alphabet as the manifest.
SERVICE_NAME_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9_-]*\.[a-z][a-z0-9_]*$")

# name -> callable. Rebound wholesale (one GIL-atomic assignment) by ``set_plugin_services``.
_SERVICES: dict[str, Callable] = {}
# name -> {"plugin_id", "description"}; parallel to _SERVICES so its values stay THE callable.
_SERVICE_META: dict[str, dict] = {}


def is_service_name(name: object) -> bool:
"""True for a well-formed ``<plugin_id>.<name>`` service name."""
return isinstance(name, str) and len(name) <= 128 and bool(SERVICE_NAME_RE.match(name))


def set_plugin_services(mapping: Mapping[str, Callable] | None, meta: Mapping[str, dict] | None = None) -> None:
"""Replace the live service set (called at build + on every plugin reload).

Wholesale replacement, not a merge: a reload that drops a plugin drops its services too.
Malformed names and non-callables are skipped (the registry already refuses them; this is
the second line for a duck-typed bundle)."""
global _SERVICES, _SERVICE_META
fresh = {n: fn for n, fn in (mapping or {}).items() if is_service_name(n) and callable(fn)}
_SERVICES = fresh
_SERVICE_META = {n: dict(m) for n, m in (meta or {}).items() if n in fresh}


def get_service(name: str) -> Callable | None:
"""The callable registered as ``name``, or ``None`` when no loaded plugin provides it."""
return _SERVICES.get(name) if isinstance(name, str) else None


def service_names() -> list[str]:
"""Registered service names, sorted — the introspection half, for a status surface."""
return sorted(_SERVICES)


def service_meta(name: str) -> dict | None:
"""``{"plugin_id", "description"}`` for a registered service, or ``None``."""
m = _SERVICE_META.get(name)
return dict(m) if m is not None else None
22 changes: 22 additions & 0 deletions graph/plugins/loader.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,8 @@ class PluginLoadResult:
thread_id_resolver: object = None # (request_metadata, session_id) -> str (#571); last plugin wins
chat_commands: dict = field(default_factory=dict) # token -> handler; user-only chat control commands
components: dict = field(default_factory=dict) # component-v1 kind -> props validator (#3617)
services: dict = field(default_factory=dict) # "<plugin_id>.<name>" -> callable (ADR 0116)
service_meta: dict = field(default_factory=dict) # name -> {plugin_id, description}
meta: list[dict] = field(default_factory=list)


Expand Down Expand Up @@ -1096,6 +1098,25 @@ def load_plugins(config, *, core_tool_names: set[str] | None = None) -> PluginLo
log.warning("[plugins] %s: component %s collides — skipped", manifest.id, name)
continue
result.components[name] = validator
for name, fn in getattr(registry, "services", {}).items(): # plugin services (ADR 0116)
# register_service namespaces every name, but `registry.services` is a plain dict a
# plugin could write to directly — so the loader re-checks: a plugin provides ONLY
# under its own id, never as (or over) another plugin's service.
if not name.startswith(f"{manifest.id}."):
log.warning(
"[plugins] %s: service %s is outside this plugin's namespace (%s.*) — skipped",
manifest.id,
name,
manifest.id,
)
continue
if name in result.services: # namespaced per plugin, so only a duplicate plugin id hits this
log.warning("[plugins] %s: service %s collides — skipped", manifest.id, name)
continue
result.services[name] = fn
smeta = getattr(registry, "service_meta", {}).get(name)
if smeta:
result.service_meta[name] = smeta
entry["loaded"] = True
entry["tools"] = [t.name for t in kept]
# Count the conventional skills/ dir too (auto-discovered above) — counting only
Expand All @@ -1108,6 +1129,7 @@ def load_plugins(config, *, core_tool_names: set[str] | None = None) -> PluginLo
entry["mcp_servers"] = len(registry.mcp_servers)
entry["chat_commands"] = [f"/{t}" for t in registry.chat_commands]
entry["components"] = sorted(getattr(registry, "components", {}))
entry["services"] = sorted(getattr(registry, "services", {}))
result.meta.append(entry)
log.info(
"[plugins] loaded %s: %d tool(s), %d skill dir(s), %d route(s), "
Expand Down
Loading
Loading