Skip to content

Latest commit

 

History

History
317 lines (217 loc) · 24.6 KB

mcp-apps.md

File metadata and controls

317 lines (217 loc) · 24.6 KB

MCP Apps in AgentTools

This document explains how MCP Apps work in AgentTools and how to extend them.

Overview

MCP Apps is the first official MCP extension (io.modelcontextprotocol/ui), enabling servers to deliver interactive HTML user interfaces that render inside MCP hosts (Claude Desktop, VS Code, etc.) in sandboxed iframes.

AgentTools uses MCP Apps to provide:

  • Interactive Wolfram|Alpha results displayed in an embedded notebook viewer
  • Rich evaluation output from WolframLanguageEvaluator with interactive cloud notebooks
  • Embedded notebook viewers for Wolfram Cloud notebooks

When a client does not support MCP Apps, all tools fall back to their standard text and image output, maintaining full backward compatibility.

How It Works

Capability Negotiation

MCP Apps support is negotiated during the initialize handshake:

  1. The client advertises support for the io.modelcontextprotocol/ui extension in its capabilities
  2. The server detects this and echoes the extension in its response
  3. For the rest of the session, the server enriches tool definitions and results with UI metadata

The server checks two conditions before enabling MCP Apps:

  • The client must advertise io.modelcontextprotocol/ui in capabilities.extensions
  • The MCP_APPS_ENABLED environment variable must not be set to "false"

Cloud deployments. MCP Apps are also supported by cloud-deployed servers, whose stateless HTTP transport has no session store to hold $clientSupportsUI between requests. There, the negotiated capability is carried in a self-describing Mcp-Session-Id header that the client echoes on each request, re-establishing the same flag per request. If a client does not echo the session ID, MCP Apps simply stays off (fail-safe).

UI Resources

UI resources are HTML apps served via the MCP resources/read endpoint. Each resource is identified by a ui:// URI (e.g., ui://wolfram/wolframalpha-viewer).

Resources are loaded from HTML files in the Assets/Apps/ directory at server startup. Each HTML file can have an accompanying .json metadata file with the same base name.

The server handles these MCP methods for UI resources:

Method Description
resources/list Returns the list of available UI resources (empty if MCP Apps is not active)
resources/read Returns the HTML content and metadata for a specific UI resource

Tool-UI Linkage

Tools can be associated with a UI resource. When the client supports MCP Apps, the tools/list response includes _meta.ui metadata on each linked tool:

{
  "name": "WolframAlpha",
  "description": "...",
  "inputSchema": { ... },
  "_meta": {
    "ui": {
      "resourceUri": "ui://wolfram/wolframalpha-viewer",
      "visibility": ["model", "app"]
    }
  }
}

The host uses this metadata to preload the HTML app and render it alongside tool results.

UI-Enhanced Tool Results

When MCP Apps is active, certain tools return enhanced results with _meta containing a notebookUrl. The host forwards this metadata to the rendered app, which can then embed the notebook interactively.

Tools with UI-enhanced behavior:

Tool Enhancement
WolframAlpha Deploys a cloud notebook with formatted Wolfram|Alpha pods and returns notebookUrl in _meta
WolframLanguageEvaluator Deploys a cloud notebook with evaluation results and returns notebookUrl in _meta

These enhancements require MCP Apps support and a way to host the notebook. The helper deployCloudNotebookForMCPApp deploys it with CloudDeploy when the kernel is connected to the Wolfram Cloud; otherwise — or if CloudDeploy fails — it uploads the notebook to a public notebook hosting API that needs no cloud connection (see Notebook Hosting Without a Cloud Connection). The session flag $deployCloudNotebooks gates delivery as a whole: it starts out True and is set to False only when the hosting API turns out to be unavailable, after which the tools fall back to their standard (non-UI) results for the rest of the session rather than surfacing an internal failure or retrying on every call. A CloudDeploy failure on its own only clears the $useCloudDeploy flag, so later notebooks go straight to the hosting API.

Cloud notebooks are deployed with AppearanceElements -> None by default, which hides the footer links that would not be clickable inside the MCP App iframe. Some cloud accounts reject this option with CloudDeploy::appearancenotsup; in that case the deployment is transparently retried without AppearanceElements, and the unsupported status is cached in a session flag ($includeAppearanceElements) so subsequent deployments skip the failing attempt.

The fallback is per-tool:

  • WolframLanguageEvaluator always has a text/image result it can return, so it degrades in place.
  • WolframAlpha has no text-only fallback app view, so its entry in $toolUIAssociations is itself conditional on $deployCloudNotebooks — when the flag is False, no _meta.ui is attached to the tool definition and the client never sees it as a UI-enabled tool.

Notebook Hosting Without a Cloud Connection

When the kernel is not cloud connected (! TrueQ @ $CloudConnected), or when CloudDeploy fails, deployCloudNotebookForMCPApp falls back to the public notebook hosting API at $notebookUploadEndpoint (https://www.wolframcloud.com/obj/agenttoolsmngr/CAG/api/1.0/UploadNotebook):

  1. uploadNotebook exports the notebook to a temporary .nb file, which is deleted again once the request completes.
  2. uploadNotebookFile does not send files larger than $notebookUploadSizeLimit (10 MB, the API's limit) at all; the result is $Failed for that notebook only.
  3. notebookUploadResponse POSTs the file as the multipart Notebook field, bounded by $notebookUploadTimeout.
  4. notebookUploadResult interprets the JSON answer. On success ({"success": true, "code": 200, "uuid": "…", "result": "https://www.wolframcloud.com/obj/<uuid>"}) the result URL is returned; it has the same UUID form as a notebook deployed with CloudDeploy, so the <result uuid="…"> marker and the viewers work unchanged. An error answer with a 4xx code (e.g. "tag": "NotebookTooLarge" or "InvalidNotebookFormat") concerns only this notebook: the result is $Failed and nothing is disabled. Any other outcome — a connection failure or timeout, a server error, or a non-JSON answer such as the login redirect produced by an endpoint that is not deployed — means the API is unavailable, so disableNotebookDelivery sets $deployCloudNotebooks to False (there is no delivery method left to fall back to) and $Failed is returned.

Hosted notebooks are temporary (removed within 24 hours) and, like notebooks deployed with Permissions -> {"All" -> {"Read", "Interact"}}, readable by anyone with the unguessable URL. The fallback steps are recorded in the server log as CloudDeployNotebookFailed, NotebookUploadSkipped, NotebookUploadRejected, and NotebookUploadFailed entries.

Notebook Delivery: Cloud vs. Inline

By default, a UI-enhanced notebook is deployed to the Wolfram Cloud and its URL is sent to the app in _meta.notebookUrl. An experimental alternative serializes the notebook and embeds it inline, avoiding the cloud round-trip. The delivery method is selected by the MCP_APPS_NOTEBOOK_METHOD environment variable:

MCP_APPS_NOTEBOOK_METHOD Behavior of deployCloudNotebookForMCPApp
unset (default) Deploys the notebook (with CloudDeploy, or via the notebook hosting API) and returns its URL
"Inline" Returns ExportString[nb, "NB"] — the serialized notebook itself — instead of a URL

The same notebookUrl field carries both forms. Each viewer app (evaluator-viewer.html, notebook-viewer.html, wolframalpha-viewer.html) decides how to embed based on the value: a string starting with http is embedded as a cloud URL, while any other value is passed to WolframNotebookEmbedder.embed as an inline notebook expression ({expr: ...}).

Inline embedding is experimental and not yet the default. Neither method requires a cloud connection: the UI-enhanced path is gated on $deployCloudNotebooks regardless of the delivery method, and the "Inline" branch only asserts the flag rather than deploying.

When inline embedding is active, graphics can render empty in the embedded notebook. The delayedDisplay helper works around this for WolframLanguageEvaluator output: any output boxes containing GraphicsBox/Graphics3DBox are serialized and reconstructed asynchronously inside a DynamicModule (showing a progress indicator until ready). Outside inline mode, or for output without graphics, delayedDisplay returns the boxes unchanged.

Rendering the Notebook: Embedder vs. Cross-Origin Iframe

WolframNotebookEmbedder.embed does not use an iframe. It fetches wolframcloud.com/notebooks/embedding and injects the cloud notebook engine's scripts (mainScript + otherScripts) directly into the app document, where the engine then runs. That engine uses eval/new Function and WebAssembly, all of which the browser gates on the app iframe's CSP script-src 'unsafe-eval' (WASM additionally on 'wasm-unsafe-eval').

Strict MCP hosts build the app sandbox CSP without 'unsafe-eval', and there is no way for the server to add it. Goose, for example, constructs the CSP server-side (crates/goose/src/acp/mcp_app_proxy.rs) and runs every declared csp domain through a validator (normalize_csp_source) that rejects any entry containing a ' (single quote) — so quoted keyword-sources like 'unsafe-eval'/'wasm-unsafe-eval' are dropped, exactly as data: in connectDomains is dropped. Under such a CSP the injected engine throws EvalError and the notebook never renders, even though the embedder's embed() promise resolves.

To render the notebook anyway, each embedding viewer (evaluator-viewer, notebook-viewer, wolframalpha-viewer) probes eval capability once at startup with cspAllowsEval (a synchronous new Function(""), which throws under a no-'unsafe-eval' policy):

  • eval permitted → use WolframNotebookEmbedder as before (in-document render, fit-to-content sizing).
  • eval blocked → embedNotebookViaIframe points a plain cross-origin <iframe> straight at the cloud notebook URL. The notebook then renders inside wolframcloud.com's own origin under its CSP (which permits unsafe-eval), fully isolated from the app's CSP. The app's frame-src already allows https://www.wolframcloud.com, and the notebook is deployed chrome-free (AppearanceElements -> None), so the framed page shows just the notebook.

The fallback is additive — nothing changes on eval-permitting hosts. Two constraints are worth noting:

  • Sizing. A cross-origin iframe can't be measured by the app, so it opens at a default height (NOTEBOOK_IFRAME_HEIGHT, 200 px; notebook-viewer uses a finite positive maxHeight tool argument when given) that the user can grow with a drag handle along the frame's bottom edge (makeResizableFrame), with internal scrolling, rather than the embedder's fit-to-content sizing. A native corner resize grip is avoided because the framed notebook's own scrollbar sits on top of it and swallows the clicks.
  • Cloud URLs only. Only an http(s) cloud URL can be framed. Inline notebooks (MCP_APPS_NOTEBOOK_METHOD="Inline") carry a serialized expression with no URL, so on an eval-blocked host they fall through to the text/image result — another reason inline delivery remains experimental.

Recovering the Notebook URL When _meta Is Dropped

The notebookUrl is delivered to the app through _meta, which is meant to reach the app without entering model context. (The MCP Apps spec also defines structuredContent for this, but the server deliberately does not send it: some clients discard a tool result's content — text and images — entirely when structuredContent is present, which we do not want.) Some hosts, however, drop _meta from tool results (ext-apps#696), so the app never receives the URL directly and can only render the text/image fallback. Those same hosts also do not forward app-initiated resources/read (they answer it with JSON-RPC -32601 "Method not found"), so the app cannot ask the server for the URL either.

The one channel that does survive is the tool result's content. So for cloud-notebook results, makeNotebookUIResult wraps the content in a <result uuid="…">…</result> marker whose uuid identifies the deployed cloud notebook (a text item before and after the result text):

<result uuid="e0f29bea-667b-4780-b36b-59de225e660e">
Out[1]= 2543568463
</result>

Notebooks are deployed with CloudObjectNameFormat -> "UUID", so the deployed URL is already https://www.wolframcloud.com/obj/<uuid> and the uuid is recovered from it with no extra round-trip (cloudNotebookUUID). When a viewer sees a result with no notebookUrl in _meta, it falls back to extractNotebookUrlMarker, which reads the uuid from the marker and reconstructs the same cloud URL as https://www.wolframcloud.com/obj/<uuid>. Both delivery paths carry a syntaxMethod=editor query parameter on the embedded-notebook URL: the server appends it to notebookUrl (notebookEmbedURL), and the viewers append the same parameter when reconstructing from the marker — the two must stay in sync. Each text-rendering viewer also strips the surrounding <result> tags (via stripAgentOnlyText), keeping the wrapped result text, so the tags never reach the user.

This path applies only to cloud delivery: inline notebooks (MCP_APPS_NOTEBOOK_METHOD="Inline") carry the whole serialized notebook, which is delivered via _meta only, so no wrapper is added. The notebook-viewer app normally receives its URL through the tool input (arguments.url), which is unaffected by the dropped-_meta issue; it applies the same marker recovery only as a fallback when a result arrives without a prior embed.

Custom Cloud Base

All cloud URLs above assume the production cloud, https://www.wolframcloud.com. Setting the WOLFRAM_CLOUDBASE environment variable (primarily for internal purposes) points the server at a different cloud:

"env": { "WOLFRAM_CLOUDBASE": "https://www.test.wolframcloud.com" }

At server startup, setCloudBaseFromEnvironment assigns the value to $CloudBase, so notebook deployments (and every other cloud operation) target that cloud. The static app assets are adjusted to match as they are loaded from disk (loadUIResource):

  • Each viewer declares the cloud base in a var WOLFRAM_CLOUDBASE = "https://www.wolframcloud.com"; assignment, which it uses to reconstruct notebook URLs from <result uuid="…"> markers and to accept the configured origin in the iframe-fallback URL check (isWolframCloudUrl). The sandboxed JavaScript cannot read environment variables, so applyCloudBaseToHTML rewrites this assignment via string replacement when the HTML is read.
  • The JSON metadata's CSP domain lists (connectDomains, resourceDomains, frameDomains) must also allow the custom cloud, so applyCloudBaseToMeta prepends the custom base to every list that allows the default base. The default entries are kept so production URLs remain reachable (e.g. wolfr.am frames can redirect to production).

Available UI Resources

URI HTML Asset Description
ui://wolfram/wolframalpha-viewer wolframalpha-viewer.html Displays Wolfram|Alpha results with embedded notebook viewer
ui://wolfram/evaluator-viewer evaluator-viewer.html Displays Wolfram Language evaluation results with embedded notebook viewer
ui://wolfram/notebook-viewer notebook-viewer.html Generic embedded Wolfram Cloud notebook viewer
ui://wolfram/mcp-apps-test mcp-apps-test.html Diagnostic app for testing the MCP Apps pipeline

Available MCP Apps Tools

These tools are defined in $DefaultMCPTools but are not included in any default server configuration:

Tool Description
NotebookViewer Embeds an interactive Wolfram Cloud notebook given a URL
MCPAppsTest Diagnostic tool that echoes input with server metadata, useful for testing the MCP Apps pipeline

To include these tools in a custom server:

CreateMCPServer["MyServer", <|
    "Tools" -> {
        "WolframLanguageEvaluator",
        "WolframAlpha",
        "NotebookViewer"
    }
|>]

Tool-UI Associations

The mapping between tools and their UI resources is defined in $toolUIAssociations in Kernel/UIResources.wl:

Tool UI Resource URI
NotebookViewer ui://wolfram/notebook-viewer
MCPAppsTest ui://wolfram/mcp-apps-test
WolframAlpha ui://wolfram/wolframalpha-viewer (only when $deployCloudNotebooks is True)
WolframLanguageEvaluator ui://wolfram/evaluator-viewer

Disabling MCP Apps

MCP Apps can be disabled at install time:

InstallMCPServer["ClaudeDesktop", "EnableMCPApps" -> False]

This sets MCP_APPS_ENABLED=false in the server's environment, which prevents the server from negotiating UI support regardless of client capabilities.

MCP Apps are also effectively disabled when:

  • The client does not advertise the io.modelcontextprotocol/ui extension
  • The server cannot load its UI assets (graceful fallback)

Adding a New UI Resource

Step 1: Create the HTML App

Create an HTML file in Assets/Apps/:

Assets/Apps/my-app.html

The HTML file should implement the MCP Apps host-app protocol using postMessage. At minimum, the app should:

  1. Send ui/initialize to the host when ready
  2. Handle ui/notifications/tool-input and ui/notifications/tool-result messages

Step 2: Add Optional Metadata

Create a JSON metadata file with the same base name:

Assets/Apps/my-app.json

This file can contain CSP declarations and other metadata used by the host. Under csp, the host adds an implicit 'self' and appends each declared domain to the matching directive:

csp field Maps to Governs
connectDomains connect-src fetch/XHR/WebSocket, and data:/streaming WebAssembly loads
resourceDomains script-src, style-src, img-src, … External scripts, styles, images, fonts
frameDomains frame-src Nested iframes (e.g. the embedded notebook)

Apps that embed a notebook with WolframNotebookEmbedder must include "data:" in connectDomains: the embedder's WXFWeb library instantiates a WebAssembly module from a data: URI, which the browser governs under connect-src. The evaluator-viewer, notebook-viewer, and wolframalpha-viewer apps declare this.

CSP only governs whether a request may start; cross-origin responses are still subject to CORS, which the host cannot influence through this metadata.

Step 3: Associate with a Tool

Add the tool-to-resource mapping in $toolUIAssociations in Kernel/UIResources.wl:

$toolUIAssociations = <|
    (* ... existing entries ... *)
    "MyTool" -> "ui://wolfram/my-app"
|>;

The URI is derived from the HTML filename: ui://wolfram/<basename>.

Step 4: Write Tests

Add tests in Tests/ for the new resource. See the existing test files (Tests/MCPApps.wlt, Tests/MCPAppsTest.wlt, etc.) for patterns.

Architecture

Key Files

File Description
Kernel/UIResources.wl UI resource registry, capability detection, tool-UI metadata
Kernel/Server/Shared.wl Protocol handling for resources/list, resources/read, and _meta forwarding
Kernel/CommonSymbols.wl Shared symbols for MCP Apps ($clientSupportsUI, $uiResourceRegistry, etc.)
Kernel/InstallMCPServer.wl "EnableMCPApps" option and MCP_APPS_ENABLED environment variable
Kernel/Messages.wl Error messages for UI resources
Assets/Apps/ HTML and JSON files for UI resources
Kernel/Tools/NotebookViewer.wl NotebookViewer tool definition
Kernel/Tools/MCPAppsTest.wl MCPAppsTest diagnostic tool definition
Kernel/Tools/WolframAlpha.wl UI-enhanced Wolfram|Alpha evaluation
Kernel/Tools/WolframLanguageEvaluator.wl UI-enhanced code evaluation

Key Symbols

Symbol Context Description
$clientSupportsUI Common Whether the current client supports MCP Apps
$uiResourceRegistry Common Association of loaded UI resources keyed by URI
$toolUIAssociations Common Mapping of tool names to UI resource URIs (entries may be RuleDelayed to gate on $deployCloudNotebooks)
$deployCloudNotebooks Common Session flag gating notebook delivery for UI-enhanced results; starts out True (no cloud connection needed) and is set to False once the notebook hosting API turns out to be unavailable
deployCloudNotebookForMCPApp Common Shared helper that delivers a notebook for a UI-enhanced tool result and returns its URL — via CloudDeploy when cloud connected, otherwise (or if CloudDeploy fails) via the notebook hosting API — or returns the serialized notebook when MCP_APPS_NOTEBOOK_METHOD is "Inline"; returns $Failed when the notebook could not be delivered
$useCloudDeploy UIResources (private) Whether CloudDeploy is still worth trying (a cloud connection is required as well); cleared for the rest of the session after a CloudDeploy failure
cloudDeployNotebook UIResources (private) Deploys a notebook to the user's cloud account (AgentTools/Notebooks/<hash>.nb, URL in UUID form) when cloud connected; returns $Failed otherwise or on failure
uploadNotebook UIResources (private) Uploads a notebook to the hosting API via a temporary file (uploadNotebookFile, which skips files over $notebookUploadSizeLimit) and returns the hosted URL or $Failed
notebookUploadResponse UIResources (private) The HTTP POST to $notebookUploadEndpoint on its own (bounded by $notebookUploadTimeout), so tests can substitute canned responses
notebookUploadResult UIResources (private) Interprets the API's answer: the hosted URL on success, $Failed for a 4xx error concerning only this notebook, and disableNotebookDelivery (clears $deployCloudNotebooks) when the API gave no usable answer
makeNotebookUIResult Common Builds the UI-enhanced tool result from the text content and the delivered notebook value: carries notebookUrl in _meta (not structuredContent, which some clients treat as a replacement for content), and for cloud URLs wraps the content in a <result uuid="…">…</result> marker (the dropped-_meta workaround); returns $Failed when deployment failed
delayedDisplay Common Wraps WolframLanguageEvaluator output boxes so graphics reconstruct asynchronously when notebooks are embedded inline; a no-op outside inline mode or for graphics-free output
clientSupportsUIQ Common Checks if an initialize message advertises UI support
mcpAppsEnabledQ Common Checks the MCP_APPS_ENABLED environment variable
initializeUIResources Common Loads HTML assets into the resource registry
listUIResources Common Returns the resource list for resources/list
readUIResource Common Handles resources/read requests
toolUIMetadata Common Returns _meta.ui for a tool name
withToolUIMetadata Common Augments a tool list with UI metadata
notebookEmbedURL UIResources (private) Appends the syntaxMethod=editor query parameter to a deployed notebook URL for _meta.notebookUrl; inline (non-URL) values pass through
setCloudBaseFromEnvironment ServerLocal` (private) Applies the WOLFRAM_CLOUDBASE environment variable to $CloudBase at server startup
applyCloudBaseToHTML UIResources (private) Rewrites a viewer's var WOLFRAM_CLOUDBASE = "…" assignment when a custom cloud base is in effect
applyCloudBaseToMeta UIResources (private) Prepends a custom cloud base to the CSP domain lists in app JSON metadata

Related Documentation