Support runtime hot-loading of MCP servers and agents

[ankitdas-volgapartners]

Summary

PilotSwarm workers currently load MCP server and agent configuration from startup-time inputs such as plugin directories, .mcp.json, and inline worker options. That makes MCP and agent availability effectively fixed for a running worker process.

We should support runtime registration and hot-loading of MCP servers and agents so users or admins can add integrations without manually editing worker config and restarting/redeploying the worker fleet.

Problem

Today, adding an MCP server generally requires updating .mcp.json or worker/plugin configuration, then restarting or redeploying workers so the new config is loaded and passed into future Copilot sessions.

This blocks an important product/admin workflow:

• a user or admin discovers a new integration they need;
• they add the MCP server or agent from the product UI/API;
• relevant sessions should be able to use it without manual worker reconfiguration;
• secrets must remain out of prompts and transcripts.

This was discussed as a framework-level capability rather than a quick workaround.

Expected behavior

Users/admins can dynamically register MCP servers and agents at runtime. The resulting configuration is scoped to the right owner, propagated to the relevant workers/sessions, and made available without exposing API keys or requiring a manual worker restart.

Proposed approach

Add an admin/user-facing configuration surface, likely backed by durable storage, for managing MCP server and agent definitions.

A possible shape:

1. Persist MCP server and agent definitions with owner scope, visibility, enabled/disabled state, and secret references.
2. Expose management APIs for create/update/delete/list operations.
3. Have workers refresh or subscribe to config changes and apply them to newly created or newly hydrated sessions.
4. Keep secrets in a proper secret store or server-side encrypted storage, and pass only resolved runtime config to the SDK.
5. Surface the same capability in the UI/admin panel so users do not need to edit .mcp.json by hand.

Acceptance criteria

• A user/admin can add an MCP server while the system is running.
• MCP server config can be scoped per user, team, or deployment-level policy.
• Newly added MCP servers become available to relevant agents/workers without manual worker restart/redeploy.
• Agents can also be added or updated through the same runtime-managed catalog where appropriate.
• API keys and secrets are never exposed in prompts, transcripts, logs, or client-visible config.
• Existing static plugin-dir and .mcp.json loading continues to work for bootstrap/default integrations.

Security and policy considerations

• Validate allowed transports, commands, URLs, headers, and tool exposure before enabling a server.
• Support admin approval or allowlists for local/stdio MCP servers, since those can execute processes.
• Prefer secret references over raw secret values in stored MCP config.
• Make audit events available for MCP/agent config changes.
• Define how revocation works for active sessions that already have a server loaded.

Suggested metadata

• Type: Feature / framework gap
• Priority: High
• Labels: framework, mcp, runtime, feature-request
• Assignee: Ankit