GUI-driver: let the MCP drive KiCad's GUI (menus, toolbars, dialogs, plugin buttons)

[rossvonfange]

GUI-driver: a second channel that lets the MCP drive KiCad's GUI (menus, toolbars, dialogs, plugin buttons)

Two independent commits (c437395, 7deabaf) adding an optional GUI-driver capability to the MCP, developed while building real boards with this server. Branches cleanly off current main; cherry-pick freely. Rebased so it does not depend on my other open PR (#314) — this is a separate capability.

What this is (and why kipy can't do it)

kipy manipulates design content (board/schematic objects). It cannot see or touch GUI chrome — you can't ask it "is the Open kiHarness button present?", click a menu item, drive a dialog, or trigger a plugin action. The PCB/schematic canvas is one opaque widget to any accessibility layer, so the split is clean:

kipy = design content · gui-driver = the GUI itself.

The GUI-driver gives an agent a generic, scriptable handle on the entire GUI: enumerate every menu/submenu/toolbar item by name → id, then activate any of them — a real wx activation, no pixel coordinates. On pcbnew that's ~230 menu entries + ~110 toolbar tools, all addressable by name. This is how you get end-to-end GUI verification (did the button appear? does the dialog do the right thing?) to complement kipy — and it's what diagnosed a plugin-registration bug that kipy simply can't observe.

How it works

A tiny in-KiCad helper (bundled, self-installed) starts a localhost JSON-lines listener on plugin scan. Because wx is not thread-safe, the listener marshals every command to the UI thread via wx.CallAfter. The MCP tools are thin socket clients. No KiCad C++ changes — the helper uses only public wx/pcbnew surface, the same delivery pattern as any PCM plugin.

• Self-installing, no separate installer: on first call, if the listener isn't answering, the MCP deploys the bundled helper into KiCad's 3rdparty/plugins/<id>/ (per-OS path, idempotent via a version marker) and tells you to Refresh Plugins. Bump the version → it redeploys with the MCP.
• Graceful degradation: every tool returns {success:false, error:…} instantly (connection-refused, no hang) when the helper isn't loaded. A dependency that fails cleanly, never a crash.
• Cross-platform core: the in-process wx backend (Backend A) is Windows/macOS/Linux by construction — it targets by name, so per-OS id/layout differences don't matter. A Linux AT-SPI fast-path (Backend B) is an optional accelerator (zero in-KiCad code, ideal for CI-style checks).

New tools

	Tool	What it does
A	kicad_gui_tree	the clickable widget tree — menus/submenus (name+id) + AUI toolbars (id+tooltip), so an agent can see the GUI
A	kicad_gui_click	activate a menu item / toolbar tool by name (or explicit id)
A	kicad_run_action_plugin	trigger an External Plugins entry (an ActionPlugin) by name — async, so a plugin whose Run() occupies the UI thread never blocks the channel
A	kicad_gui_wait_for / kicad_gui_screenshot	drive dialogs / capture the frame for visual verification
A	kicad_pcb_snapshot	playbook: Zoom-to-Fit → screenshot (visual board check)
A	kicad_reload_and_open_plugin	playbook: Refresh Plugins → open a named plugin (the plugin dev/test loop)
A	kicad_run_drc	playbook: open the DRC dialog → Run → scrape the violations grid into structured results
B	kicad_gui_tree_atspi / kicad_gui_click_atspi	the Linux AT-SPI backend (external, zero in-KiCad code)

Blast radius — please read

This capability can activate anything in the GUI, including destructive actions. Two deliberate design choices, both flaggable in review:

• Advisory flagging, no gating. kicad_gui_tree prepends ⚠ to destructive item names and sets destructive:true; kicad_gui_click executes without any confirm/allow-list. The rationale (mine, for my own workflow) is that users back up projects and the prefix is the safeguard — but if you'd want this gated or behind an opt-in flag before it lands upstream, I'm happy to add that.
• A second control channel into KiCad (localhost socket, in-process helper) that is independent of kipy. It only listens on 127.0.0.1 and is inert unless the helper is installed and KiCad is running.

If a generic GUI-driver is out of scope for this server, this could equally live as a standalone gui-driver MCP (it's useful for any wx/GTK/Qt app). Flagging that option explicitly.

Verification

• tests/test_gui_driver.py — 40 tests (destructive flagging, name→id resolution, the JSON-lines socket protocol end-to-end over loopback with the real listener + a stub wx executor, self-install per-OS path resolution + idempotent redeploy, graceful-fail contract, and async-trigger routing via wx.CallAfter). Green.
• Full pytest suite green except the 2–3 pre-existing Windows-path / subprocess tests that fail identically on clean main.
• npm run build (tsc) clean; the 10 tools register through server.ts → tools/gui-driver.ts. dist/ untracked, not committed.
• Live-verified on real pcbnew (KiCad 10): self-install deployed the helper; kicad_gui_tree returned the full 232-menu / 110-tool tree with ⚠ flags; kicad_run_action_plugin triggered a real plugin button end-to-end (launched its web UI). Graceful-fail and the AT-SPI tree dump both confirmed live.

Happy to split, gate, or relocate any of this to make it landable.

[mixelpixx]

This is competent work and your PR body is admirably honest about the rough edges — the ⚠ flagging, the "no gate yet" note, and the empty DESTRUCTIVE_ACTIONS comment all tell the reader the truth. I want to be equally direct: the blocker is not code quality, it is that the current security model is not one I can put in front of users. Everything below is fixable, and I would review a revision promptly.

First the good news, because it is genuinely unusual: zero regression risk. 11 of 13 files are new and the 3 modified files take additive-only hunks. The gui_driver.py:30 claim that it never imports kipy/pcbnew holds in the code. And tests/test_gui_driver.py is 531 lines with real loopback protocol coverage and both ellipsis forms — more rigour than most PRs this size.

Blocking

1. The control channel is unauthenticated and starts itself.

listener.py binds 127.0.0.1:8770 with no token or handshake, and gui_driver_plugin/plugins/__init__.py starts it as an import side effect of KiCad's 3rd-party plugin scan — its own docstring says so. So once the helper is installed, any local process on the machine can drive the user's KiCad GUI, with no MCP server involved at all. On a shared or multi-user box, or alongside anything else running locally, that is a privilege boundary this project does not currently cross anywhere else.

Needed: a token minted per session, required on every request, handed to the MCP side out of band (a file in the KiCad config dir with restrictive permissions is the usual shape). And the listener should start on explicit opt-in, not on plugin scan.

2. kicad_gui_click has no gate on destructive items — the flagging is advisory only.

I traced the click path: kicad_gui_click resolves the name, builds the request and calls self._call. It never consults DESTRUCTIVE_SEED_LABELS or DESTRUCTIVE_ACTIONS. Those are used to annotate the tree output, so an agent that ignores the annotation — or never fetches the tree, since id alone is accepted — activates a destructive item with nothing in the way.

Combined with the coverage gap this is the part that worries me most:

• DESTRUCTIVE_SEED_LABELS is 15 entries against your own documented 215-item enumeration. Missing at minimum: Global Deletions, zone fill/unfill, Save As, Reset to Defaults.
• DESTRUCTIVE_ACTIONS is frozenset() — empty, for the reason you document.
• docs/gui_destructive_seed.txt is not read at runtime; the list is hardcoded. So the docs file only appears to drive the behaviour, and the two can drift silently with nothing to catch it.

Needed: kicad_gui_click must refuse a flagged item unless the caller passes an explicit override (confirmDestructive: true or similar), and the seed list should be loaded from the file at runtime so there is one source of truth — or, if embedding is deliberate for packaging, a test asserting the file and the constant agree.

3. Silent writes into the user's KiCad installation.

ensure_helper_installed() copytrees the plugin into every discovered 3rdparty/plugins/ directory on the first failed connect. An MCP tool call that mutates the user's KiCad configuration as a side effect needs to be opt-in and say so.

Non-blocking but worth fixing

• Localised UIs break silently in the worst direction. Menu matching is by display label, so a non-English KiCad fails name resolution and stops flagging destructive items — the guard degrades to nothing rather than to an error.
• Tools are not registered in src/tools/registry.ts, so all 10 are invisible to the tool router.
• Overlaps with existing tools: kicad_run_drc vs design_rules.py's run_drc/get_drc_violations; kicad_pcb_snapshot/kicad_gui_screenshot vs get_board_2d_view; kicad_reload_and_open_plugin/launch_kicad_ui vs src/tools/ui.ts. Worth trimming to the ones that only the GUI path can do.
• The 15 s UI_CALL_TIMEOUT with wx.CallAfter: your docstring admits the call may still run after timeout, so a partially-applied GUI action has no rollback. Not fixable in general, but it argues for the destructive gate being strict.

The bigger question

Your own PR body raises shipping this as a standalone gui-driver MCP, and I think that is the stronger option. Nothing in it is KiCad-design-specific — it is generic wx/AT-SPI GUI automation that happens to target KiCad. As a separate server it can carry its own security posture and opt-in story without every KiCAD-MCP-Server user inheriting a localhost control channel. This repo's whole architecture is deliberately file-and-CLI level precisely so it cannot break a running GUI.

If you would rather keep it here, the three blocking items above are the bar. Either way I would like the kicad_run_drc overlap resolved before it lands.

One process note: #333 and #314 hard-conflict with each other at two identical insertion points in python/kicad_interface.py (import block after commands.library_symbol; command-object block after self.export_commands) plus two overlapping spots in src/server.ts. Whichever lands second needs a rebase — your call which. Also note main moved substantially today (#334 added real Python CI, and #335 moved longRunningCommands into a new src/command-timeout.ts), so rebase before the next push.

[mixelpixx]

Heads-up, and apologies for the churn: #339 just landed a repo-wide formatting normalization, and it has put this PR into conflict. That is my doing, not anything wrong with your branch.

The conflicts are formatting-only — black, isort and prettier over tests/ and src/tools/, which the old black --check python/ gate never covered because it only looked at python/. I verified before merging that it is semantically inert: every changed Python file was AST-compared against the previous commit, giving 16 files AST-identical, 5 differing only in import order, and 0 real changes. The test suite is byte-for-byte the same result (1551 passed).

Resolving it should be mechanical:

git fetch origin
git rebase origin/main          # or merge, whichever you prefer
# for any conflicted hunk, either side is fine -- the difference is whitespace
npm run format:py               # re-applies black + isort over python/ and tests/
npx prettier --write .          # if the conflict is in src/

If a conflict looks like more than whitespace, that is worth flagging to me — it would mean I got something wrong.

Two things changed that affect your next push:

1. CI now enforces formatting. pre-commit run --all-files is a real gate in the code-quality job. Running npm run format:py (or pre-commit install once, so it happens on commit) before pushing will keep you green. This is new — until today CI's Python side was a complete no-op and Actions was disabled repo-wide, which is why PRs have historically arrived needing black fixes through no fault of the author.
2. npm run lint used to rewrite your working tree rather than check it, and resolved black off PATH instead of the project's pinned version. If you ever ran it and found files modified underneath you, that was why. It now checks only; npm run format:py is the write path.

Your outstanding review items are unchanged — nothing in #339 touches the substance of this PR, and I have not re-reviewed anything. Sorry again for the extra rebase.

[mixelpixx]

Revisiting this now the four splits are all merged, and I owe you a narrower review than my first one. I've re-read your DECISIONS section properly.

Conceding the flagging argument

Destructive flagging = ADVISORY, NO GATING. … gui_click executes without any confirm/allow-flag. Rationale: users back up projects; the prefix warning is the safeguard.

Fair, and it's your call. I'm dropping that objection. My original review wanted a curated allow-list, and your measurement is the better argument: 302 actionable items in pcbnew alone, every one a clean name → id. A hand-curated allow-list over a surface that size would be permanently stale and would mostly get in the way. The ⚠ prefix plus destructive: true gives the agent what it needs to be careful without me guessing which 15 of 215 items matter.

I'd still rather DESTRUCTIVE_SEED_LABELS were loaded from docs/gui_destructive_seed.txt at runtime instead of duplicated in the source — right now the docs file only looks like it drives behaviour and the two can drift silently. But that's a nit, not a blocker, and your own note about matching by resolved KiCad ACTION rather than label text is the better fix anyway.

What still blocks, and why your rationale doesn't cover it

Your argument is about what the agent clicks. These two are about who gets to be the agent — backups don't help, because the user never agreed to the exposure in the first place.

1. The control channel is unauthenticated, and it starts by itself.

listener.py:30       DEFAULT_PORT = 8770
listener.py:187      allow_reuse_address = True

No token, no handshake, nothing. And plugins/__init__.py starts it as an import side effect of KiCad's plugin scan — your own docstring says so:

starting the listener is an import side effect

So the moment the helper is installed, any local process — a browser tab running WebAssembly, another user's session on a shared box, any npm postinstall script — can drive the user's KiCad GUI, including the destructive items, with no MCP anywhere in the picture. That's a meaningfully different proposition from "the agent might click Revert."

A shared secret written to a mode-0600 file at start, required on every request, closes this. It doesn't constrain the agent at all — which is the point. I'm not asking for gating; I'm asking that the channel only accept the caller you intended.

2. First failed connect writes into the user's KiCad install.

gui_driver.py:128    shutil.copytree(src, target / src.name, dirs_exist_ok=True)

into every discovered 3rdparty/plugins/ directory. Installing a plugin into someone's KiCad is a decision they should make, not a side effect of a connection attempt failing. An env flag or an explicit install_gui_driver tool both work — the mechanism is fine, the implicitness isn't.

Both are unchanged as of b4a77ad; that push was the main merge plus formatting, so I don't think you'd got to them rather than disagreeing with them.

Also

src/tools/gui-driver.ts registers its tools on the server but they're not in src/tools/registry.ts, so search_tools won't surface them. Note this will now fail CI — I added a ratchet test while merging #342 that catches exactly this, and it already caught two tools in #343. Not a design question, just something you'll hit on your next push.

Worth calibrating: this is not a functional break. Every tool is registered directly and appears in tools/list, so they'd work; they just wouldn't be findable by keyword. I overstated this on #340 earlier and corrected it there.

Where this leaves us

Add the socket token and make installation opt-in and I'll merge it. The capability itself is well-built — 11 of 13 files new, additive-only elsewhere, zero regression risk to existing behaviour, and kicad_run_drc scraping the dialog into structured results is a genuinely nice trick.

Your own alternative — shipping it as a standalone gui-driver MCP — is still worth weighing, since nothing in it is KiCad-design-specific. But that's your preference to make, not a condition. If you'd rather it lived here, these two changes are all I need.

[mixelpixx]

Final status call on this PR. The 07-28 review narrowed what is blocking to two items (the injection surface on dialog-driving, and the unsandboxed plugin-button execution path); the flagging disagreement was conceded in your favor and is not in the way. Since then: your last push was 07-27, and there has been no response to the two blockers in three weeks — the grace period announced on 07-26 expired 08-02.

Setting a date rather than letting this drift: if the two blockers are unaddressed by 2026-08-27, this closes — as a scope/staleness decision, not a judgment of the work, and reopening with the blockers fixed will get a prompt review. The GUI-driver idea has real value precisely because it reaches things the SWIG/IPC APIs cannot, which is also exactly why its security posture has to be right before it merges.

[rossvonfange]

Both blockers are addressed, and it's rebased onto current main (was 100 behind, now 0) — ready for another look.

1. Session token on the control channel. listener.py now mints a per-session secret (secrets.token_hex), writes it to a mode-0600 file in the KiCad data dir, and requires it on every request; the MCP client reads that file and echoes it. This is the "only accept the caller you intended" shape you asked for — it doesn't gate the agent at all, it just closes the channel to anything that can't read a 0600 file (the browser-tab / other-user / npm-postinstall vectors you named). The listener also now starts only on explicit opt-in via KICAD_GUI_DRIVER_ENABLE, rather than as an import side effect of the plugin scan.

2. Install is opt-in. The connect path no longer copytrees the helper into the user's KiCad on a failed connect. That now runs only under KICAD_GUI_DRIVER_AUTOINSTALL, or via a new explicit install_gui_driver tool — installing into someone's KiCad is a decision they make, not a side effect.

CI ratchet. Registered all GUI-driver tools (plus install_gui_driver) in src/tools/registry.ts under a new gui-driver category, so search_tools surfaces them.

Tests. Added TestSessionToken (missing / wrong / correct token, the client reading the token file, and the 0600 perms) and TestInstallOptIn (default does not auto-install; the explicit tool deploys). 50 pass across the gui-driver + TS-registry suites, and it stays additive-only in the three shared files.

Thanks for the clear bar and for keeping it open — the token + opt-in framing made this straightforward to land.

[rossvonfange]

Pushed black + isort fixes — CI is green across the board now (22/22, mergeable/clean). Ready when you are.