diff --git a/README.md b/README.md index bd6b4c5..069ca18 100644 --- a/README.md +++ b/README.md @@ -65,8 +65,8 @@ accessory. One bridge, one pairing, however many devices. something this plugin can change. - **Apple Home only, as a claim.** Alexa, Google Home and SmartThings are untested and unclaimed; nothing here says they will or won't work. -- **A second npm package** (`indigo-matter-bridge`), installed from its own menu item. - Note it is **not on the npm registry yet**, so export cannot be brought up until it is. +- **A second npm package** (`indigo-matter-bridge`), on the npm registry and installed + from its own menu item. Setup, pairing, recovery and troubleshooting: [full install guide](https://simons-plugins.github.io/indigo-matter/INSTALL.html). How it diff --git a/docs/HANDOVER.md b/docs/HANDOVER.md index ee6cf95..92daf7d 100644 --- a/docs/HANDOVER.md +++ b/docs/HANDOVER.md @@ -1,14 +1,139 @@ # indigo-matter — Build Handover -**Last updated:** 2026-08-06 11:01 UTC -**Active work:** `fix/141-restore-endpoints-at-startup` — issue #141, the -empty-aggregator restart bug. See the section immediately below; the `main` -summary in this header describes the last merge, not that branch. -**Branch:** `main` — PRs #106, #107, #108 all merged. -**Version:** `2026.7.13` -**Tests:** 1005 passing (`cd indigo-matter && /Library/Frameworks/Python.framework/Versions/Current/bin/python3 -m pytest -q`) -**Deployed:** jarvis is running the #103-era plugin (v2026.7.10) plus **E5's export half, deployed 2026-08-05 with 3 accessories live in Apple Home**. **Nothing from #106/#107/#108, and no part of E6 or E7, has been deployed** — so the #104 supervision fixes are NOT yet live on jarvis. -**Status:** **Wi-Fi AND Thread validated with real hardware**, and the **export half's E0 pairing gate PASSED on 2026-08-04** with live two-way control on 2026-08-05 (see *Live validation on jarvis*). #104's three server-supervision faults are fixed and merged (#107). #105's diagnostic is merged (#108) but **#105 stays open** — the bridged-endpoint path still has no real-bridge validation. `domio-code` #236 is fixed and closed (domio-code PR #237). Known-open: #105, plus the older #43, #46, #21–#24. +**Last updated:** 2026-08-06 14:05 UTC +**Active work:** `feat/134-menu-sections` — **PR #144 OPEN**, CI green, awaiting +Simon's go-ahead. Menu grouping only; see the §#134 section below. +**Branch:** `main` — **PR #142 MERGED** (`3e1229d`, merge commit, `[no-release]` +so **no GitHub release was cut** — Simon is cutting that himself). Latest +release remains v2026.7.23. +**Version:** plugin `2026.8.5` on the open branch (`2026.8.4` on `main`), +bridge-node `0.7.0` (**published to npm**; `DEFAULT_INSTALL_SPEC` pins it). +**Tests:** **2256 Python** (2252 on `main`), **383 TS** — both green. +(`python3 -m pytest -q` · `cd bridge-node && npm run build && npm test`) +**Deployed:** jarvis runs plugin `2026.8.4` + bridge-node `0.7.0`, paired to +**Apple Home AND Alexa simultaneously** (3 fabrics: 2 Apple, 1 Alexa). +**Status:** export v1 feature-complete and live. #141 **fixed and confirmed** +(below). **#143 is the open one** — see the 2026-08-06 §#143 section; its +defect B was investigated hard today and the leading theory was **withdrawn**, +so read that before touching it. + +**NEXT UP (Simon, 2026-08-06):** ~~tidy `MenuItems.xml` into groups~~ — **done**, +see the §#134 section immediately below. The constraint it warned about was +wrong in the useful direction: **separators do exist**. + +--- + +## 2026-08-06 — issue #134: the menu, and the separator the docs deny + +Plugin `2026.8.5`. Suites: **2256 Python** (from 2252), pylint **9.42** +unchanged. Branch `feat/134-menu-sections`. TS untouched. + +**An empty, self-closing `` renders as a menu separator.** +The canonical `MenuItems.xml` reference lists `` as *required* and +documents neither separators nor submenus, which is why the note this replaces +said grouping could only be ordering plus name prefixes. It is wrong — or at +least incomplete. Perceptive Automation's own bundled plugins use the form +(**Timers and Pesters**, **Timed Devices**, **Virtual Devices**, **Alexa**), +`Actions.xml` takes it too (``), and +**15 of the plugins installed on jarvis** rely on it. Simon knew from the +Timers and Pesters *action* list that sections were possible; the docs alone +would never have said so. + +Worth generalising: the canonical docs are the authority on what is +*supported*, not on what *works*. When a UI affordance is missing from them, +grep the installed plugins before concluding it does not exist — +`grep -rlE "]*/>" …/Plugins/*/Contents/Server\ Plugin/MenuItems.xml` +answered this in one command. + +**Five sections**, everyday first, destructive last: devices Indigo controls · +devices Indigo publishes · matter-server plumbing · export-bridge plumbing · +backup and recovery. The two `Install/update` items — #134's actual complaint, +clicked for one another live — now sit in **different** sections rather than +merely adjacent, because they are separately versioned, separately installed +npm packages. + +**`Export fabric backup…` → `Back up the Matter fabric…`** (and `Restore +fabric backup…` → `Restore a fabric backup…`). "Export" meant save-to-disk in +one item and the outbound bridge in eight others. Menu **ids and callback +methods are unchanged**, so nothing in `plugin.py`, `bridge_agent.py` or +`server_process.py` moved; `spec.install_menu_name` still matches a real +``, which `test_bridge_agent.py` pins. + +**The tests assert the ORDER**, for the same reason #141's do: a membership +check passes just as happily with all sixteen items back in one flat list. +Mutation-verified both ways — deleting the four separators, and deleting only +`sepServerFromBridge` so the two install sections merge, each fail exactly the +three layout tests and nothing else. + +**Not done here, deliberately:** #132 (the Rebuild Endpoint Map warning +overstates what the action does) is dialog *text* in the same file and a +separate bug — reordering it and rewriting it in one commit would have made +the diff unreviewable. #131 (sort exported devices to the top of the picker) +is untouched. + +**Stale, spotted in passing, not fixed:** `docs/INSTALL.md` §Overview still +says the bridge package "is not on the npm registry yet, so this half cannot +be installed today". `indigo-matter-bridge@0.7.0` is published. That sentence +belongs to the E8 docs pass, along with #137/#138. + +--- + +## 2026-08-06 (late) — issue #143: state wrong in Alexa. READ THIS BEFORE RESUMING IT + +**Status: unresolved, parked for beta testers. The node is EXONERATED.** + +Two halves, one root shape — we lean on matter.js's **attribute-change** +machinery for things that are not changes: + +* **Defect A (inbound), still the substantive half.** Ecosystem commands are + emitted from `onOff$Changed` listeners, so an invoke matching the endpoint's + current state changes nothing and reaches Indigo never. Suspected, **not + proven**. Untouched by today's work. +* **Defect B (outbound).** New exports show stale, then *unresponsive*, in + **Alexa only** (Apple is fine) — then **come right on their own after minutes + with no intervention**. Best current reading: **Alexa-side convergence lag**, + probably not our bug. + +**Do NOT re-assert the "born-vs-changed" theory as proven.** It is real in code +(`createEndpoint` bakes spec states into `initialState`, so an attribute born +`true` emits no change report) and the fix is cheap — construct from role +defaults, then `applyStates` after `aggregator.add()`, exactly as `update()` +already does. But it does **not** explain what was observed: a genuine write +also failed to move Alexa, and Alexa then fixed itself with no write at all. +I posted a "B2 CONFIRMED" verdict on the issue and then **withdrew it**; both +comments are there, read to the end. + +**Measured, so don't re-measure it:** a controlled toggle produced `I/ReportData` +on all three live sessions (1 Apple, 2 Alexa Echo hubs), each acked in ~15ms. +Alexa = fabric index 4, **two wildcard subscriptions**, maxInterval ≈188s. +Outbound reporting to Alexa works, *including* for endpoints added after the +subscription existed. + +**Three blind alleys — do not re-walk:** +1. matter.js does **not** flush a constructor-supplied `initialState` to its + store, so `…/indigo-matter-bridge/root.parts.aggregator.parts..onOff.onOff` + is **NOT** a reading of the live attribute. Two said `false` for accessories + that were on and correct in both apps. Cost about an hour. +2. "the node resurrects a stale persisted value on re-export" — tested three + ways (in-process, across a restart, via `upsert_endpoint`); **all pass**. + Those tests are committed (`e22e861`, `bridge-node/test/restore.test.ts`) and + they are what exonerate the node. The stored value was always right; the bug + was an *unreported* one, which no assertion on `stateOf` can catch. +3. **Never credit the last thing you did.** Something here fixes itself on its + own timescale. "The plugin restart fixed it" was claimed twice and was wrong + both times. Any intervention needs a control before causation is claimed. + +**To settle it** (needs beta testers, more than one Alexa setup): export a +device that is **on**; record Alexa at t+0/+1/+5/+15 min **without touching +anything**; watch `root.subscriptions.subscriptions` for a re-subscribe at the +moment it comes right. Then repeat with the create-path fix applied — if Alexa +is right at t+0 the fix matters, if the curve is unchanged it is cosmetic. + +**Also open, adjacent:** #140 — the drift line +(`indigo-459564566 expected 5, got 2`) now fires on **every** attach after the +factory reset, and `endpoint-map.json` records number `5` twice (Hallway Lamp +and Georges Pendant). Harmless — live numbers are unique — but it is the reset's +fingerprint and belongs in #140. --- diff --git a/docs/INSTALL.md b/docs/INSTALL.md index 9939f19..336ec5b 100644 --- a/docs/INSTALL.md +++ b/docs/INSTALL.md @@ -9,7 +9,8 @@ npm package**. This guide covers both: managed-LaunchAgent bring-up on the reference Indigo server. - **the Matter export bridge** — the optional outbound half, which publishes selected *Indigo* devices to Apple Home as Matter accessories. **Steps E1–E4**. - **It cannot be installed today**: its npm package has not been published — see + Its npm package (`indigo-matter-bridge`) is on the registry and installs from + its own menu item — see [Exporting Indigo devices](#exporting-indigo-devices-indigo-as-a-matter-bridge). If you just installed the plugin and Indigo is logging @@ -18,7 +19,8 @@ running but matter-server is not. ## Contents -- [Overview](#overview) +- [Overview](#overview) — including + [the Plugins ▸ Matter menu](#the-plugins--matter-menu) - [Prerequisites](#prerequisites) - [Step 1 — Install Node.js 22](#step-1--install-nodejs-22) - [Step 2 — Install matter-server](#step-2--install-matter-server) @@ -89,12 +91,27 @@ A few things to know before you start: occasional rough edges. - **There is a second, optional half.** The plugin can also work the other way round and publish selected *Indigo* devices to Apple Home as Matter accessories, from a separate - bridge process with its own npm package. **That package is not on the npm registry yet, - so this half cannot be installed today.** When it is: nothing is exported and no bridge - process runs until you add a device to the export list, and it needs Node (Step 1) and - a storage path (Step 3) — *not* a working matter-server. See + bridge process with its own npm package (`indigo-matter-bridge`), installed from + **Plugins ▸ Matter ▸ Install/update the Matter export bridge**. Nothing is exported + and no bridge process runs until you add a device to the export list, and it needs + Node (Step 1) and a storage path (Step 3) — *not* a working matter-server. See [Exporting Indigo devices](#exporting-indigo-devices-indigo-as-a-matter-bridge). +### The Plugins ▸ Matter menu + +The menu is grouped into five sections, separated by dividers. Both halves of the +plugin have their own install item, and they are **not** interchangeable: they are +separately versioned npm packages, and running the wrong one restarts a half you +did not mean to touch. + +| Section | Items | What it covers | +|---|---|---| +| 1 | *Commission device by setup code (advanced)…*, *Decommission Matter device…* | Matter devices Indigo **controls** — the everyday actions | +| 2 | *Manage Matter Exports…*, *Pair Matter Bridge…*, *Unpair an Ecosystem…* | Indigo devices Indigo **publishes** — the everyday actions | +| 3 | *Install/update matter-server*, *Restart…*, *Reinstall (clean)…*, *Open matter-server log…* | The inbound controller's own plumbing | +| 4 | *Install/update the Matter export bridge*, *Reinstall… (clean)…*, *Stop…* | The outbound bridge node's own plumbing | +| 5 | *Back up the Matter fabric…*, *Restore a fabric backup…*, *Rebuild Matter Endpoint Map…*, *Reset Matter Export Pairings…* | Backup and recovery. The last two are destructive — see [the two destructive recovery actions](#the-two-destructive-recovery-actions) | + --- ## Prerequisites @@ -571,9 +588,9 @@ This directory holds the fabric root CA private key and every device's operation **Losing it means re-commissioning every Matter device from scratch.** Indigo's own database backup does **not** cover it. -Use **Plugins ▸ Matter ▸ Export fabric backup…**. It writes a timestamped zip into a +Use **Plugins ▸ Matter ▸ Back up the Matter fabric…**. It writes a timestamped zip into a `backups/` directory beside the storage dir and prunes old ones, and -**Restore fabric backup…** puts one back (moving the current fabric aside first, so a +**Restore a fabric backup…** puts one back (moving the current fabric aside first, so a bad restore is reversible). Copy the zips somewhere off-machine as well — a backup on the same disk is not a backup. @@ -607,7 +624,7 @@ Losing *that* is the number-one real-world cause of duplicated accessories: ever ecosystem re-creates every accessory, and their names, rooms and automations go with the old ones. -**Export fabric backup…** includes this directory in the zip. **Restore does not +**Back up the Matter fabric…** includes this directory in the zip. **Restore does not yet put it back** — it reports the bridge files it found and skips them, because restoring them safely needs the bridge node stopped and that wiring is a follow-up. So for now, if you need to restore the bridge side, copy diff --git a/docs/MATTER.md b/docs/MATTER.md index 9cd8ee1..e08ef06 100644 --- a/docs/MATTER.md +++ b/docs/MATTER.md @@ -121,7 +121,7 @@ leaves the others untouched; only a factory reset wipes them all. The plugin's fabric lives inside matter-server's storage directory. It is the single point of total loss — if it's destroyed, every device must be re-commissioned — which is why the plugin ships fabric **backup and restore** -(menu items: *Export fabric backup…* / *Restore fabric backup…*). +(menu items: *Back up the Matter fabric…* / *Restore a fabric backup…*). > **Why Apple Home calls our fabric "Matter Test" — this is expected.** When the > plugin joins a device, Apple Home shows a "joined a new network" notification @@ -576,7 +576,7 @@ Common failure modes, in rough order of likelihood: - **Capability missing (no energy states, no colour, …).** Firmware (see above), then diagnostics to inspect advertised clusters. -**Backups.** Export a fabric backup after commissioning anything you'd hate to +**Backups.** Back up the fabric after commissioning anything you'd hate to re-pair. The plugin keeps rotating zips and a restore is menu-driven and reversible. diff --git a/docs/matter.html b/docs/matter.html index ecc6b61..a726a54 100644 --- a/docs/matter.html +++ b/docs/matter.html @@ -148,8 +148,8 @@

Fabrics & multi-admin: one device, many controllers

others untouched; only a factory reset wipes them all.

The plugin’s fabric lives inside matter-server’s storage directory. It is the single point of total loss — destroyed, every device must be re-commissioned — which is why the - plugin ships fabric backup and restore (menu items: Export fabric - backup… / Restore fabric backup…).

+ plugin ships fabric backup and restore (menu items: Back up the + Matter fabric… / Restore a fabric backup…).

Expected behaviour Why Apple Home calls our fabric “Matter Test”. When the plugin joins a @@ -364,7 +364,7 @@

Troubleshooting

Persistently dead usually means a factory reset wiped the fabric — re-commission.
  • Capability missing. Firmware first (§ 10), then diagnostics.
    • -

    Backups. Export a fabric backup after commissioning anything you’d hate +

    Backups. Back up the fabric after commissioning anything you’d hate to re-pair. The plugin keeps rotating zips; restore is menu-driven and reversible.

    diff --git a/indigo-matter.indigoPlugin/Contents/Info.plist b/indigo-matter.indigoPlugin/Contents/Info.plist index ecf99a1..b176d2a 100644 --- a/indigo-matter.indigoPlugin/Contents/Info.plist +++ b/indigo-matter.indigoPlugin/Contents/Info.plist @@ -20,7 +20,7 @@ IwsApiVersion 1.0.0 PluginVersion - 2026.8.4 + 2026.8.5 ServerApiVersion 3.6 diff --git a/indigo-matter.indigoPlugin/Contents/Server Plugin/MenuItems.xml b/indigo-matter.indigoPlugin/Contents/Server Plugin/MenuItems.xml index 0051392..d8018f6 100644 --- a/indigo-matter.indigoPlugin/Contents/Server Plugin/MenuItems.xml +++ b/indigo-matter.indigoPlugin/Contents/Server Plugin/MenuItems.xml @@ -1,29 +1,35 @@ + - - Install/update matter-server - menuInstallMatterServer - - - Restart matter-server - menuRestartMatterServer - - - Reinstall matter-server (clean)… - menuReinstallMatterServerClean - - - - - - - - - - - Open matter-server log… - menuShowMatterServerLogs - + Commission device by setup code (advanced)… menuCommissionDeviceManually @@ -57,10 +63,14 @@ + + + - - Rebuild Matter Endpoint Map… - menuRebuildEndpointMap - - - - - - - - - - - Reset Matter Export Pairings… - menuResetBridgePairings - - - - - - - - - - - - + + Install/update matter-server + menuInstallMatterServer + + + Restart matter-server + menuRestartMatterServer + + + Reinstall matter-server (clean)… + menuReinstallMatterServerClean + + + + + + + + + + + Open matter-server log… + menuShowMatterServerLogs + + + + + Install/update the Matter export bridge menuInstallBridgeNode - Reinstall the Matter export bridge (clean)… menuReinstallBridgeNodeClean @@ -296,12 +297,31 @@ You normally never need this: the bridge starts and stops with your export list. + + + + - Export fabric backup… + Back up the Matter fabric… menuExportFabricBackup - Restore fabric backup… + Restore a fabric backup… menuRestoreFabricBackup @@ -316,4 +336,41 @@ You normally never need this: the bridge starts and stops with your export list. + + Rebuild Matter Endpoint Map… + menuRebuildEndpointMap + + + + + + + + + + + Reset Matter Export Pairings… + menuResetBridgePairings + + + + + + + + + + + + diff --git a/tests/test_plugin_module.py b/tests/test_plugin_module.py index 010bc6c..4f4fb30 100644 --- a/tests/test_plugin_module.py +++ b/tests/test_plugin_module.py @@ -106,6 +106,94 @@ def test_dynamic_list_methods_exist_on_plugin(plugin_cls): assert checked >= 2 # getFabricBackups + getMatterNodes at minimum +# --------------------------------------------------------------------------- +# issue #134 — the Plugins ▸ Matter menu's five sections +# --------------------------------------------------------------------------- + +MENU_SECTIONS = [ + # 1 · Matter devices Indigo controls + ["Commission device by setup code (advanced)…", + "Decommission Matter device…"], + # 2 · Matter devices Indigo publishes + ["Manage Matter Exports…", + "Pair Matter Bridge…", + "Unpair an Ecosystem…"], + # 3 · matter-server + ["Install/update matter-server", + "Restart matter-server", + "Reinstall matter-server (clean)…", + "Open matter-server log…"], + # 4 · the export bridge node + ["Install/update the Matter export bridge", + "Reinstall the Matter export bridge (clean)…", + "Stop the Matter export bridge…"], + # 5 · backup and recovery + ["Back up the Matter fabric…", + "Restore a fabric backup…", + "Rebuild Matter Endpoint Map…", + "Reset Matter Export Pairings…"], +] + + +def _menu_items(): + return ET.parse(SERVER_PLUGIN / "MenuItems.xml").getroot().findall("MenuItem") + + +def _menu_sections() -> list[list[str]]: + """MenuItems.xml split on its separators. + + A separator is an EMPTY, self-closing ```` — undocumented + (the canonical reference lists ```` as required) but what Perceptive + Automation's own bundled plugins use. + """ + sections: list[list[str]] = [[]] + for item in _menu_items(): + name = item.findtext("Name") + if name is None: + sections.append([]) + else: + sections[-1].append(name) + return sections + + +def test_menu_is_grouped_into_the_documented_sections(): + """The ORDER is the change (#134), so the order is what is asserted. + + A membership-only check would pass just as happily with all sixteen items + back in one flat list, which is the state this exists to prevent. + """ + assert _menu_sections() == MENU_SECTIONS + + +def test_the_two_install_items_are_separated(): + """#134: 'Install/update matter-server' and 'Install/update the Matter + export bridge' sat eleven apart in one flat list and were clicked for one + another live — reinstalling the inbound controller when the outbound + bridge was meant. They are separately versioned, separately installed npm + packages, so they must not share a section.""" + sections = _menu_sections() + controller = next(i for i, s in enumerate(sections) if "Install/update matter-server" in s) + bridge = next(i for i, s in enumerate(sections) + if "Install/update the Matter export bridge" in s) + assert controller != bridge + + +def test_separators_carry_nothing_but_a_unique_id(): + """Give a separator a and it becomes a clickable item that does + nothing; give it a CallbackMethod and the wiring test above starts + demanding a method that should not exist.""" + separators = [i for i in _menu_items() if i.findtext("Name") is None] + assert len(separators) == len(MENU_SECTIONS) - 1 + for sep in separators: + assert sep.get("id"), "a separator still needs an id" + assert list(sep) == [], f"separator {sep.get('id')} must be empty" + + +def test_every_menu_item_id_is_unique(): + ids = [i.get("id") for i in _menu_items()] + assert len(ids) == len(set(ids)), f"duplicate MenuItem ids: {ids}" + + def test_relay_devices_do_not_redefine_native_onoffstate(): """onOffState is native to the relay base type — redefining it in makes Indigo reject the device ("native state keys cannot be overriden").