From 0a95103bfd6a8030011d06cb4ed17cfbc9e48b3f Mon Sep 17 00:00:00 2001 From: Simon Clark Date: Tue, 4 Aug 2026 18:41:22 +0100 Subject: [PATCH 1/2] docs(export): PRD research pass + bridge protocol spec Fold the 2026-08-04 research findings into the export PRD and add BRIDGE_PROTOCOL.md, the plugin<->bridge-node local protocol contract both the TypeScript node and the Python client will be built against. PRD changes: - Status: Accepted, build in progress; decisions of 2026-08-04 recorded - Endpoint identity: matter.js keys persisted endpoint numbers on the string Endpoint.id (verified at 0.17.8) -> id derives from the Indigo device ID; drift detector + storage-loss refuse-to-start required - Matter side binds UDP 5540 with the Aggregator at EP1 (Alexa constraint, cheap now, painful to retrofit); no conflict with matter-server (TCP 5580) - Valve and Fan roles descoped to v2: their matter.js clusters are stubs and ecosystem support is poor - Pressure/Flow sensors kept, Apple-ignores caveat documented - Pairing UX: manual code via event log, QR via IWS page; passcode and discriminator randomised per install - XOQ3 answered (AgentSpec extraction, pulled forward to before E1), XOQ4 answered (matter.js generates a self-signed chain for the configured VID; examples' 0xFFF1/0x8000), XOQ6 answered (~145MB floor, ~0.3MB/endpoint, measured) - Bridge node ships as an exact-pinned published npm package (indigo-matter-bridge); matter.js itself exact-pinned Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01JxqGhP3DcENf68AZK21U4S --- docs/BRIDGE_PROTOCOL.md | 229 ++++++++++++++++++ docs/PRD-indigo-matter-export.md | 183 ++++++++++---- .../Contents/Info.plist | 2 +- 3 files changed, 369 insertions(+), 45 deletions(-) create mode 100644 docs/BRIDGE_PROTOCOL.md diff --git a/docs/BRIDGE_PROTOCOL.md b/docs/BRIDGE_PROTOCOL.md new file mode 100644 index 0000000..a05911b --- /dev/null +++ b/docs/BRIDGE_PROTOCOL.md @@ -0,0 +1,229 @@ +# BRIDGE_PROTOCOL.md — plugin ⇄ bridge-node local protocol + +**Version:** 1 (`protocolVersion: 1`) +**Transport:** WebSocket, JSON text frames, loopback only +**Peers:** the Indigo plugin (client) and the `indigo-matter-bridge` node +(server). We author both ends and ship them together. +**Governing docs:** [`PRD-indigo-matter-export.md`](./PRD-indigo-matter-export.md) §4.4, +ADR-0006. + +This is the outbound twin of the controller protocol that `protocol.py` / +`matter_client.py` speak. It reuses the same envelope grammar so the client +machinery and test doubles generalise, but it is **not** matter-server's +protocol and has no rename firewall: a field rename here is a coordinated edit +to both peers in one release. What protects us instead is the **version +handshake** — launchd deliberately keeps an old bridge node running across +plugin reloads (PM-B), so plugin/node version skew is the failure mode that +will actually occur. + +## 1. Envelope grammar + +Identical shapes to the controller protocol: + +| Kind | Direction | Shape | +|---|---|---| +| Request | plugin → node | `{"message_id": "", "command": "", "args": {…}}` | +| Success response | node → plugin | `{"message_id": "", "result": …}` | +| Error response | node → plugin | `{"message_id": "", "error_code": "", "details": ""}` | +| Event | node → plugin | `{"event": "", "data": {…}}` (no `message_id`) | + +`message_id` is an opaque string chosen by the plugin, echoed verbatim. +Unknown *fields* are ignored by both peers (forward compatibility inside a +protocol version). Unknown *commands* get `error_code: "unknown_command"`. +Unknown *events* are logged and dropped by the plugin. + +## 2. Handshake + +On every new connection, before anything else: + +1. Node sends a bare frame (not an event, mirroring `server_info`): + `{"protocolVersion": 1, "bridgeVersion": "", "matterJsVersion": ""}` +2. Plugin sends `attach` (§3.1). If `protocolVersion` differs from the + plugin's own, the plugin does **not** attach: it surfaces an error telling + the user to restart/update the bridge agent (typically the plugin was + updated while launchd kept the old node alive). The node accepts exactly + one attached client at a time; a second `attach` supersedes the first + (the old socket is closed) — the plugin's reconnect loop relies on this. + +## 3. Commands (plugin → node) + +### 3.1 `attach` + +Declares the client and delivers the desired endpoint set in one shot. + +```json +{"command": "attach", "args": { + "protocolVersion": 1, + "pluginVersion": "2026.8.1", + "endpoints": [ , … ] +}} +``` + +Result: `{"status": }` (§5.1). The node reconciles its live +endpoint set against `endpoints` — creating, updating and removing as needed — +so a fresh connection is always a full reconcile (PRD §5.4 `startup`). + +### 3.2 `upsert_endpoint` + +```json +{"command": "upsert_endpoint", "args": {"endpoint": }} +``` + +Creates the endpoint if absent, updates label/reachable/state if present. +Idempotent. Result: `{"endpointNumber": }`. + +### 3.3 `remove_endpoint` + +```json +{"command": "remove_endpoint", "args": {"indigoDeviceId": 123456789}} +``` + +Removes the child endpoint (`endpoint.close()`); the persisted endpoint-number +allocation is **retained** so re-adding the same device restores the same +number. Idempotent — removing an absent endpoint succeeds. Bulk removals are +paced ~100ms apart by the node. + +### 3.4 `set_state` + +```json +{"command": "set_state", "args": {"indigoDeviceId": 123456789, "states": {"onOff": true}}} +``` + +Pushes Indigo-originated state outward. `states` keys are role-specific (§4.2). +The node applies them as **local** (offline-context) writes so they are not +echoed back as commands. The plugin sends this fire-and-forget (it must never +block Indigo's device thread on the result). + +### 3.5 `set_reachable` + +```json +{"command": "set_reachable", "args": {"indigoDeviceId": 123456789, "reachable": false}} +``` + +Split from `set_state` because it maps to Bridged Device Basic Information, +not the functional cluster, and is driven by device enable/disable rather than +state change. + +### 3.6 `get_status` + +Result: `` (§5.1). Used by the watchdog tick and the §5.5 +config readout. + +### 3.7 `get_pairing` + +Result: +```json +{"commissioned": false, + "manualPairingCode": "34970112332", + "qrPairingCode": "MT:Y.K90IRV01KA0648G00", + "fabrics": [ , … ]} +``` + +### 3.8 `remove_fabric` + +```json +{"command": "remove_fabric", "args": {"fabricIndex": 2}} +``` + +### 3.9 `factory_reset` + +Wipes commissioning credentials and starts advertising fresh (PRD §6 "reset +all pairings"). The endpoint-number map is **preserved** — a reset must not +scramble identities if the user re-pairs the same ecosystems. + +## 4. Shapes + +### 4.1 `EndpointSpec` + +```json +{"indigoDeviceId": 123456789, + "role": "onOffLight", + "label": "Kitchen Lamp", + "reachable": true, + "states": {"onOff": true}, + "options": {}} +``` + +- `indigoDeviceId` — the immutable Indigo device ID. The node derives + `Endpoint.id` and `UniqueID` from it (PRD §4.3); it is the identity key + everywhere in this protocol. +- `role` — one of the v1 role enum (§4.2). A role change for an existing + endpoint is **rejected** (`error_code: "role_change"`); the plugin must + remove and re-add, because ecosystems cache device types per endpoint. +- `label` — Bridged Device Basic Information `NodeLabel`. +- `options` — role-specific extras (e.g. window-covering polarity). + +### 4.2 Roles and their state keys (v1) + +| `role` | Matter device type | `states` keys | +|---|---|---| +| `onOffPlugInUnit` | On/Off Plug-in Unit | `onOff: bool` | +| `onOffLight` | On/Off Light | `onOff: bool` | +| `dimmableLight` | Dimmable Light | `onOff: bool`, `level: 0-100` | +| `colorTemperatureLight` | Color Temperature Light | + `colorTempMireds: int` | +| `extendedColorLight` | Extended Color Light | + `hue: 0-360`, `saturation: 0-100` | +| `windowCovering` | Window Covering | `position: 0-100` (100 = open, inbound convention; polarity in `options`) | +| `doorLock` | Door Lock | `locked: bool` | +| `occupancySensor` | Occupancy Sensor | `occupied: bool` | +| `contactSensor` | Contact Sensor | `contact: bool` (true = closed) | +| `temperatureSensor` | Temperature Sensor | `temperatureC: float` | +| `humiditySensor` | Humidity Sensor | `humidityPct: float` | +| `lightSensor` | Light Sensor | `lux: float` | +| `pressureSensor` | Pressure Sensor | `pressureKPa: float` | +| `flowSensor` | Flow Sensor | `flowM3h: float` | +| `thermostat` | Thermostat | `localTemperatureC`, `heatingSetpointC`, `coolingSetpointC`, `systemMode` | + +Units are Indigo-natural at the protocol boundary (°C, %, lux, 0–100 levels); +the node owns the conversion to Matter wire units (0.01°C, mireds, the +illuminance log scale, 0–254 levels). Exactly one converter per role, in the +node, next to the cluster it feeds. + +### 4.3 `StatusReport` (§5.1) and `FabricInfo` + +```json +{"commissioned": true, + "fabrics": [{"fabricIndex": 1, "label": "Apple Home", "vendorId": 4937}], + "endpointCount": 12, + "endpoints": [{"indigoDeviceId": 123456789, "endpointNumber": 2, "role": "onOffLight"}], + "drift": []} +``` + +`drift` lists any `UniqueID → endpointNumber` mappings that changed since last +persist (PRD §4.3 drift detection); non-empty drift is surfaced as a plugin +error, never auto-repaired. + +## 5. Events (node → plugin) + +| Event | `data` | Meaning | +|---|---|---| +| `command` | `{"indigoDeviceId", "command", "args"}` | Ecosystem-originated action, e.g. `{"command": "onOff", "args": {"value": true}}`, `{"command": "moveToLevel", "args": {"level": 40}}`, `{"command": "lock"}` | +| `fabrics_changed` | `{"fabrics": […], "change": "added"\|"deleted"\|"updated"}` | Pairing/unpairing activity | +| `commissioned` / `decommissioned` | `{}` | First fabric added / last removed | +| `drift_detected` | `{"drift": […]}` | Endpoint-number drift found at startup | + +`command` events carry the same role-relative vocabulary as `set_state` keys. +The plugin resolves `indigoDeviceId` through the allow-list before acting; a +command for a device no longer exported gets no action and a warning (PRD §7 +race row — the node responds to the ecosystem with failure when the endpoint +is already gone). + +## 6. Invariants + +1. **Loopback only.** The node binds `127.0.0.1`; there is no auth on the + socket because there is no remote surface. +2. **The plugin is the source of truth for the export set**; the node is the + source of truth for commissioning state and endpoint numbers. Neither peer + caches the other's domain across reconnects — `attach` reconciles. +3. **Identity flows one way:** `indigoDeviceId` → `Endpoint.id` → persisted + endpoint number. Nothing is ever keyed on list position or label. +4. **State pushes are echo-guarded** in the node (`ctx.offline`); the plugin + never receives a `command` event for a change it pushed. +5. **Version skew fails closed:** mismatched `protocolVersion` means no + attach, an error in the Indigo log, and untouched pairings. + +## 7. Testing contract + +Golden frames for every command/response/event pair live beside the Python +tests (the `test_golden_real.py` pattern) and are the cross-language contract: +the TypeScript node's protocol tests consume the same fixtures, so a frame +change that only updates one side fails that side's suite. diff --git a/docs/PRD-indigo-matter-export.md b/docs/PRD-indigo-matter-export.md index eadb6ee..cf6d7bd 100644 --- a/docs/PRD-indigo-matter-export.md +++ b/docs/PRD-indigo-matter-export.md @@ -1,10 +1,12 @@ # PRD — Indigo Matter Export (Indigo as a Matter bridge) -**Status:** Draft — scoping +**Status:** Accepted — build in progress **Owner:** Simon **Governing ADR:** [`../../docs/adr/0006-indigo-as-matter-bridge.md`](../../docs/adr/0006-indigo-as-matter-bridge.md) (accepted 2026-08-03) **Companion PRD:** [`PRD-indigo-matter-plugin.md`](./PRD-indigo-matter-plugin.md) (historical — the inbound/controller build) -**Last updated:** 2026-08-03 +**Local protocol spec:** [`BRIDGE_PROTOCOL.md`](./BRIDGE_PROTOCOL.md) +**Last updated:** 2026-08-04 (research pass: matter.js 0.17.8 verified by execution; scope and +packaging decisions taken — see §5.2 exclusions, §10, §11) ## 1. Summary @@ -105,8 +107,22 @@ from the controller agent: - **Endpoint stability (XG4)** is a hard requirement. Endpoint IDs and Bridged Device Basic Information `UniqueID` values MUST be allocated once, persisted, and derived from the Indigo device ID — never from list position or iteration - order. A monotonic persisted allocator, mirroring `MatterFabricStore.nextNodeID()` - in the Domio work, is the known-good shape. + order. matter.js keys its persisted endpoint numbers solely on the string + `Endpoint.id` (verified at 0.17.8): a stable `id` gives stable numbers across + restarts, reorderings and removals, while an omitted `id` falls back to + positional `part0/part1/…` and silently swaps identities when a device is + removed. Therefore: `id` = the immutable Indigo device ID (sanitised), + supplied explicitly, never reused, never mutated. +- **Drift detection.** The bridge node persists a `UniqueID → endpoint number` + map and warns loudly if any mapping changes at startup (matterbridge's + `checkEndpointNumbers()` pattern) — the only way to make "is it us or the + controller?" falsifiable in the field. +- **Storage loss is the #1 real-world accessory-duplication cause** (not logic + bugs): a missing/relocated storage dir reallocates every endpoint number and + every ecosystem re-creates every accessory, losing names, rooms and + automations. The storage path must survive plugin and Indigo upgrades, is + backed up alongside the controller's, and "storage missing but previously + commissioned" is a loud refuse-to-start (§7), never a silent re-init. ### 4.4 Local protocol @@ -117,6 +133,19 @@ state changes outward, and the bridge node pushes ecosystem commands inward. nothing new to learn, `matter_client.py` is a working reference for the client half, and the existing test doubles generalise. The bridge node listens on loopback only, on a configurable port defaulting adjacent to the controller's. +Unlike `protocol.py` there is **no rename firewall** — we own both ends and ship +them together — but the handshake carries a `protocolVersion`, because launchd +deliberately keeps the old node running across plugin reloads and version skew +is the failure that will actually happen. Full spec: [`BRIDGE_PROTOCOL.md`](./BRIDGE_PROTOCOL.md). + +The **Matter side** of the node binds UDP **5540** (the Matter default) with the +Aggregator at **Endpoint 1**. This is Alexa's hard requirement (it discovers +nothing on any other port, and needs EP1 to be the aggregator); Alexa is +unclaimed (XOQ1) but the constraint costs nothing now and is painful to +retrofit. No conflict with the controller: `matter-server` listens on TCP 5580 +plus ephemeral UDP, not 5540. mDNS is pinned to the primary interface (reusing +the `primaryInterface` pref) — matter.js's own mDNS stack defaults to all +interfaces and breaks on Macs with VPN/utun interfaces. ## 5. Components @@ -163,22 +192,22 @@ safest interpretation (plug/light) rather than guessing. |---|---|---|---| | Relay | Plug *(default)* | On/Off Plug-in Unit | Safest default | | Relay | Light | On/Off Light | | -| Relay | Lock | Door Lock | Requires ecosystem PIN/confirm semantics; see §7 | -| Relay | Valve | Water Valve | Inherit inbound's flood-safe toggle behaviour | +| Relay | Lock | Door Lock | Requires ecosystem PIN/confirm semantics; see §7. matter.js's DoorLock implementation is its most complete (users/credentials/schedules, encrypted at rest) — the risk is ecosystem UX, not the library | +| Relay | Valve | *Not exportable in v1* | Descoped 2026-08-04; see below | | Relay | Garage door | *Not exportable in v1* | Polarity + safety; see below | | Dimmer | Light | Dimmable Light | | | Dimmer (colour) | Light | Extended Color Light | Colour-temp-only devices → Color Temperature Light | -| Dimmer | Window covering | Window Covering | Polarity declared per export (100% = open, inbound convention) | -| Dimmer | Fan | Fan | Where the Indigo device is a fan modelled as a dimmer | +| Dimmer | Window covering | Window Covering | Polarity declared per export (100% = open, inbound convention). Must implement `handleMovement()` — matter.js's default snaps to target instantly | +| Dimmer | Fan | *Not exportable in v1* | Descoped 2026-08-04; see below | | Sensor (binary, motion) | — | Occupancy Sensor | | | Sensor (binary, contact) | — | Contact Sensor | | | Sensor (numeric, °C) | — | Temperature Sensor | | | Sensor (numeric, %RH) | — | Humidity Sensor | | | Sensor (numeric, lux) | — | Light Sensor | | -| Sensor (numeric, pressure) | — | Pressure Sensor | | -| Sensor (numeric, flow) | — | Flow Sensor | | -| Thermostat | — | Thermostat | Setpoints, modes; fan merged if present | -| SpeedControl | Fan | Fan | Map speed index → percent | +| Sensor (numeric, pressure) | — | Pressure Sensor | Apple Home ignores this type (Google supports it); exported anyway, documented | +| Sensor (numeric, flow) | — | Flow Sensor | Apple Home ignores this type (Google supports it); exported anyway, documented | +| Thermostat | — | Thermostat | Setpoints, modes; fan merged if present. matter.js provides the cluster machinery; the HVAC logic is ours | +| SpeedControl | Fan | *Not exportable in v1* | Descoped 2026-08-04; see below | Matter device-type IDs are deliberately omitted here; take them from the matter.js device-type catalogue at implementation rather than transcribing @@ -189,7 +218,9 @@ them into a PRD where they can rot. | Excluded | Why | |---|---| | Any device created by `indigo-matter` | Loop guard (XNG3), enforced at §5.1 | -| Sprinkler devices | Matter has no irrigation-controller type; per-zone Water Valve is a lossy fit. v2 candidate | +| Valve role | **Descoped 2026-08-04.** matter.js's `ValveConfigurationAndControlServer` is an empty stub — the whole command surface would be ours to implement — and ecosystem support is poor (Apple unresolved, Alexa ignores the type). v2 candidate | +| Fan role (Dimmer- or SpeedControl-backed) | **Descoped 2026-08-04.** matter.js's `FanControlServer` only seeds a default `fanMode`; all fan behaviour would be ours to implement. v2 candidate | +| Sprinkler devices | Matter has no irrigation-controller type; per-zone Water Valve is a lossy fit (and Water Valve itself is descoped). v2 candidate | | MultiIO devices | No coherent single-accessory representation | | `custom` devices with no resolvable role | Includes the plugin's own energy-meter type | | Garage doors | Needs the polarity handling the catalog doesn't yet carry (`onState` true = closed, turnOn = close), and mis-mapping is a physical-safety issue. Blocked on the catalog role/polarity work | @@ -205,13 +236,30 @@ Node** child endpoint per exported device carrying Bridged Device Basic Information (`NodeLabel`, `Reachable`, `UniqueID`) — the standard bridge topology the plugin already consumes inbound (`MATTER.md:238-244`). -- **Max fabrics:** set to allow at least 5 concurrent ecosystems. +- **Distribution:** the bridge node is a **published npm package** + (`indigo-matter-bridge`, TypeScript, decided 2026-08-04), exact-pinned by the + plugin the same way `matter-server@1.2.2` is — the existing + `npm install --prefix` machinery works unchanged. matter.js itself is + **exact-pinned** (no caret): patch releases have changed what Apple Home + renders with zero code change on the bridge side. +- **Max fabrics:** matter.js defaults `supportedFabrics` to 254, so ≥5 + concurrent ecosystems needs no action. Note Apple consumes **two** slots + (iCloud Keychain sync). - **Reachable** must track the Indigo device's enabled/available state so - ecosystems grey out unavailable accessories instead of timing out. + ecosystems grey out unavailable accessories instead of timing out. Prefer + `Reachable = false` over endpoint removal for anything temporary. - **Removal:** dropping a device from the allow-list removes its endpoint and - updates the aggregator's `PartsList`; ecosystems remove the accessory. + updates the aggregator's `PartsList` (automatic in matter.js); ecosystems + remove the accessory. Bulk removals are rate-limited (~100ms apart, + matterbridge's pattern) so controllers see one subscription update each. +- **Echo guard:** matter.js attribute-change events fire for our own + Indigo-originated writes as well as controller commands; the bridge node + discriminates on the event context (`ctx.offline`) or the Indigo↔ecosystem + loop is infinite. - **Endpoint count:** no hard cap in v1, but log a warning past ~100 exports — - ecosystem per-home accessory limits (Apple's in particular) will bite first. + ecosystem per-home accessory limits will bite first (Alexa hard-caps at 50 + bridged devices; Apple degrades past ~200). Memory is not the constraint: + measured ~145MB RSS floor + ~0.3MB per endpoint at 0.17.8 (XOQ6 answered). ### 5.4 Lifecycle hooks @@ -233,6 +281,12 @@ fabric slots used/remaining). Per-export settings live in the §5.1 dialog. - **Pair:** a menu action surfaces the bridge node's setup code and QR payload. The user adds it in each ecosystem's app as they would any Matter accessory. + **Mechanism** (Indigo dialogs have no dynamic labels and no image fields): + the manual code is written to the event log — the plugin's established + pattern for runtime strings — and the QR is rendered on an IWS-served page, + reachable from the same menu action. Passcode and discriminator are + **randomised per install** (identical passcodes produce identical pairing + codes across installs — verified) and persisted by the bridge node. - **Uncertified prompt:** expect an uncertified-accessory warning in every ecosystem (ADR-0006). Document it as *expected* in `INSTALL.md`, with the Homebridge parallel, so it doesn't read as a fault. @@ -256,8 +310,10 @@ fabric slots used/remaining). Per-export settings live in the §5.1 dialog. - **XAC1.** Fresh install: no bridge process running, nothing paired, nothing exported. - **XAC2.** Exporting one relay starts the bridge node and yields a pairing code. -- **XAC3.** Bridge pairs into **Apple Home and at least one non-Apple ecosystem** - from the same code, both controlling the device. +- **XAC3.** Bridge pairs into **Apple Home** from the displayed code and + controls the device. Multi-fabric capability is proven by adding a **second + admin we already own** — Domio's own fabric (ADR-0005) or a matter-server + controller — not by requiring a third-party ecosystem we cannot test (XOQ1). - **XAC4.** Command round-trip both directions within 500ms (XG2). - **XAC5.** Endpoint IDs and `UniqueID`s survive plugin reload, bridge-node restart and Mac reboot with no accessory duplication (XG4). @@ -275,47 +331,86 @@ fabric slots used/remaining). Per-export settings live in the §5.1 dialog. | # | Milestone | Gating criterion | |---|---|---| -| E0 | Bridge node skeleton | Node process starts, exposes an aggregator with one hard-coded endpoint, pairs into Apple Home | -| E1 | **Google Home pairing spike** | Determines whether an uncertified bridge pairs at all. **Runs before E2** — it gates what v1 can advertise | -| E2 | Local protocol + plugin client | Plugin drives endpoint create/remove over WS | -| E3 | Allow-list + UI-D dialog | Devices selectable with role; loop guard live (XAC6, XAC9) | -| E4 | Relay + dimmer export | XAC2, XAC3, XAC4 | -| E5 | Sensors + thermostat export | Mapping table complete for v1 | -| E6 | Endpoint persistence | XAC5 — the highest-risk correctness requirement | -| E7 | Pairing/unpairing UX + fabric readout | §6 complete | -| E8 | launchd agent + failure recovery | §7, XAC7, XAC8 | -| E9 | Docs | `INSTALL.md` export section, uncertified-prompt explanation, `MATTER.md` outbound architecture | - -E0 and E1 are the validation loop. Until E1 resolves, v1's ecosystem claims are -unknown; everything after E2 is mechanical. +| E0 | Bridge node skeleton — **the validation gate** | Node process starts, exposes an aggregator with one hard-coded endpoint, and **pairs into Apple Home**. If an uncertified bridge will not pair here, the design is dead and nothing after this matters | +| E1 | Local protocol + plugin client | Plugin drives endpoint create/remove over WS | +| E2 | Allow-list + UI-D dialog | Devices selectable with role; loop guard live (XAC6, XAC9) | +| E3 | Relay + dimmer export | XAC2, XAC3, XAC4 | +| E4 | Sensors + thermostat export | Mapping table complete for v1 | +| E5 | Endpoint persistence | XAC5 — the highest-risk correctness requirement | +| E6 | Pairing/unpairing UX + fabric readout | §6 complete | +| E7 | launchd agent + failure recovery | §7, XAC7, XAC8 | +| E8 | Docs | `INSTALL.md` export section, uncertified-prompt explanation, ecosystems-untested note (§10), `MATTER.md` outbound architecture | + +**E0 is the whole validation loop.** It answers the only question that can kill +the feature — *will any ecosystem pair an uncertified bridge?* — on hardware +already present for the TBR. E0 runs **on jarvis** (decided 2026-08-04): the +real deployment host, same L2 as the Apple hub. Everything after E1 is +mechanical. + +**Sequencing note (XOQ3 outcome):** the `AgentSpec` extraction of +`server_process.py` lands as its own behaviour-preserving PR between E0 and +E1, so E7 is wiring, not refactoring. + +There is deliberately **no per-ecosystem spike**. A test earns its place by +changing a decision, and a Google Home or Alexa result changes none: we ship to +whatever pairs, and the only remedy for a refusal is a real vendor ID, which +§12 has already ruled out as disproportionate. Those ecosystems are therefore +untested-and-unclaimed (§10), not blockers. ## 10. Open questions -- **XOQ1.** Does Google Home pair an uncertified test-VID bridge, and does it - require Developer Console registration? **Blocking on scope claims** — E1. -- **XOQ2.** Alexa and SmartThings tolerance — assumed permissive, unverified. -- **XOQ3.** Does `server_process.py` generalise cleanly to a second agent, or - does it need extracting first? Audit before E8. -- **XOQ4.** Which DAC/VID the bridge node presents, and whether matter.js's - default test credentials are used as-is. Distinct from ADR-0005's *fabric* - VID (ADR-0006, "Two different vendor IDs, same number"). +- **XOQ1.** Whether Google Home pairs an uncertified test-VID bridge (it may + require Developer Console registration), and likewise Alexa and SmartThings. + **Not blocking, and not scheduled.** No hardware to test them on, no decision + hangs on the answer, and the only fix for a refusal — a real vendor ID — is + out of scope by §12. Treat as **untested and unclaimed**: promise Apple Home + in the docs, say nothing about the rest, and let the first user report settle + it. Revisit only if one of them becomes a hard requirement. +- **XOQ3. Answered (2026-08-04 audit).** It needs extracting first: identity + (launchd label, package name, stamp files, log names, argv, port) is + module-global, and two agents sharing the applied-plist stamp would trigger + spurious bootout cycles. But ~700 of its 1092 lines parameterise cleanly + behind a frozen `AgentSpec`; the extraction is behaviour-preserving and is + **pulled forward to before E1** so the hard-won recovery machinery (plist + digest, loaded-but-dead recovery, orphan reaper) is never duplicated. +- **XOQ4. Answered (2026-08-04 research).** matter.js hardcodes no VID: it + generates a fresh self-signed PAA→PAI→DAC chain at runtime for whatever + `vendorId`/`productId` is configured, with a Certification Declaration + signed by the CHIP *development* CD key (`certificationType: Test`). The + 0xFFF1/0x8000 values come from its examples, not the library. v1 uses the + test-range VID 0xFFF1 deliberately (uncertified is the honest posture, + ADR-0006); still distinct from ADR-0005's *fabric* VID. - **XOQ5.** Whether role belongs in `indigo-device-catalog` now rather than as plugin-local metadata later — cross-repo, and it would unblock garage doors. -- **XOQ6.** Bridge-node memory footprint at 100+ endpoints on an 8GB jarvis, - given the workspace's existing memory-pressure history. +- **XOQ6. Answered (2026-08-04, measured at 0.17.8):** ~145MB RSS floor, + ~0.3MB per additional endpoint, ~177MB at 100 endpoints, startup <0.5s. + Budget ~200MB on jarvis — comparable to one more mid-sized plugin, and far + below the MQTT-leak class of problem. Watchdog still tracks it (§5.4). ## 11. Dependencies - **matter.js** device-role API — new coupling, per ADR-0006's accepted cost. + Concretely: `@matter/main` + `@matter/nodejs` (0.17.8 at time of writing), + **exact-pinned**. Upstream repo is now the `matter-js` GitHub org (Open Home + Foundation). Do **not** depend on `@matter/examples` — stale on npm; the + live examples are in the repo's `examples/` tree. - **Node.js ≥ 22.13.0** — already required. - **Ecosystem tolerance of uncertified accessories** — external, unowned, XOQ1. + Apple Home's "Add Anyway" flow is the documented, working path (Homebridge + 2.0 and Homey ship on it); Apple DTS's position on uncertified bridges is + "behaviour is undefined", so it can tighten under us — a risk we accept. - **`indigo-device-catalog`** — soft dependency for v2 role defaults (XOQ5). ## 12. Out of scope for v1 (v2+ candidates) - Role/polarity defaults sourced from `indigo-device-catalog`. - Garage doors (blocked on the above). +- Valve export (matter.js cluster is an empty stub; ecosystem support poor). +- Fan export (matter.js cluster is a stub; includes SpeedControl-backed fans). - Sprinkler export as per-zone Water Valves. -- Power/energy export via Matter's Electrical Sensor type. +- Power/energy export via Matter's Electrical Sensor type (Apple ignores it in UI). - CSA certification and a real vendor ID. -- Multiple bridge nodes / per-ecosystem export sets. +- Multiple bridge nodes / per-ecosystem export sets. (If a device class ever + destabilises a whole bridge in Apple Home — the matterbridge RVC precedent — + per-device isolation onto its own server node is the known fix; design the + endpoint model so that promotion doesn't require re-architecture.) diff --git a/indigo-matter.indigoPlugin/Contents/Info.plist b/indigo-matter.indigoPlugin/Contents/Info.plist index 0bb8ea1..07554bb 100644 --- a/indigo-matter.indigoPlugin/Contents/Info.plist +++ b/indigo-matter.indigoPlugin/Contents/Info.plist @@ -20,7 +20,7 @@ IwsApiVersion 1.0.0 PluginVersion - 2026.7.21 + 2026.7.22 ServerApiVersion 3.6 From e509575e0feaa174749e51bf8c68fc12647d4ca6 Mon Sep 17 00:00:00 2001 From: Simon Clark Date: Tue, 4 Aug 2026 18:52:32 +0100 Subject: [PATCH 2/2] docs(export): apply PR-118 review findings MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Protocol spec (BRIDGE_PROTOCOL.md): - open_commissioning_window + window_closed event; get_pairing now models the non-durable passcode (windowOpen/expiry) instead of a static code - set_state response contract: node always responds; plugin sends without awaiting; client MUST log unmatched error responses; per-connection ordering guaranteed - full per-role command table (names + args) beside the state-key table; systemMode domain and 0-100<->0-254 rounding pinned - error-code catalogue (1.1); node-side version_mismatch behaviour; attach deadline for unattached sockets; supersede rationale corrected (half-open socket recovery, not the reconnect loop) - mass-removal guard on attach (intent: replace_all) - factory_reset preserveEndpointNumbers flag + rebuild_endpoint_map as the PRD §7 explicit-rebuild path; endpoint map persisted outside the matter.js storage context - fixed §5.1->§4.3 cross-refs, unified StatusReport shape, pinned WS port 5581, golden frames as shared JSON fixture files PRD: - thermostat row: no fan in v1 (fan descope applied consistently) - §7 lock/valve row trimmed to lock; added 5540 bind-conflict row - §4.2 bridge storage now stated as sacred as the controller's - milestone gating untangled: XAC2 lands at E7, XAC3's code display at E6, E3 gates on round-trip against a manually started node - 5540/EP1 claim attributed to matter.js ECOSYSTEMS.md, made a pref - XAC3/XOQ1/§9 now cite ADR-0007 (drafted in the workspace repo, proposed) instead of narrowing accepted ADR-0006 silently - MATTER.md citation fixed; XOQ2 tombstone; UI-C gap note; §5.4 watchdog gains RSS logging; §5.5 two-port readout CLAUDE.md: drop stale hardcoded PluginVersion. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01JxqGhP3DcENf68AZK21U4S --- CLAUDE.md | 2 +- docs/BRIDGE_PROTOCOL.md | 228 +++++++++++++++++++++++-------- docs/PRD-indigo-matter-export.md | 108 +++++++++------ 3 files changed, 242 insertions(+), 96 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index c1a39ba..0517dd8 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -69,7 +69,7 @@ client is portable to python-matter-server as a fallback. See `docs/MATTER.md` + Inherits workspace standards from [root CLAUDE.md](../CLAUDE.md#common-standards-apply-to-every-project-unless-its-claudemd-overrides). Key points: -- **Version bump per PR**: `Info.plist` `PluginVersion` (format `YYYY.R.P`, currently `2026.0.1`). +- **Version bump per PR**: `Info.plist` `PluginVersion` (format `YYYY.R.P`; the plist is the source of truth for the current value). - **Testing**: `pytest` (`pyproject.toml`, pylint + 120-char like netro). matter-server is mocked at the WS layer (`tests/fakes.py`); the `indigo` module is mocked. Run: `cd indigo-matter && pytest`. - **Merge**: GitHub PR only, never `--admin`, never squash, wait for CI green, wait for user go-ahead. diff --git a/docs/BRIDGE_PROTOCOL.md b/docs/BRIDGE_PROTOCOL.md index a05911b..a7a3247 100644 --- a/docs/BRIDGE_PROTOCOL.md +++ b/docs/BRIDGE_PROTOCOL.md @@ -1,11 +1,15 @@ # BRIDGE_PROTOCOL.md — plugin ⇄ bridge-node local protocol **Version:** 1 (`protocolVersion: 1`) -**Transport:** WebSocket, JSON text frames, loopback only +**Transport:** WebSocket, JSON text frames, loopback only, default port **5581** +(pref-configurable; the controller's WS is 5580) **Peers:** the Indigo plugin (client) and the `indigo-matter-bridge` node (server). We author both ends and ship them together. **Governing docs:** [`PRD-indigo-matter-export.md`](./PRD-indigo-matter-export.md) §4.4, -ADR-0006. +ADR-0006 (workspace-level). + +Section references: a bare `§N` refers to this document; PRD sections are +always written `PRD §N`. This is the outbound twin of the controller protocol that `protocol.py` / `matter_client.py` speak. It reuses the same envelope grammar so the client @@ -28,10 +32,34 @@ Identical shapes to the controller protocol: | Event | node → plugin | `{"event": "", "data": {…}}` (no `message_id`) | `message_id` is an opaque string chosen by the plugin, echoed verbatim. +**Every request gets exactly one response**, including ones the plugin chooses +not to await (§3.4). Frames on one connection are processed strictly in +receipt order, so a `set_state` sent after an `upsert_endpoint` on the same +socket is applied after it. + Unknown *fields* are ignored by both peers (forward compatibility inside a protocol version). Unknown *commands* get `error_code: "unknown_command"`. Unknown *events* are logged and dropped by the plugin. +### 1.1 Error codes + +The complete `error_code` domain for protocol version 1. Anything else is a +bug in the node. + +| `error_code` | Meaning | +|---|---| +| `unknown_command` | Command name not in §3 | +| `malformed_args` | Args missing/mistyped for the command | +| `version_mismatch` | `attach` carried a different `protocolVersion`; node refuses and closes the socket after responding | +| `not_attached` | Any command other than `attach` before a successful `attach` on this connection | +| `unknown_device` | `indigoDeviceId` has no live endpoint | +| `unknown_role` | `role` not in the §4.2 enum | +| `role_change` | `upsert_endpoint` tried to change an existing endpoint's role (§4.1) | +| `mass_removal_refused` | `attach` would remove every live endpoint without `"intent": "replace_all"` (§3.1) | +| `endpoint_map_invalid` | Node is in the refuse-to-start state (PRD §7); only `get_status`, `get_pairing` and `rebuild_endpoint_map` are accepted | +| `commissioning_window_failed` | Matter stack refused to open the enhanced commissioning window | +| `internal` | Unexpected node-side failure; `details` carries the message | + ## 2. Handshake On every new connection, before anything else: @@ -41,9 +69,16 @@ On every new connection, before anything else: 2. Plugin sends `attach` (§3.1). If `protocolVersion` differs from the plugin's own, the plugin does **not** attach: it surfaces an error telling the user to restart/update the bridge agent (typically the plugin was - updated while launchd kept the old node alive). The node accepts exactly - one attached client at a time; a second `attach` supersedes the first - (the old socket is closed) — the plugin's reconnect loop relies on this. + updated while launchd kept the old node alive). Node-side, a mismatched + `attach` gets `error_code: "version_mismatch"` and the socket is closed — + skew fails closed from both directions. + +The node accepts exactly one attached client at a time; a new successful +`attach` supersedes the incumbent (the old socket is closed). This exists to +recover from half-open sockets: if the plugin's TCP connection dies silently +(plugin crash, reload), the node would otherwise hold a dead "attached" client +forever and refuse the reconnecting plugin. A connection that completes the +handshake but has not attached within **10 seconds** is closed by the node. ## 3. Commands (plugin → node) @@ -59,9 +94,16 @@ Declares the client and delivers the desired endpoint set in one shot. }} ``` -Result: `{"status": }` (§5.1). The node reconciles its live -endpoint set against `endpoints` — creating, updating and removing as needed — -so a fresh connection is always a full reconcile (PRD §5.4 `startup`). +Result: `` (§4.3). The node reconciles its live endpoint set +against `endpoints` — creating, updating and removing as needed — so a fresh +connection is always a full reconcile (PRD §5.4 `startup`). + +**Mass-removal guard.** If `endpoints` is empty (or would remove every live +endpoint) while the node currently serves a non-empty set, the node refuses +with `error_code: "mass_removal_refused"` unless the args carry +`"intent": "replace_all"`. Removing every exported accessory from every paired +ecosystem must be deliberate (the §5.1 allow-list being emptied), never the +side effect of a stale or buggy client attaching with a default state. ### 3.2 `upsert_endpoint` @@ -80,8 +122,9 @@ Idempotent. Result: `{"endpointNumber": }`. Removes the child endpoint (`endpoint.close()`); the persisted endpoint-number allocation is **retained** so re-adding the same device restores the same -number. Idempotent — removing an absent endpoint succeeds. Bulk removals are -paced ~100ms apart by the node. +number. Idempotent — removing an absent endpoint succeeds with +`{"removed": false}`; a live removal returns `{"removed": true}`. Bulk +removals are paced ~100ms apart by the node. ### 3.4 `set_state` @@ -91,8 +134,15 @@ paced ~100ms apart by the node. Pushes Indigo-originated state outward. `states` keys are role-specific (§4.2). The node applies them as **local** (offline-context) writes so they are not -echoed back as commands. The plugin sends this fire-and-forget (it must never -block Indigo's device thread on the result). +echoed back as `command` events. Result: `{}` on success, or a normal error +response (`unknown_device`, `malformed_args`). + +The plugin sends this **without awaiting the result** — it must never block +Indigo's device thread. The response still arrives; the client's frame loop +MUST log any error response it cannot match to a waiting future (rather than +dropping it silently), because an unnoticed `set_state` failure looks exactly +like "the ecosystem shows stale state". This is a required behaviour change +from `matter_client.py`, which drops unmatched responses without a log line. ### 3.5 `set_reachable` @@ -102,34 +152,82 @@ block Indigo's device thread on the result). Split from `set_state` because it maps to Bridged Device Basic Information, not the functional cluster, and is driven by device enable/disable rather than -state change. +state change. `reachable` is also present on `EndpointSpec`; both paths write +the same attribute and last-write-wins — there is no precedence rule. +Result: `{}`. ### 3.6 `get_status` -Result: `` (§5.1). Used by the watchdog tick and the §5.5 -config readout. +Result: `` (§4.3) — the same shape `attach` returns. Used by the +watchdog tick and the PRD §5.5 config readout. ### 3.7 `get_pairing` +Reports pairing state. A Matter commissioning passcode is **not durable**: +once the first fabric commissions, the basic window closes and the original +code stops working; each additional admin needs an *enhanced* commissioning +window with a freshly derived code (§3.8). + Result: ```json -{"commissioned": false, - "manualPairingCode": "34970112332", - "qrPairingCode": "MT:Y.K90IRV01KA0648G00", +{"commissioned": true, + "windowOpen": false, + "windowExpiresAt": null, + "manualPairingCode": null, + "qrPairingCode": null, "fabrics": [ , … ]} ``` -### 3.8 `remove_fabric` +`manualPairingCode`/`qrPairingCode` are non-null only while a window is open +(always true for the never-commissioned initial state, whose codes are the +persisted originals). + +### 3.8 `open_commissioning_window` + +```json +{"command": "open_commissioning_window", "args": {"durationSeconds": 900}} +``` + +Opens an enhanced commissioning window so another ecosystem can be added. +Result: `{"manualPairingCode": "...", "qrPairingCode": "MT:...", +"windowExpiresAt": ""}`. Fails with +`commissioning_window_failed` if the stack refuses (e.g. a window is already +open in a conflicting state). Emits `window_closed` (§5) when it expires or a +commissioner completes. + +### 3.9 `remove_fabric` ```json {"command": "remove_fabric", "args": {"fabricIndex": 2}} ``` -### 3.9 `factory_reset` +Result: `{}`. + +### 3.10 `factory_reset` Wipes commissioning credentials and starts advertising fresh (PRD §6 "reset -all pairings"). The endpoint-number map is **preserved** — a reset must not -scramble identities if the user re-pairs the same ecosystems. +all pairings"). The endpoint-number map is **preserved by default** — a reset +must not scramble identities if the user re-pairs the same ecosystems. This +requires the map to be persisted **outside** the matter.js storage context +that its factory reset wipes: the node keeps it in its own file +(`endpoint-map.json`) in the bridge storage dir, written through the same +persistence layer as the drift detector's baseline. + +```json +{"command": "factory_reset", "args": {"preserveEndpointNumbers": true}} +``` + +Passing `false` wipes the map too — the "explicit rebuild" of PRD §7, for the +case where the map itself is what's corrupt. + +### 3.11 `rebuild_endpoint_map` + +The recovery path out of the `endpoint_map_invalid` refuse-to-start state +(PRD §7 "Endpoint map lost/corrupt"). Reallocates endpoint numbers for the +current endpoint set from scratch and persists a new map. This **will** +duplicate accessories in paired ecosystems — that is exactly why it is a +separate, explicit command that the plugin only issues after the user +confirms via a warning dialog. Result: ``. ## 4. Shapes @@ -153,32 +251,43 @@ scramble identities if the user re-pairs the same ecosystems. - `label` — Bridged Device Basic Information `NodeLabel`. - `options` — role-specific extras (e.g. window-covering polarity). -### 4.2 Roles and their state keys (v1) - -| `role` | Matter device type | `states` keys | -|---|---|---| -| `onOffPlugInUnit` | On/Off Plug-in Unit | `onOff: bool` | -| `onOffLight` | On/Off Light | `onOff: bool` | -| `dimmableLight` | Dimmable Light | `onOff: bool`, `level: 0-100` | -| `colorTemperatureLight` | Color Temperature Light | + `colorTempMireds: int` | -| `extendedColorLight` | Extended Color Light | + `hue: 0-360`, `saturation: 0-100` | -| `windowCovering` | Window Covering | `position: 0-100` (100 = open, inbound convention; polarity in `options`) | -| `doorLock` | Door Lock | `locked: bool` | -| `occupancySensor` | Occupancy Sensor | `occupied: bool` | -| `contactSensor` | Contact Sensor | `contact: bool` (true = closed) | -| `temperatureSensor` | Temperature Sensor | `temperatureC: float` | -| `humiditySensor` | Humidity Sensor | `humidityPct: float` | -| `lightSensor` | Light Sensor | `lux: float` | -| `pressureSensor` | Pressure Sensor | `pressureKPa: float` | -| `flowSensor` | Flow Sensor | `flowM3h: float` | -| `thermostat` | Thermostat | `localTemperatureC`, `heatingSetpointC`, `coolingSetpointC`, `systemMode` | - -Units are Indigo-natural at the protocol boundary (°C, %, lux, 0–100 levels); -the node owns the conversion to Matter wire units (0.01°C, mireds, the -illuminance log scale, 0–254 levels). Exactly one converter per role, in the -node, next to the cluster it feeds. - -### 4.3 `StatusReport` (§5.1) and `FabricInfo` +### 4.2 Roles: state keys and commands (v1) + +Each role defines two vocabularies: the **state keys** the plugin pushes via +`set_state`, and the **commands** the node emits as `command` events when an +ecosystem acts. Both are enumerated here in full; there is no other source. + +| `role` | Matter device type | `set_state` keys | `command` names (args) | +|---|---|---|---| +| `onOffPlugInUnit` | On/Off Plug-in Unit | `onOff: bool` | `onOff {"value": bool}` | +| `onOffLight` | On/Off Light | `onOff: bool` | `onOff {"value": bool}` | +| `dimmableLight` | Dimmable Light | `onOff: bool`, `level: 0-100` | `onOff`, `setLevel {"level": 0-100}` | +| `colorTemperatureLight` | Color Temperature Light | + `colorTempMireds: 153-500` | + `setColorTemp {"colorTempMireds": int}` | +| `extendedColorLight` | Extended Color Light | + `hue: 0-360`, `saturation: 0-100` | + `setColor {"hue": 0-360, "saturation": 0-100}` | +| `windowCovering` | Window Covering | `position: 0-100` (100 = open, inbound convention; polarity in `options`) | `goToPosition {"position": 0-100}`, `stopMotion {}` | +| `doorLock` | Door Lock | `locked: bool` | `lock {}`, `unlock {}` | +| `occupancySensor` | Occupancy Sensor | `occupied: bool` | — (sensors emit no commands) | +| `contactSensor` | Contact Sensor | `contact: bool` (true = closed) | — | +| `temperatureSensor` | Temperature Sensor | `temperatureC: float` | — | +| `humiditySensor` | Humidity Sensor | `humidityPct: float` | — | +| `lightSensor` | Light Sensor | `lux: float` | — | +| `pressureSensor` | Pressure Sensor | `pressureKPa: float` | — | +| `flowSensor` | Flow Sensor | `flowM3h: float` | — | +| `thermostat` | Thermostat | `localTemperatureC: float`, `heatingSetpointC: float`, `coolingSetpointC: float`, `systemMode: str` | `setHeatingSetpoint {"valueC": float}`, `setCoolingSetpoint {"valueC": float}`, `setSystemMode {"mode": str}` | + +- `systemMode` domain (both directions): `"off" | "heat" | "cool" | "auto"`. + The node owns the mapping to/from Matter's `SystemModeEnum` integers. +- `level`/`position` are integers 0–100 in both directions; the node owns the + 0–254 Matter LevelControl conversion and its rounding (round-half-up, with + 0 ↔ off preserved exactly). +- Thermostat **fan is not part of v1** (the Fan descope, PRD §5.2); there are + no fan state keys or commands. v2 candidate. +- Units are Indigo-natural at the protocol boundary (°C, %, lux, 0–100); + the node owns all Matter wire conversions (0.01°C, mireds bounds-clamping, + the illuminance log scale). Exactly one converter per role, in the node, + next to the cluster it feeds. + +### 4.3 `StatusReport` and `FabricInfo` ```json {"commissioned": true, @@ -196,12 +305,12 @@ error, never auto-repaired. | Event | `data` | Meaning | |---|---|---| -| `command` | `{"indigoDeviceId", "command", "args"}` | Ecosystem-originated action, e.g. `{"command": "onOff", "args": {"value": true}}`, `{"command": "moveToLevel", "args": {"level": 40}}`, `{"command": "lock"}` | +| `command` | `{"indigoDeviceId", "command", "args"}` | Ecosystem-originated action; names and args exactly as enumerated per role in §4.2 | | `fabrics_changed` | `{"fabrics": […], "change": "added"\|"deleted"\|"updated"}` | Pairing/unpairing activity | | `commissioned` / `decommissioned` | `{}` | First fabric added / last removed | +| `window_closed` | `{"reason": "expired"\|"commissioned"}` | The enhanced commissioning window ended | | `drift_detected` | `{"drift": […]}` | Endpoint-number drift found at startup | -`command` events carry the same role-relative vocabulary as `set_state` keys. The plugin resolves `indigoDeviceId` through the allow-list before acting; a command for a device no longer exported gets no action and a warning (PRD §7 race row — the node responds to the ecosystem with failure when the endpoint @@ -210,7 +319,8 @@ is already gone). ## 6. Invariants 1. **Loopback only.** The node binds `127.0.0.1`; there is no auth on the - socket because there is no remote surface. + socket because there is no remote surface. The mass-removal guard (§3.1) + exists because "local process" still includes stale plugin instances. 2. **The plugin is the source of truth for the export set**; the node is the source of truth for commissioning state and endpoint numbers. Neither peer caches the other's domain across reconnects — `attach` reconciles. @@ -219,11 +329,17 @@ is already gone). 4. **State pushes are echo-guarded** in the node (`ctx.offline`); the plugin never receives a `command` event for a change it pushed. 5. **Version skew fails closed:** mismatched `protocolVersion` means no - attach, an error in the Indigo log, and untouched pairings. + attach (both peers enforce it), an error in the Indigo log, and untouched + pairings. +6. **Destructive operations are explicit:** emptying the endpoint set needs + `intent: "replace_all"`; discarding endpoint identity needs + `preserveEndpointNumbers: false` or `rebuild_endpoint_map`. Neither can + happen as a default. ## 7. Testing contract -Golden frames for every command/response/event pair live beside the Python -tests (the `test_golden_real.py` pattern) and are the cross-language contract: -the TypeScript node's protocol tests consume the same fixtures, so a frame -change that only updates one side fails that side's suite. +Golden frames for every command/response/event pair live as **JSON fixture +files** under `tests/fixtures/bridge_protocol/`, consumed by both the Python +suite and the TypeScript node's protocol tests — a deliberate departure from +`test_golden_real.py`'s inline-dict pattern, which a TS suite cannot import. +A frame change that only updates one side fails that side's suite. diff --git a/docs/PRD-indigo-matter-export.md b/docs/PRD-indigo-matter-export.md index cf6d7bd..69fa88c 100644 --- a/docs/PRD-indigo-matter-export.md +++ b/docs/PRD-indigo-matter-export.md @@ -2,7 +2,7 @@ **Status:** Accepted — build in progress **Owner:** Simon -**Governing ADR:** [`../../docs/adr/0006-indigo-as-matter-bridge.md`](../../docs/adr/0006-indigo-as-matter-bridge.md) (accepted 2026-08-03) +**Governing ADR:** [`../../docs/adr/0006-indigo-as-matter-bridge.md`](../../docs/adr/0006-indigo-as-matter-bridge.md) (accepted 2026-08-03; workspace-level — the path resolves in the multi-repo workspace checkout, not on GitHub), as amended by ADR-0007 (validation-evidence criterion) **Companion PRD:** [`PRD-indigo-matter-plugin.md`](./PRD-indigo-matter-plugin.md) (historical — the inbound/controller build) **Local protocol spec:** [`BRIDGE_PROTOCOL.md`](./BRIDGE_PROTOCOL.md) **Last updated:** 2026-08-04 (research pass: matter.js 0.17.8 verified by execution; scope and @@ -91,10 +91,11 @@ generalising `server_process.py` rather than duplicating it. Two differences from the controller agent: - It is **not started at all** while the allow-list is empty (XG5). -- It has no data directory to treat as sacred in the controller's sense, but it - *does* hold commissioned-fabric credentials for paired ecosystems — losing - them un-pairs every ecosystem and forces re-pairing. Back it up alongside the - controller's storage. +- Its data directory is **every bit as sacred as the controller's**, for + different reasons: it holds the commissioned-fabric credentials for every + paired ecosystem (losing them un-pairs everything) *and* the endpoint-ID + allocation map (losing that duplicates every accessory in every ecosystem — + §4.3). Back it up alongside the controller's storage. ### 4.3 Storage @@ -132,18 +133,24 @@ state changes outward, and the bridge node pushes ecosystem commands inward. **Decision:** mirror the controller's WebSocket + JSON message shape. It costs nothing new to learn, `matter_client.py` is a working reference for the client half, and the existing test doubles generalise. The bridge node listens on -loopback only, on a configurable port defaulting adjacent to the controller's. +loopback only, on a configurable port defaulting to **5581** (the controller's +WS is 5580). Unlike `protocol.py` there is **no rename firewall** — we own both ends and ship them together — but the handshake carries a `protocolVersion`, because launchd deliberately keeps the old node running across plugin reloads and version skew is the failure that will actually happen. Full spec: [`BRIDGE_PROTOCOL.md`](./BRIDGE_PROTOCOL.md). -The **Matter side** of the node binds UDP **5540** (the Matter default) with the -Aggregator at **Endpoint 1**. This is Alexa's hard requirement (it discovers -nothing on any other port, and needs EP1 to be the aggregator); Alexa is -unclaimed (XOQ1) but the constraint costs nothing now and is painful to -retrofit. No conflict with the controller: `matter-server` listens on TCP 5580 -plus ephemeral UDP, not 5540. mDNS is pinned to the primary interface (reusing +The **Matter side** of the node binds UDP **5540** (the Matter default, also +pref-configurable) with the Aggregator at **Endpoint 1**. matter.js's +ECOSYSTEMS.md documents both as Alexa's hard requirement (it discovers nothing +on any other port and needs EP1 beside the root); we have not verified this +(XOQ1), but the constraint costs nothing now and is painful to retrofit. No +conflict with the controller: `matter-server` listens on TCP 5580 plus +ephemeral UDP, not 5540. But 5540 *is* contended by any other Matter device +stack on the same Mac — Homebridge 2.x, matterbridge, an HA container in host +mode — so a bind failure is a first-class §7 failure mode, surfaced with the +holder named, and the pref is the escape hatch (moving off 5540 forfeits the +documented Alexa behaviour). mDNS is pinned to the primary interface (reusing the `primaryInterface` pref) — matter.js's own mDNS stack defaults to all interfaces and breaks on Macs with VPN/utun interfaces. @@ -164,7 +171,9 @@ devices — which rules out a plain multi-select. **Decision: UI-D**, falling back to **UI-B** if Indigo's XML dialog list controls prove unworkable at scale — the same "recommended start, documented -fallback" treatment PM-B/PM-A got in the original PRD. +fallback" treatment PM-B/PM-A got in the original PRD. (UI-C, a hybrid +variant, was folded into UI-D during drafting; the lettering gap is +deliberate.) The candidate list MUST be filtered by plugin ID to exclude `indigo-matter`'s own devices, making the loop guard (XNG3) structural rather than a runtime check. @@ -206,7 +215,7 @@ safest interpretation (plug/light) rather than guessing. | Sensor (numeric, lux) | — | Light Sensor | | | Sensor (numeric, pressure) | — | Pressure Sensor | Apple Home ignores this type (Google supports it); exported anyway, documented | | Sensor (numeric, flow) | — | Flow Sensor | Apple Home ignores this type (Google supports it); exported anyway, documented | -| Thermostat | — | Thermostat | Setpoints, modes; fan merged if present. matter.js provides the cluster machinery; the HVAC logic is ours | +| Thermostat | — | Thermostat | Setpoints, modes. No fan in v1 (the FanControl descope applies here too); v2 candidate. matter.js provides the cluster machinery; the HVAC logic is ours | | SpeedControl | Fan | *Not exportable in v1* | Descoped 2026-08-04; see below | Matter device-type IDs are deliberately omitted here; take them from the @@ -234,7 +243,9 @@ silently missing. One Matter node: root endpoint, an **Aggregator** endpoint, and one **Bridged Node** child endpoint per exported device carrying Bridged Device Basic Information (`NodeLabel`, `Reachable`, `UniqueID`) — the standard bridge -topology the plugin already consumes inbound (`MATTER.md:238-244`). +topology the plugin already consumes inbound (the "Bridges" section of +`MATTER.md`; in code, `matter_model.py`'s BridgedDeviceBasicInformation +handling and `device_sync.py`'s `DEVICE_TYPE_AGGREGATOR`). - **Distribution:** the bridge node is a **published npm package** (`indigo-matter-bridge`, TypeScript, decided 2026-08-04), exact-pinned by the @@ -269,24 +280,34 @@ topology the plugin already consumes inbound (`MATTER.md:238-244`). plugin reloads must not un-pair ecosystems). - `deviceUpdated` — diff relevant states, push outward. - `deviceDeleted` — remove from allow-list, drop the endpoint. -- `runConcurrentThread` — health check, reconnect, reconcile drift. +- `runConcurrentThread` — health check, reconnect, reconcile drift, and + periodic bridge-node RSS logging (the XOQ6 watchdog). ### 5.5 Configuration UI Plugin config gains an **Export** section: enable/disable export wholesale, -bridge-node port, and a pairing-status readout (which ecosystems are paired, -fabric slots used/remaining). Per-export settings live in the §5.1 dialog. +the two bridge-node ports (local-protocol WS, default 5581; Matter UDP, +default 5540), and a pairing-status readout (which ecosystems are paired, +fabric slots used/remaining). The readout earns its place even though +matter.js defaults to 254 fabric slots: what users actually need to see is +*which* ecosystems hold a fabric and whether a commissioning window is open, +not slot arithmetic. Per-export settings live in the §5.1 dialog. ## 6. Pairing and fabric management -- **Pair:** a menu action surfaces the bridge node's setup code and QR payload. - The user adds it in each ecosystem's app as they would any Matter accessory. - **Mechanism** (Indigo dialogs have no dynamic labels and no image fields): - the manual code is written to the event log — the plugin's established - pattern for runtime strings — and the QR is rendered on an IWS-served page, - reachable from the same menu action. Passcode and discriminator are - **randomised per install** (identical passcodes produce identical pairing - codes across installs — verified) and persisted by the bridge node. +- **Pair:** a menu action surfaces a pairing code and QR payload. The user + adds the bridge in each ecosystem's app as they would any Matter accessory. + A commissioning passcode is **not durable**: once the first ecosystem + commissions, the original code stops working, and each further admin needs + an *enhanced commissioning window* with a freshly derived code — so the + menu action is "open a pairing window" (`open_commissioning_window`, + BRIDGE_PROTOCOL §3.8), not "show the code". **Display mechanism** (Indigo + dialogs have no dynamic labels and no image fields): the manual code is + written to the event log — the plugin's established pattern for runtime + strings — and the QR is rendered on an IWS-served page, reachable from the + same menu action. Passcode and discriminator are **randomised per install** + (identical passcodes produce identical pairing codes across installs — + verified) and persisted by the bridge node. - **Uncertified prompt:** expect an uncertified-accessory warning in every ecosystem (ADR-0006). Document it as *expected* in `INSTALL.md`, with the Homebridge parallel, so it doesn't read as a fault. @@ -302,18 +323,22 @@ fabric slots used/remaining). Per-export settings live in the §5.1 dialog. | Bridge node down | Plugin marks export status degraded; Indigo devices unaffected; agent restarted by launchd; reconcile on reconnect | | Ecosystem sends a command for a deleted Indigo device | Endpoint already removed; if racing, return failure rather than silently dropping | | Indigo device disabled | `Reachable` = false; accessory greys out | -| Allow-list emptied | Endpoints removed; agent stopped; pairings retained unless explicitly reset | -| Lock/valve command | Never auto-confirm destructive state changes; honour the inbound flood-safe/lock conventions | -| Endpoint map lost/corrupt | Refuse to auto-reallocate; surface an error and require an explicit rebuild, since silent reallocation duplicates accessories in every paired ecosystem | +| Allow-list emptied | Endpoints removed (the deliberate `intent: "replace_all"` path, BRIDGE_PROTOCOL §3.1); agent stopped; pairings retained unless explicitly reset | +| Lock command | Never auto-confirm destructive state changes; honour the inbound lock conventions | +| Matter UDP port (5540) already bound | Another Matter device stack (Homebridge 2.x, matterbridge, HA) holds it; surface the error naming the holder; the port pref is the escape hatch (§4.4) | +| Endpoint map lost/corrupt | Refuse to auto-reallocate; surface an error and require an explicit rebuild (`rebuild_endpoint_map`, BRIDGE_PROTOCOL §3.11), since silent reallocation duplicates accessories in every paired ecosystem | ## 8. Acceptance criteria - **XAC1.** Fresh install: no bridge process running, nothing paired, nothing exported. -- **XAC2.** Exporting one relay starts the bridge node and yields a pairing code. +- **XAC2.** Exporting one relay starts the bridge node and yields a pairing + code, within XG1's 30 seconds. (Lands at E7 — the start-on-export wiring.) - **XAC3.** Bridge pairs into **Apple Home** from the displayed code and controls the device. Multi-fabric capability is proven by adding a **second admin we already own** — Domio's own fabric (ADR-0005) or a matter-server controller — not by requiring a third-party ecosystem we cannot test (XOQ1). + This is a deliberate narrowing of ADR-0006's confirmation criterion, + recorded in **ADR-0007**. - **XAC4.** Command round-trip both directions within 500ms (XG2). - **XAC5.** Endpoint IDs and `UniqueID`s survive plugin reload, bridge-node restart and Mac reboot with no accessory duplication (XG4). @@ -334,28 +359,30 @@ fabric slots used/remaining). Per-export settings live in the §5.1 dialog. | E0 | Bridge node skeleton — **the validation gate** | Node process starts, exposes an aggregator with one hard-coded endpoint, and **pairs into Apple Home**. If an uncertified bridge will not pair here, the design is dead and nothing after this matters | | E1 | Local protocol + plugin client | Plugin drives endpoint create/remove over WS | | E2 | Allow-list + UI-D dialog | Devices selectable with role; loop guard live (XAC6, XAC9) | -| E3 | Relay + dimmer export | XAC2, XAC3, XAC4 | +| E3 | Relay + dimmer export | XAC4 both directions, and XAC3's Apple Home control, against a manually started bridge node (start-on-export is E7's; the code display is E6's) | | E4 | Sensors + thermostat export | Mapping table complete for v1 | | E5 | Endpoint persistence | XAC5 — the highest-risk correctness requirement | -| E6 | Pairing/unpairing UX + fabric readout | §6 complete | -| E7 | launchd agent + failure recovery | §7, XAC7, XAC8 | +| E6 | Pairing/unpairing UX + fabric readout | §6 complete, including XAC3's displayed-code pairing flow | +| E7 | launchd agent + failure recovery | §7, XAC1, XAC2, XAC7, XAC8 | | E8 | Docs | `INSTALL.md` export section, uncertified-prompt explanation, ecosystems-untested note (§10), `MATTER.md` outbound architecture | **E0 is the whole validation loop.** It answers the only question that can kill the feature — *will any ecosystem pair an uncertified bridge?* — on hardware already present for the TBR. E0 runs **on jarvis** (decided 2026-08-04): the real deployment host, same L2 as the Apple hub. Everything after E1 is -mechanical. +well-understood, though not all of it is easy — E5 remains the highest-risk +correctness milestone (§4.3). **Sequencing note (XOQ3 outcome):** the `AgentSpec` extraction of `server_process.py` lands as its own behaviour-preserving PR between E0 and E1, so E7 is wiring, not refactoring. There is deliberately **no per-ecosystem spike**. A test earns its place by -changing a decision, and a Google Home or Alexa result changes none: we ship to -whatever pairs, and the only remedy for a refusal is a real vendor ID, which -§12 has already ruled out as disproportionate. Those ecosystems are therefore -untested-and-unclaimed (§10), not blockers. +changing a decision, and a Google Home or Alexa result changes none: we ship +to whatever pairs, and the remedies for a refusal — a real vendor ID, or +per-ecosystem developer-console registration — are weighed and declined in +**ADR-0007**. Those ecosystems are therefore untested-and-unclaimed (§10), +not blockers. ## 10. Open questions @@ -365,7 +392,10 @@ untested-and-unclaimed (§10), not blockers. hangs on the answer, and the only fix for a refusal — a real vendor ID — is out of scope by §12. Treat as **untested and unclaimed**: promise Apple Home in the docs, say nothing about the rest, and let the first user report settle - it. Revisit only if one of them becomes a hard requirement. + it. Revisit only if one of them becomes a hard requirement. The scope + narrowing relative to ADR-0006's confirmation criterion is recorded in + ADR-0007. (XOQ2, Alexa/SmartThings tolerance, was folded into this question + in the 2026-08-04 revision; the numbering gap is deliberate.) - **XOQ3. Answered (2026-08-04 audit).** It needs extracting first: identity (launchd label, package name, stamp files, log names, argv, port) is module-global, and two agents sharing the applied-plist stamp would trigger