Skip to content

fix: one multi-contract EventWatcher (collapse per-membership watchers)#125

Merged
AquiGorka merged 1 commit into
mainfrom
hotfix/single-event-watcher
Jun 22, 2026
Merged

fix: one multi-contract EventWatcher (collapse per-membership watchers)#125
AquiGorka merged 1 commit into
mainfrom
hotfix/single-event-watcher

Conversation

@AquiGorka

Copy link
Copy Markdown
Contributor

Why

The provider boots one EventWatcher per active council membershipevent-watcher/index.ts looped over memberships and called ensureWatcher(channelAuthId), each spinning up new EventWatcher({ contractId }) with its own getEvents poll + catch-up. With ~10+ active memberships that is 10+ independent pollers on boot, heavy enough to blow the deploy's health-check grace → Fly wait timeout → rollback (why the BOOT_SYNC secret is stuck Staged).

What

One watcher. A single EventWatcher polls getEvents once with all active councils' channel-auth contract IDs batched into the filter, and dispatches each event by the contract it came from.

  • Single watcher shapeEventWatcher now holds a Set<contractId> instead of one contractId; one boot start-ledger resolution, one poll loop, one catch-up regardless of membership count.
  • Batched contractIdsfetchChannelAuthEvents(rpc, contractIds[], startLedger) chunks into the RPC limits (5 contractIds/filter, 5 filters/call ≈ 25/call); >25 contracts fan out into the minimum number of sequential calls and log when the count is unexpectedly large. The poll advances the cursor to the least-advanced call's latestLedger so a lagging filter can never skip events.
  • Dispatch by event.contractId — each parsed event is tagged with the contract that emitted it (taken from the raw RPC event), routed to the existing handlers (channel_state_changed → registry/withdraw-only gate; provider_added/removed → membership). No change to what each handler does.
  • Membership changes update the set in place — join (addCouncilWatcher) and boot add the contractId via watcher.addContract; the single watcher keeps running. On deactivation the contract is removed only when no active membership still references it (reference-counted, so one PP leaving a shared council doesn't blind the watcher for others).
  • Out-of-retentiononResync now re-queries every watched council and reconciles (converge-by-query), per affected contract.

/health

Already fully decoupled from watcher state — src/http/v1/health/routes.ts:16 returns a static { status: "ok", version, deps } and never reads watcher progress. Left as-is.

Tests

  • service: tags each event with its source contract; one call covers multiple contracts; empty set never queries RPC; >25 contracts batch within the 5×5 limits with no contract dropped.
  • process: getContractIds reflects in-place add/remove; one watcher polls every contract added before start; a removed contract is no longer polled (remove → stops); empty set holds the cursor and skips the RPC.

deno fmt --check, deno lint, and the unit suite are green locally (the one flaky failure — signed-payload_test.ts, a time-based maxAge test — passes in isolation and is unrelated to this change). Version bumped 0.9.1 → 0.9.2.

Out of scope

The BOOT_SYNC iac value (already staged — this PR makes the deploy able to land it). council-platform watches its single council contract (no per-X loop found). Asset-lifecycle semantics unchanged.

…atcher

Boot spun up one EventWatcher per active council membership (N pollers,
N catch-ups), making startup heavy enough to blow the deploy health-check
grace and trigger Fly rollback. Replace the N-watchers model with a single
EventWatcher that polls every active council's channel-auth contract in one
batched getEvents call (5 contractIds/filter, 5 filters/call) and dispatches
each event by its source contract.

Membership changes mutate the watcher's contract set in place
(addContract/removeContract) instead of creating/destroying watchers; the
single watcher keeps running. Removal is reference-counted so one PP leaving
a shared council does not blind the watcher for others still in it.

Bump version 0.9.1 -> 0.9.2.
@AquiGorka
AquiGorka merged commit d72dfad into main Jun 22, 2026
7 checks passed
@AquiGorka
AquiGorka deleted the hotfix/single-event-watcher branch June 22, 2026 12:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant