fix: one multi-contract EventWatcher (collapse per-membership watchers)#125
Merged
Conversation
…atcher Boot spun up one EventWatcher per active council membership (N pollers, N catch-ups), making startup heavy enough to blow the deploy health-check grace and trigger Fly rollback. Replace the N-watchers model with a single EventWatcher that polls every active council's channel-auth contract in one batched getEvents call (5 contractIds/filter, 5 filters/call) and dispatches each event by its source contract. Membership changes mutate the watcher's contract set in place (addContract/removeContract) instead of creating/destroying watchers; the single watcher keeps running. Removal is reference-counted so one PP leaving a shared council does not blind the watcher for others still in it. Bump version 0.9.1 -> 0.9.2.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The provider boots one
EventWatcherper active council membership —event-watcher/index.tslooped over memberships and calledensureWatcher(channelAuthId), each spinning upnew EventWatcher({ contractId })with its owngetEventspoll + catch-up. With ~10+ active memberships that is 10+ independent pollers on boot, heavy enough to blow the deploy's health-check grace → Flywait timeout→ rollback (why theBOOT_SYNCsecret is stuckStaged).What
One watcher. A single
EventWatcherpollsgetEventsonce with all active councils' channel-auth contract IDs batched into the filter, and dispatches each event by the contract it came from.EventWatchernow holds aSet<contractId>instead of onecontractId; one boot start-ledger resolution, one poll loop, one catch-up regardless of membership count.fetchChannelAuthEvents(rpc, contractIds[], startLedger)chunks into the RPC limits (5 contractIds/filter, 5 filters/call ≈ 25/call); >25 contracts fan out into the minimum number of sequential calls and log when the count is unexpectedly large. The poll advances the cursor to the least-advanced call'slatestLedgerso a lagging filter can never skip events.event.contractId— each parsed event is tagged with the contract that emitted it (taken from the raw RPC event), routed to the existing handlers (channel_state_changed→ registry/withdraw-only gate;provider_added/removed→ membership). No change to what each handler does.addCouncilWatcher) and boot add the contractId viawatcher.addContract; the single watcher keeps running. On deactivation the contract is removed only when no active membership still references it (reference-counted, so one PP leaving a shared council doesn't blind the watcher for others).onResyncnow re-queries every watched council and reconciles (converge-by-query), per affected contract./healthAlready fully decoupled from watcher state —
src/http/v1/health/routes.ts:16returns a static{ status: "ok", version, deps }and never reads watcher progress. Left as-is.Tests
getContractIdsreflects in-place add/remove; one watcher polls every contract added before start; a removed contract is no longer polled (remove → stops); empty set holds the cursor and skips the RPC.deno fmt --check,deno lint, and the unit suite are green locally (the one flaky failure —signed-payload_test.ts, a time-basedmaxAgetest — passes in isolation and is unrelated to this change). Version bumped 0.9.1 → 0.9.2.Out of scope
The
BOOT_SYNCiac value (already staged — this PR makes the deploy able to land it). council-platform watches its single council contract (no per-X loop found). Asset-lifecycle semantics unchanged.