fix(cache): recover invalidation after failed prime

app/core/cache/invalidation.py

@@ -89,8 +89,10 @@ async def initialize(self) -> None:
 
         On success ``_poll_initialized`` is set to ``True``. If the read fails the
         method raises with state unchanged (baseline empty, ``_poll_initialized``
-        still ``False``), so the caller can degrade to the first-poll-baselines
-        behavior.
+        still ``False``), so the caller can retry before background polling. If
+        the caller continues, ``start()`` arms conservative callback delivery for
+        the first successfully observed versions instead of accepting them as a
+        callback-less baseline.
         """
         session = self._session_factory()
         try:
@@ -118,7 +120,10 @@ async def prime(self) -> None:
 
         Mirrors ``initialize``'s error contract: if the baseline read fails the
         poller stays uninitialized (``_poll_initialized`` still ``False``) and
-        this method raises so the caller can retry or explicitly degrade.
+        this method raises so the caller can retry before background polling.
+        Starting without a successful retry makes the first recovered poll
+        reconcile positive versions through their callbacks before acknowledging
+        them.
         ``_poll_once`` swallows the read error, so a silent success here would
         let the first *background* poll absorb a peer bump as the initial
         baseline, voiding the delivery guarantee priming exists to provide.
@@ -129,6 +134,13 @@ async def prime(self) -> None:
     async def start(self) -> None:
         if self._task and not self._task.done():
             return
+        # Callback-less baseline acquisition is safe only before process-local
+        # state can be served. Once background polling starts, a missing baseline
+        # means the observed versions are uncertain: treat each positive first
+        # observation as a change and reconcile it through the normal callback /
+        # acknowledgement path. A successful prime already set this flag after
+        # recording exact versions, so normal startup remains unchanged.
+        self._poll_initialized = True
         self._stop.clear()
         self._task = asyncio.create_task(self._run())
 

app/main.py

@@ -358,12 +358,12 @@ async def lifespan(app: FastAPI):
     try:
         await cache_poller.prime()
     except Exception:
-        # prime() raises when the baseline version read fails; degrade to
-        # first-poll-baselines (matching initialize()'s contract) rather than
-        # continuing as if the seed succeeded. A peer bump landing before the
-        # first background poll may then be absorbed as the initial baseline
-        # and only converge on the fallback TTL / next bump, but the failure is
-        # surfaced here instead of silently voiding the delivery guarantee.
+        # prime() raises when the baseline version read fails, leaving the poller
+        # uninitialized so an explicit retry would remain baseline-only. Startup
+        # continues, but start() arms conservative recovery: the first successful
+        # background read invokes callbacks for positive versions before
+        # acknowledging them, so a peer bump cannot become a callback-less
+        # baseline after local caches are warm.
         logger.warning("cache invalidation baseline prime failed", exc_info=True)
     try:
         await routing_availability_cache.refresh_from_db()

openspec/changes/recover-cache-invalidation-after-prime-failure/.openspec.yaml

@@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-07-30

openspec/changes/recover-cache-invalidation-after-prime-failure/design.md

@@ -0,0 +1,60 @@
+## Context
+
+`CacheInvalidationPoller.prime()` normally records every namespace version before startup warms process-local caches. If that read fails, startup logs the failure, warms the routing and model-registry state, and starts background polling with no known versions. The first successful poll currently treats every observed row as a callback-less baseline, even if a peer advanced it after the failed read.
+
+The poller therefore has two distinct lifecycle phases: explicit baseline acquisition before process-local state is served, and background reconciliation after that state may be warm. Only the former may accept an observed version without invoking its callbacks.
+
+## Goals / Non-Goals
+
+**Goals:**
+
+- Recover conservatively after a failed startup baseline read.
+- Run registered callbacks before acknowledging positive versions first observed during background polling.
+- Preserve callback retry and monotonic version-acknowledgement behavior.
+- Preserve baseline-only behavior when `prime()` is explicitly retried before background polling.
+- Prove the fix at both the warmed upstream-route resolver cache and the account-routing / bridge-session reuse seam.
+
+**Non-Goals:**
+
+- Failing startup or adding startup retries.
+- Changing namespace versions, bump ordering, callback registration, poll intervals, or cache TTLs.
+- Adding configuration, schema, migrations, or cross-replica payloads.
+- Changing normal startup behavior when baseline priming succeeds.
+
+## Decisions
+
+### Background start ends callback-less baseline acquisition
+
+`start()` will transition an uninitialized poller into conservative background-reconciliation mode before it creates the polling task. A successful prime has already made this transition with recorded versions. After a failed prime, the transition makes a positive version with no known baseline satisfy the existing change predicate, so the namespace callback runs before acknowledgement.
+
+This uses the existing `_poll_initialized` state rather than adding a second failure latch. The relevant distinction is not whether one particular read failed, but whether the caller is still explicitly acquiring a pre-service baseline or has started background polling after caches may be warm.
+
+Alternatives considered:
+
+- Mark the poller initialized inside the `prime()` failure path: rejected because an explicit `prime()` retry could then invoke callbacks even though its documented purpose is baseline-only acquisition.
+- Retry or fail startup: rejected because it changes availability policy and is broader than the stale-cache defect.
+- Clear routing caches directly in `app.main` after the exception: rejected because it duplicates callback wiring, does not cover later first-observed namespace rows, and leaves other poller consumers inconsistent.
+- Add a separate recovery latch: workable, but redundant with the existing lifecycle boundary and adds state combinations without improving the guarantee.
+
+### Recovery uses normal callback acknowledgement rules
+
+Recovery will use `_run_callbacks()` and the existing per-namespace acknowledgement path. A callback failure leaves that namespace unacknowledged and the next poll retries it. Existing monotonic handling for concurrent `bump_local()` acknowledgements remains unchanged.
+
+### The integration proofs keep the real routing gates
+
+The regressions will use two pollers sharing the integration database. One keeps a resolved upstream-route outcome warm across a failed prime and a peer `upstream_route` bump. The other keeps a routing snapshot seeded `ACTIVE` across a peer status change plus `account_routing` bump and checks the actual bridge-session reuse predicate. Together they prove that recovery changes externally relevant routing decisions, not merely a callback counter.
+
+## Risks / Trade-offs
+
+- [A failed prime followed by background start may replay callbacks for versions that predate startup] → Callbacks are designed to be idempotent local reconciliation; this conservative replay happens only on the exceptional no-baseline path and is safer than serving unknown cache state.
+- [A recovery callback can fail] → The existing unacknowledged-version retry path remains authoritative and is covered by focused tests.
+- [The background task could poll before recovery mode is active] → Transition lifecycle state synchronously in `start()` before creating the task.
+- [A namespace has no row yet] → No callback runs until a later bump creates a positive version; that first observed version is then treated as changed.
+
+## Migration Plan
+
+No data or configuration migration is required. Deploy as an application-only change. Rollback restores the previous callback-less first-poll fallback after a failed prime; no persisted state requires reversal.
+
+## Open Questions
+
+None.

openspec/changes/recover-cache-invalidation-after-prime-failure/proposal.md

@@ -0,0 +1,28 @@
+## Why
+
+A transient failure while priming cache-invalidation versions leaves the poller without a baseline, but startup continues and warms routing state. A peer mutation that lands afterward can then be accepted by the first successful background poll as a callback-less baseline, leaving a stale routing decision in service until another bump or restart.
+
+## What Changes
+
+- Make background polling recover fail-safe when no version baseline was recorded: observed positive namespace versions are delivered through their callbacks before they are acknowledged.
+- Preserve baseline-only semantics for an explicit `prime()` retry before background polling begins.
+- Cover the production-sensitive upstream-route cache and account-routing bridge-reuse paths with two-replica regressions that prove warmed decisions are invalidated after the version read recovers.
+- Replace the model-catalog contract's documented callback-less degradation with the same conservative recovery behavior.
+
+## Capabilities
+
+### New Capabilities
+
+None.
+
+### Modified Capabilities
+
+- `query-caching`: Require the first successful background poll after a failed startup baseline read to reconcile observed namespaces before acknowledging their versions.
+- `model-catalog-compat`: Keep surfacing a failed baseline prime while requiring background recovery to invoke the model-registry callback instead of accepting a callback-less first-poll baseline.
+
+## Impact
+
+- `app/core/cache/invalidation.py`: background-poller lifecycle state.
+- `app/main.py`: startup failure-semantics documentation.
+- Cache-invalidation integration coverage for the warmed account-routing / bridge-reuse path and existing model-registry startup behavior.
+- No API, configuration, dependency, database-schema, or migration change.