[kriszyp]

⊙ Problem

A subscription checkpoint can certify an origin only if its stored floor means a complete local prefix. Item 2's floors do not (harper-pro#922 items 3–4). Replication resumes at max(position, below(F)). So a transaction committed below the sender's position during a disconnect is filtered, yet the floor past it stays stored. Nothing revokes a floor when a peer downgrades. A floor names no history identity or table coverage.

🤔 Does replication need to change for a subscription feature?

Yes: a token cannot recover entries replication omitted, so completeness must be proven where data flows. A whole prefix requires floor-only resume, generation and coverage on the certificate, and revocation. Level-1 peers are unchanged.

💡 Solution

originFloors level 2 carries { origin: [floor, generation, fullCoverage] | 0 } between two level-2 peers, advertised only by a build whose engine syncs the floor record before advertising it (rocksdb-js syncWal()). A level-2 route resumes exactly below its stored floor and records where its proven prefix begins. Holes, widened routes and downgrades are marked or revoked. A base copy records what it wrote in rows of its own, and core's checkpoint admission voids any route start those rows cover.

🤔 Floor-only resume re-sends up to a certificate interval of a busy origin on reconnect

Resuming at below(F) rather than the position replays what lies between them, about one 30 s interval of a busy origin. That duplication makes a stored floor a complete prefix; the receiver deduplicates by identity (owner ruling 3a). Advertising level 1 reverses it per route.

⚖️ Alternatives

🤔 Who writes a route's [seq, peer] row?

Only core's sequence updater for that route. A copy's effect on other routes lives in its own [copyBound, peer] rows, written by core's writeCopyBounds (harper#3685); rewriting other routes' rows raced the updater. Recording it in the copy's own transaction is not crash-safe (#1134).

🤔 Rebase onto sharding (#1116)

Sharded routes take no floor-only start and report no full coverage, so a multi-shard database admits no checkpoint yet (deferral below).

🤔 Which row does a route mark?

The row core writes, [seq, receivingDataFromNodeIds[0]], not the socket's peer on a failover subscription. Marking the peer's row failed every read-back. Inline retries starved sharding RPCs; retries now use the timer.

🤔 When may a resume certify?

Only on a socket's first request. A later request on that socket keeps its floor-only start but records no certifiedFrom: the sender's certificates still come from the old scan.

🤔 What keeps a dropped projection record a hole, and what fills it?

An immutable hole row, written before the frame continues. A later clean full pass from any peer fills it. The supported fill is rebuilding or replacing the node (owner decision); it restores no admission until multi-shard databases certify.

🤔 Scope: what this PR fixes and what it defers

Fixed here: one writer per [seq, P] row, copy bounds in core admission, level-2 certificates and revocation. Findings outside that, or failing closed, are filed below.

Review findings by group

Group	Root cause	Findings	Where
(a) fixed	Committed copy-bound cache trusted another pass's droppable reservation	acb9246c57b4 43d9b0cdaff9 79fe22c4c194	this PR
(a) fixed	Persisting after settlement recreated the reservation	00e0810a13e8	this PR
(a) fixed	Settling a resumed pass dropped coverage for rows applied past its cursor	72b84e768b51 85ab21ec64ad 238bf979cdda	this PR
(a) fixed	Bound rows rescanned on every load	fce3ec0201ac 4dd9d4d19ed0 6ac76a1b7edc	this PR
(a) fixed	A failed seq-row write never reached onFailure, so settled marks were not restored	e908e5160fcc 26f54d289722	harper#3649
(a) fixed	DESIGN wording, an unused reader kind, a duplicate JSDoc	10955718de8c 23fb66ca41e6 8c6376082589 62fbea7b88ad 3e99cf2b9b80	this PR
(a) accepted	No unit test for withholding at send time; the cluster downgrade case covers it	6eb80ca1b8de	—
(b)	Widening missed after a restart when only a relayed entry was partial	da2a355d3a8f a8ec00b5de22 c4a4d5b5d469 69b54eb3ea40 f6d281cc0871 23198bd0d7e5 6895a4fa7d82	#1142
(b)	Route marks and barriers on copies over level-0/1 links	6cd60f539035 0747669bc2a3 c009b177a6c2 c3d4ce6370f2 8a7bd2b89c71 601a145a39bf	#1143
(b)	Whether every row joins the copy-start commit	ee3112723b06 f412b930fc54 0acdd2741138 72f7d61a139a 849f35bd3b6c	#1144
(b)	Route facts not scoped to their generation and connection	c270b88ee348 29d8d2da2625 6549a1e5faaa 247b71f29a9b 4c6e2fc2412a 7cf0df1e945d e4f6508a1b21 51eeffe90a50 758cb83ad1d9	#1145
(b)	Copy-pass completion: restore before finalization, stuck cursor removal	5f72a66bbbf2 7fa8347d5e7f 0e0bda27443d 120974ab1c7e 2d1031542b98 1f4888f5d2c9	#1146
(b) admission	Rollback of the persisted semantics; live boundary below a certified start	3c04c6e4c0b3 (+ core 411148f0f6b5 9dc0b8f1a507 b02ad4d37a82)	harper#3647

Earlier deferrals: #1134 (bound in the copy's own transaction), #1135, #1136, #1137, harper#3636, harper#3648, harper#3669.

Approaches weighed

• Detect a violation on apply instead of binding the origin generation: unsound for a restored origin, whose next certificate is higher and whose filtered keys never reach a detector (owner ruling 6a).
• Clear a revocation on a later, higher floor: would accept a token whose inclusive seek skips the violating key. Rejected (planning review).
• Leave tokens out when a known origin is reached only over a partial route: contradicts strict availability; issuance is disabled instead.
• Framing-Verdict: chosen-approach-sound (ce4683c6692b), for one writer per row with copy bounds applied at admission.
• Projection holes (this round): one mutable hole row per peer, released by that peer's next clean pass, lost a later loss merged into a captured row, revalidated old checkpoints (a snapshot restores no audit entry), and stranded a departed peer's hole. Rechecks returned better-alternative-exists (42e6e216d43c, then 602b067138d7), adopted: immutable revisions, each origin's own boundary, a fill from any peer. Gemini: chosen-approach-sound (602b067138d7).
• Hole recovery: no existing operation starts a from-the-beginning pass (update_node start_time only raises the start, remove_node keeps the [seq, peer] rows, FORCE_CLONE reuses the data directory). The owner chose node rebuild or replacement as the supported fill, a full-copy operation as #1220, and no automatic re-copy: the recurring drop causes would loop full copies.

🔧 Changes

• replication/protocolCapabilities.ts: ORIGIN_FLOORS_CAPABILITY = 2, peerCertifiesCheckpointFloors reads the resolved level, buildLocalCapabilities advertises level 2 only when the local floors are durable, resolvePeerCapabilities takes the lower of the two sides.
• replication/copyCursorWatermark.ts: CopyCursorValue persists widened, fullCoverage, and maxCopiedVersionByOrigin across copy resume.
• replication/replicationConnection.ts: parseOriginCertificates and encodeOriginCertificate carry [floor, generation, fullCoverage] or 0 per level; collectRelayableCertificates forwards level-2 fields only from a whole, hole-free row whose socket certifies now and whose certifiedFrom is above any [copyBound, originId]; PendingRouteState and SharedRouteSafetyState key marks and revocations by generation, settle them on commit, and verify them via a post-seq-write txnStream barrier read-back; the floor-only cursor vector starts a checkpoint-grade row below its floor and records certifiedFrom; a copy writes only its own bound rows: a reservation committed before the first row, with logarithmic headroom above the copy start, then settled to the exact versions once its rows are durable; a row waits only on bounds that stay committed (readCommittedCopyBounds); bounds are read through core's fail-closed reader; a level-2 row whose socket dropped a level relays nothing until revoked, at send time too; a level-1 floor clear and a coverage restore ride the route's own end_txn; a failed sequence-row write releases a frame's commit slot once; the downgrade path revokes every level-2 row of a peer whose handshake stopped certifying; the sender tracks certified generations and sends 0 once per revoked generation; flushRouteState sends marks right after the request without awaiting the subscription placeholder; CopyPassState decides whether a completed copy clears partialHistory and restores fullCoverage behind the txnStream barrier; the shared-status slot holds the level.
• core: pinned to harper main 2dcd7fb82, main's own pin, which contains the merged companion harper#3649; this PR no longer moves it.
• replication/replicationConnection.ts (this round): routeRowId addresses marks, read-backs and holes to the row core writes, and verifyPersisted checks each batch against its own event's row; bindSharedRouteSafety is keyed by the subscription; only a socket's first request certifies; a projection's drop writes an immutable hole revision, read at COPY_START by readCopyHoles and filled by settleCopyBounds; one warning per newly holed origin; copyHoleStatus feeds cluster_status sharding.<db>.checkpointHoles; refused sends go through afterRefusedSend, which adopts a replacement placeholder once it resolves.
• replication/DESIGN.md: the row-ownership rule; item 33 states the certificate, the floor-only resume and the mark and revocation lifetimes; the route's row and certification needs a fresh socket; DESIGN.md: the index entry.
• Test fixture fixture-origin-log: ClosedFloors exposes the level-2 fields, CheckpointSubscription issues and resumes checkpoints, DurableSessionRecord reads and writes the MQTT session row; schema.graphql exports checkpoint_origin; config.yaml loads it; standinSyncWal.mjs provides a preload stand-in for syncWal() in cluster tests.

Product and architecture tour

Product and architecture tour

What a level-2 certificate promises

What does a receiver learn from a floor certificate that item 2's did not carry, and who may send one?

Before	After
{ origin: floor } from any peer with originFloors ≥ 1; a stored floor says nothing about the origin's history identity, the route's table coverage, or whether the peer still certifies.	{ origin: [floor, generation, fullCoverage] } between two level-2 peers; 0 revokes; a level-1 peer still receives { origin: floor } byte for byte.

Level 2 is advertised only by a RocksDB build whose engine syncs the floor record before advertising it (syncWal). The resolved level is the lower of the two sides.
A relay forwards the level-2 fields only from a row whose own socket certifies at level 2 now, never from a row with a hole or partial history, so a downgrade or a gap stops at that hop.

Floor-only resume and the route's proof

Why does a reconnect on a level-2 route start exactly below the stored floor rather than at the higher applied position?

A reconnect on a level-2 route

1. Start below the floor — the request resumes at below(F), never max(position, below(F)). A transaction reserved below the sender's position and committed during the disconnect is re-sent instead of filtered. This lets a stored floor mean a complete local prefix.
2. Record where the proof begins — the start is queued as the route's certifiedFrom, tagged with its origin generation. Core keeps the lowest start of the route that certifies the floor.
3. Mark what breaks the proof — a dropped record marks the origin hole; a route widened to full coverage marks partialHistory on every origin of that row until a copy begun afterwards completes.
4. Revoke on downgrade — when a direct peer's handshake no longer certifies at level 2, every origin on its generation-bearing row is revoked. Revocation sticks within the generation; the floor is cleared and relayed as 0.
5. Carry the marks — marks and revocations ride the next sequence update of any shape and settle only when it commits. A downgraded peer sends none, so a receiver sends its own right after the request: never inside an open frame, never awaiting the subscription placeholder.

A floor-only start is never raised by a recorded transaction time: that would let the sender filter what committed between them, the gap the floor exists to close.

Example: a peer that comes back without the capability

A restarts as an older binary (no capability bag) while B and C hold level-2 floors for it.

Outcome: B revokes A at the next request and relays 0 to C; C issues no checkpoint while A is revoked, and when A returns on level 2 the revocation sticks until A's generation changes.

Membership and generations

How can a resume on another node distinguish an origin it never saw from one the issuer certified, without naming healthy origins?

Certificates, token and resume across A, B and C

A certifies to B; B relays to C; C issues a token the client resumes on B.

flowchart LR
    A[Origin A] -- "[F, gen, full]" --> B[Node B]
    B -- "relayed [F, gen, full]" --> C[Node C]
    C -- "token: digest over members" --> Client
    Client -- "resume with token" --> B
    B -- "digest matches own rows" --> B

Loading

The digest and the resume check live in core (HarperFast/harper#3356): an XOR of sha256(origin, generation) over the members, and a resume that subtracts at most four late joiners from the node's own candidates before refusing with a 410. This PR supplies what those need from replication: the generation and coverage each certified row carries, and the revocation that removes a row from the candidates. Where two generations of one origin coexist, core's seq-row merge keeps the one whose rows saw the higher floor; this PR tags every mark and revocation with the generation it was derived under so a retained row of a removed peer never speaks for the restored origin.

✅ Verification

• Unit and cluster suites pass on the final tree (details below).

This round's evidence

• Local cluster runs on a84a36e6, one at a time: shardingBootstrap 5/5 ×2, checkpointHoleRebuild 2/2 ×3, subscriptionCheckpointResume 5/5, removedOriginResumeCursor 3/3, declarativeResidency 7/7, recordLockRestartAvailability 9/9. Replication unit suite: 1702 passing, 14 pending.
• A regression with core's real sequence updater: a copy lands between a disconnected route's read and write of its own row, and admission must refuse. Red on e0715821, where the void was written back; green now.
• Bound commits: one reservation and one settle per copy pass in the cluster suite; 10,000 future-dated rows commit 13 bounds.
• Sharded bootstrap with syncWal: shardingBootstrap.test.mjs failed 0/5 on 45ff074; it passes 5/5 on this head (runs below).
• Route marks land on the row core writes for sharded × failover routes, certification only from a socket's first request, a projection hole durable before completion and released only by a clean complete pass: each red on 45ff074. Admission after a fill: hole 410 → fill → old token still 410 → admitted after the pass. A refused send's replacement placeholder times out without its fix.
• checkpointHoleRebuild.test.mjs: a projection drops a record (decode-failure hook), B warns and lists the hole, then B is replaced and the new node holds the record and lists none. Its mkdtemp roots are removed by teardownHarper for every node, the retired one included.

Sharded bootstrap with syncWal: cause and fix

• Cause. In the four-node test, node 1 cannot verify node 2's CA, so failover routes node 2's system subscription through another node. Core writes that route's state to [seq, 2]; this PR marked and read [seq, 1]. COPY_START's partialHistory mark never landed on the row core reads, the read-back failed forever, and each failure re-queued and re-flushed inline. The http worker sat at 100%, answered no sharding RPC, and every source refused with "Sharding source has no frozen origin start".
• Fix. routeRowId() (receivingDataFromNodeIds[0]) for marks, runPostSeqWriteChecks, received floors and holes; PendingRouteState.verifyPersisted checks each batch against its own event's row; CopyPassState.routeId; bindSharedRouteSafety keyed by the subscription; a failed read-back retries on the timer.
• Test premise. At level 2 a floor-only resume needs no closed-prefix wait, so the second case restarts node 2 with HARPER_TEST_FORCE_ORIGIN_FLOORS_LEVEL_1.

Deferred, ready to file (owner's session files it)

A multi-shard database admits no subscription checkpoint. Every cross-shard route is restricted, so it reports routeFullCoverage = false (replication/replicationConnection.ts, the restrictedOrigins checks). A cross-shard origin's level-2 certificate is stored with fullCoverage: false and never qualifies in core's admission (subscriptionCheckpoint.ts qualifies). Strict availability then issues no checkpoint on any node, and admission refuses every token that names a cross-shard origin. Full coverage on such a route has to mean "every record this node should hold from that origin" for sharded tables, plus every record for ordinary tables. Copy-hole rows take effect once this lands. The warning text and DESIGN.md call it "tracked separately". Parent: #922.

Executed evidence

• A failed sequence update names no position, which threw inside the frame-failure handler; holdFailedFrame now counts it. The cursorless copy case counts the widening log line as its second full copy, since a level-2 route that stopped mid-copy is marked partial.

• The MQTT failover case retries a replacing SUBSCRIBE until the session record carries a checkpoint before it holds the transaction: core (harper#3778) adopts a first checkpoint only at a save after the floors pass what the session acknowledged.

• The sharding unit tests (residencyControls.test.mjs) shadow syncWal: rocksdb-js 2.13.0 syncs natively, so every runner is level 2 and would otherwise count checkpoint barriers beside the sharding traffic.

• PR QA ("Does a base copy through one route void a disconnected route's checkpoint certification on the same node?") qa-pr-harper-pro-1092-0c0c53d covered the pre-rebase head a1c827cd, before the rebase onto main with sharding (Replicate sharded tables through database shard maps #1116) and the core re-pin to harper main.

• unitTests/replication/originFloors.test.mjs (new cases: certificate parsing/encoding per level, relay gating by level, floor-only vector, removed-origin start, pending route state, copyBound voiding, partial-coverage CopyPassState, and clearGenlessClosedFloorOnSeqRow) and protocolCapabilities.test.mjs; full npm run test:unit (result below).

• integrationTests/cluster/subscriptionCheckpointResume.test.mjs, A–B–C with a capability-off E, 5 cases: level-2 certificates carry the generation and coverage on B (direct) and C (one hop) and not on E's link; a restart records certifiedFrom; a checkpoint issued on C names no origin and resumes on B, delivering a transaction A held open across it, and a foreign member set is refused with 410; an MQTT durable session fails over from C to B and receives the late lower-key transaction; a route widened from partial coverage to full coverage marks partialHistory and restores after a full copy while preserving lowest-wins certifiedFrom across a second floor-only reconnect; A restarted on level 1 is revoked on B and on C through B's relay, C issues no checkpoint while it is, the revocation sticks when A comes back on level 2, and C restarted without a capability bag is revoked on B; plus integrationTests/cluster/removedOriginResumeCursor.test.mjs (3/3 passing).

• The system database does not replicate in this topology, so the MQTT case copies the session row to B through the fixture where replication would carry it.

• On a rocksdb-js without native syncWal() (before 2.13.0), standinSyncWal.mjs installs a stand-in syncWal() on RocksDatabase.prototype via --import and in fixture-origin-log/resources.js until the dependency is bumped past Add syncWal() to sync the write-ahead log without a memtable flush rocksdb-js#949.

• Full npm run test:unit on the final tree: 2062 passing. subscriptionCheckpointResume.test.mjs 2/2 and 3/3 across the final heads, removedOriginResumeCursor.test.mjs 3/3.

• Checkpoint read cost (harper#3649's bench, bench host, turbo off): 125 µs with no copy bound rows, 202 µs with 16 rows of 16 origins, once per 5 s capture and per resume.

_{🤖 Generated by Claude Fable 5.1; posted via @kriszyp.}

Depends-on: HarperFast/harper#3649, HarperFast/rocksdb-js#949

Release target: main — new replication capability level, dependent on rocksdb-js syncWal() (HarperFast/rocksdb-js#949)

Related PRs: #554 overlaps, #717 overlaps, #815 overlaps, #940 overlaps, #1086 overlaps, #1101 overlaps, #1103 overlaps, #1112 overlaps, #1116 overlaps (merged 2026-10-28; sharded routes take no floor-only start and report no full coverage; all touch replication/replicationConnection.ts), #997 overlaps (core submodule pointer), #1075 overlaps (core submodule pointer, merged 2026-10-19), #1087 overlaps (residency stubs a copy skips must not clear partial history or a hole), #1065 overlaps (merged 2026-10-20; core f9890778 must descend from its core commit), #1060 overlaps (merged; its concurrent seq-row save test sits beside this PR's single-writer regression), #1140 overlaps (merged; copy-cursor and blob path in the same copy loop), #1133 overlaps (merged core sync; the core pin is now harper main), #1131 overlaps (merged core repin), #1130 overlaps (merged; sharding read path in replicationConnection.ts), #1054 overlaps (open; worker-thread default, touches replicationConnection.ts), #1177 overlaps (merged; the full-write-head exports this PR's core pin now includes), #285 overlaps (merged; forced base-copy resync in the same copy loop), #1192 overlaps (merged; non-owner read settlement in replicationConnection.ts), #1202 overlaps (open; replication pool default, same file), #1204 overlaps (merged; core bump), #1210 independent (pins the review-coverage action, which clears this PR's red review-coverage check)
Complexity: complicated

Deferred findings, each filed

Deferred findings

• a27ac0db862da1d9 major data-integrity — core/resources/subscriptionCheckpoint.ts — admission orders database generations by floor timestamp — filed Checkpoint admission orders an origin's database generations by floor timestamp harper#3636
• ab4feaca0f8afbc5 minor reliability — replication/replicationConnection.ts — one malformed [copyBound, *] row fails every persist and wedges checkpoint-capable copies in a 1011 reconnect loop — filed A malformed copy bound row stops checkpoint-capable base copies with no repair path #1137
• a596c3edd1cfc41c minor data-integrity — replication/replicationConnection.ts — route coverage not persisted for widening after restart — filed Widening after a restart is missed when only a relayed origin recorded partial coverage #1142
• d34a04883d53415d minor performance — replication/replicationConnection.ts — COPY_START manufactures route marks and barriers on legacy links — filed Checkpoint route marks and barriers run on base copies over level-0/1 links #1143
• 968396cdbac99542 minor data-integrity — replication/DESIGN.md — rollback can reopen admission on copy-voided certifiedFrom — filed Enforce the copy-bound ordering contract in core, where admission reads it harper#3647
• b0968145b2f2ad4b minor correctness — replication/replicationConnection.ts — delayed verification restores partial=true after a later copy clear — filed Scope route-state facts to the generation and connection that produced them #1145
• 3d8f16bc1ebf3966 minor concurrency — replication/replicationConnection.ts — hole callback unfenced after its await — filed Scope route-state facts to the generation and connection that produced them #1145
• 1a1399754d736386 minor reliability — replication/replicationConnection.ts — initially unscoped safety facts never bind to a generation; pending revocation blocks the next generation — filed Scope route-state facts to the generation and connection that produced them #1145
• 79e52501c546301f minor reliability — replication/replicationConnection.ts — verifyPersisted skips generation-scoped marks; unverified batches strip certifiedFrom/fullCoverage — filed Scope route-state facts to the generation and connection that produced them #1145
• 9c091d6823317a7d minor correctness — replication/replicationConnection.ts — partial history clears before copy blob finalization — filed Copy-pass completion: restore whole history only after finalization; unstick cursor removal #1146
• +26 more in the review receipt

Late Grok findings

5 surviving (highest minor) — Grok reviewed 220b2fa

• minor — replication/replicationConnection.ts:9772 — a route widened to full coverage still resumes incrementally, so rows of the newly included tables committed before the cursor are never requested. The partialHistory mark is set again on every reconnect (persistedPartial, :9766-9768). Only a fresh copy clears it, because tryRestoreCopyPass re
• minor — replication/replicationConnection.ts:9610 — the exclusion-origins-updated include path calls currentOriginCursorVector() without the floor-only callback, so buildOriginCursorVector falls through to resumeStartWithFloor (:3277-3279) and returns max(position, cursorBelowFloor(floor)). Trigger: an origin was excluded while this node re
• minor — replication/replicationConnection.ts:5486 — disputed: on a level-2 socket, a bare-floor certificate calls noteGeneration(id, undefined) and erases the cached generation, so a later revoke(id) (:5472) and hole() (:3008) carry no generation. Contrary to the lens's reading, a sibling core checkout treats an undefined generation as matching
• minor — integrationTests/cluster/fixture-origin-log/resources.js:122 — the SetRouteCoverage fixture cannot work, for four separate reasons:
• nit — replication/replicationConnection.ts:2951 — disputed: checkpointGradeRow does not check hole, so a row with a hole still resumes floor-only and gets a certifiedFrom mark. Floor-only resume starts lower, so it re-sends more; it cannot skip records. Core's checkpoint excludes rows with a hole (sibling subscriptionCheckpoint.ts:138, `:18

Not standing: none. Grok ran 17m (17m from launch to report); the fold adjudication (cursor-grok@220b2fabeb50-1b271d0a, claude) took 3m.

Late Grok findings

6 surviving (highest minor) — Grok reviewed 801d9da

• minor — replication/replicationConnection.ts:7119 — On a sharding-restricted route whose peer is still in exclusionOrigins, the received certificate is stored with index 2 (relayable) = directFullCoverage.includes(name). Only routeFullCoverage (:7043) looks at restrictedOrigins. That row, which carries a generation and `fullCoverage:false
• minor — replication/replicationConnection.ts:5387 — A completed projection settles only the current copyPassToken. Earlier projection attempts each wrote a [copyBound, peer, 'reserved', token] row at the anchor for every origin (:8021, :6893-6903). settleCopyBounds (:3943-3946) drops only its own token. Projections have no resume cursor,
• minor — replication/replicationConnection.ts:5731 — A hole recorded during projectionCopy (recordReplicationHole :6339-6341) is queued, but every projection end_txn has remoteNodeIds: [], so the mark is never attached or settled. The projection then closes the socket (:5397). A restart before another socket on the same worker flushes the
• minor — replication/replicationConnection.ts:6688 — bindSharedRouteSafety is keyed only by (database root, peer name) and runs on every socket after NODE_NAME (:7503). The inbound leg to the same peer therefore takes ownership: it strips unverified certifiedFrom from the outbound route's live state and replaces onSettled. When that inbound
• nit — replication/DESIGN.md:585 — The Row-ownership note contradicts itself. "The reservation's over-approximation lasts only while the copy runs" and "only voids starts at or below its own copy start" conflict with "An earlier attempt's reservation stays" in the same paragraph, and with copyBoundReservation headroom, which reserv
• nit — replication/replicationConnection.ts:6975 — Comments that narrate the code: :6975 ("re-read, never assumed"), the tuple restatement at :3131-3133, integrationTests/cluster/subscriptionCheckpointResume.test.mjs:61, unitTests/replication/originFloors.test.mjs:801.

Not standing: none. Grok ran 16m (16m from launch to report); the fold adjudication (cursor-grok@801d9da207ce-ec3d1342, claude) took 2m.

Late Grok findings

4 surviving (highest major) — Grok reviewed 8e07e80

• major — replication/replicationConnection.ts:5759 — A hole recorded during a sharding projection never reaches [seq, peer] before the projection settles its bounds and completes (:5414-5424). Scope: in-scope — Convergence-Class: previously-adjudicated
• major — replication/replicationConnection.ts:12644 — On a same-socket resubscription, certifiedFrom is persisted (flushRouteState at :12680) before the replacement SUBSCRIPTION_REQUEST is sent. Scope: in-scope — Convergence-Class: previously-adjudicated
• minor — replication/replicationConnection.ts:6692 — Inbound and retrieval sockets for the same (database, peer) take ownership of the shared route-safety state. Scope: in-scope — Convergence-Class: previously-adjudicated
• nit — replication/DESIGN.md:590 — The reservation-lifetime text contradicts the code. Scope: in-scope — Convergence-Class: previously-adjudicated

Not standing: none. Grok ran 9m (10m from launch to report); the fold adjudication (cursor-grok@8e07e80ac281-a29463ce, claude) took 1m.

Late Grok findings

5 surviving (highest major) — Grok reviewed 45ff074

• major — replication/replicationConnection.ts:12644 — On a same-socket resubscription, certifiedFrom is persisted before the replacement request takes effect. Scope: in-scope — Convergence-Class: previously-adjudicated
• major — replication/replicationConnection.ts:5759 — A hole recorded during a sharding projection never reaches [seq, peer] before the projection settles its bounds and completes (:5414-5424). Scope: in-scope — Convergence-Class: previously-adjudicated
• minor — replication/replicationConnection.ts:568 — A failure recorded with no position is never cleared, because the clear at :11075 compares a number against undefined. Scope: in-scope — Convergence-Class: response-code
• minor — replication/replicationConnection.ts:6692 — Inbound and retrieval sockets for the same (database, peer) take ownership of the shared route-safety state. Scope: in-scope — Convergence-Class: previously-adjudicated
• nit — replication/DESIGN.md:590 — The reservation-lifetime text contradicts the code. Scope: in-scope — Convergence-Class: previously-adjudicated

Not standing: none. Grok ran 15m (15m from launch to report); the fold adjudication (cursor-grok@45ff0749d7d3-b06e416e, claude) took 1m.

Late Grok findings

2 surviving (highest minor) — Grok reviewed a84a36e

• minor — replication/replicationConnection.ts:5583 — when the local queue refuses the copy-finish event, the relayed-origin anchors it carried are lost. drainCopyOriginAnchors moves them into pendingOriginCursors, and seqUpdateEndTxn then takes them onto the event and clears the map (replication/replicationConnection.ts:6823-6827). A refused e
• minor — replication/replicationConnection.ts:11375 — five long-standing send sites ignore a false return from send(): replication/replicationConnection.ts:8340, :10674, :11186, :11375 and :12236. So when the local queue closes while the socket stays up, the connection keeps the closed queue, and records and sequence updates are dropped w

Not standing: none. Grok ran 12m (12m from launch to report); the fold adjudication (cursor-grok@a84a36e6f8de-e965fb2c, claude) took 1m.

_{Review-Coverage: authored=claude; ran=cursor-composer,gemini,codex,cursor-muse,cursor-grok; late=cursor-grok:adjudicated; adjudicated=domain; blocked=claude(fallback)(out-of-budget),muse(failed); declined=cursor-kimi; rounds=42; full=10 @ a84a36e}

_{Review-Attention: deep ~190m (critical: copyCursorWatermark.ts, protocolCapabilities.ts +2; decisions: do-less-alternative, refused-send-recovery-by-adoption, hole-fill-by-node-replacement, sticky-revocation-within-generation, strict-checkpoint-availability, level2-gated-on-syncwal, route-row-is-remotenodeids0, release-target-unclear) @ a84a36e}

[gemini-code-assist]

Code Review

This pull request implements checkpoint-grade certificates, floor-only resume, and sticky revocation for per-origin subscription checkpoints (harper-pro#922 items 3–4). It introduces level 2 originFloors capability, allowing nodes to resume exactly below a stored floor and carry origin generation and full coverage details, with comprehensive integration and unit tests added to verify these replication behaviors. Feedback on the changes suggests explicitly checking for undefined in replication/replicationConnection.ts when updating the certified start position, rather than comparing a potentially nullish value using the <= operator.

[kriszyp]

Converted to draft so this does not merge before the round-7 pre-push review findings are fixed. The two new majors:

• A hole can be lost on a crash (replication/replicationConnection.ts:5010 / :8389): the frame's end_txn persists the cursor past a dropped record, but the hole rides a later sequence update.
• A copy can leave a stale certifiedFrom (:9499): the mark is recorded when the request is sent and survives if the request turns into a copy, so a checkpoint inside the copied span can be admitted.

Two earlier majors are still open as well: :2996 (settle before the sequence row commits) and :4478 (copy callback reads live flags after an await). A dedicated effort: high fix task will take all four, starting with a framing recheck.

— Claude Opus 5.5