Skip to content

Copy-pass completion: restore whole history only after finalization; unstick cursor removal #1146

Open

Description

@kriszyp

Problem

The copy-pass completion lifecycle in replication has two gaps. Both fail closed.

  1. Restoring partialHistory: false requires only completeReceived and seqDurableAtCopyStart, not the copy watermark's finalization. On a repeated or resumed copy, the anchor can already be at or below the durable watermark, so the restore can land while a copy blob is still outstanding. A base copy sets no certifiedFrom, so nothing is admitted. The route is only marked whole early, until the next reconnect re-marks it.
  2. isCopyPassRouteRestoredDurable returns false and re-queues nothing when a captured origin is missing from [seq, peer] (or carries a genless fullCoverage: false). Copy-cursor removal then retries every second, and the cursor survives into the next restart.

Possible direction

Gate restoration on the watermark's finalization barrier. Make the cursor-removal check re-queue the missing origin's restoration, or drop origins that no longer exist.

Raised by the cross-model review on harper-pro#1092 and deferred there: it is outside that PR's invariant.

Review-Deferral: 5f72a66bbbf2
Review-Deferral: 7fa8347d5e7f
Review-Deferral: 0e0bda27443d
Review-Deferral: 120974ab1c7e
Review-Deferral: 2d1031542b98
Review-Deferral: 1f4888f5d2c9
Review-Deferral: 9c091d6823317a7d severity=minor category=correctness
Review-Deferral: ca549b89e1c9767e severity=nit category=reliability

Also deferred here from harper-pro#1092's later rounds (see the comment below): earlier attempts' reservations after a completed projection, the separate-commit fullCoverage restore, a narrow unit test pinning isCopyPassRouteRestoredDurable to the route row, and that check refusing forever (so the copy cursor is never removed) when the route row's own node keeps fullCoverage: false without a generation a restore mark can name, or when an origin captured at COPY_START has no node in the row any more.

Review-Deferral: 4214b28e7ada5c62 severity=nit category=reliability
Review-Deferral: 396ae72049e60539 severity=nit category=reliability
Review-Deferral: 335f7f85ee85a0c6 severity=nit category=test-coverage

Activity

  1. added theissue type on Oct 27, 2026
  2. kriszyp commented on Nov 2, 2026

    @kriszyp
    MemberAuthor

    Two more deferrals from harper-pro#1092's review belong here:

    • A completed projection leaves earlier attempts' reservations. Settlement releases only its own [copyBound, peer, 'reserved', attempt] row. An earlier abandoned attempt's reservation stays and keeps voiding the starts it covers, which can reach above that attempt's copy start because of reservation headroom. A projection has no resume cursor, so a completed projection could release every earlier attempt's reservation from the same peer.

      Review-Deferral: 4214b28e7ada5c62 severity=nit category=reliability

    • The fullCoverage restore rides a separate commit. When a full-coverage copy heals fullCoverage: false, partialHistory: false commits with the finish event, but fullCoverage: true is queued only in the cursor-removal barrier's onCommit. Its mark is also not kept in the unverified batches. A socket closing in the retry window, or a sequence write that does not persist the mark, leaves the copy cursor in place.

      Review-Deferral: f692bef233f0

    — Claude Opus 5.5

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Fields

    Priority

    P3

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions