Skip to content

Base copy records COPY_START[3] relayed anchors as receiver cursors at COPY_COMPLETE before the copy tail drains #1056

Open

Description

@kriszyp

Summary

When a fresh base copy starts (replication/replicationConnection.ts), the sender collects each relayed origin log's last committed key (collectRelayedLogAnchors) before the table walk and sends that map in COPY_START[3]. On the receiver, maybeFinishCopy (replication/replicationConnection.ts:4026-4033) calls noteOriginProgress(originId, key) for every entry in copyOriginAnchors as soon as COPY_COMPLETE arrives and in-flight copy blobs settle — before the sender's append-order copy tail (boundaryRange) has streamed entries appended to those relayed logs during the copy walk.

Impact

For an exact-anchored relayed log, the table walk covers all records present in the primary tables when the walk scanned them, while the subsequent append-order copy tail (resumeAfterExactStart) is responsible for delivering any relayed transaction appended to the copy source's log after anchor was snapshotted — including transactions whose origin timestamp key is older than anchor (late-arriving relayed writes).

Because the receiver persists anchor as its originLogKey cursor at COPY_COMPLETE:

  1. If the connection drops after COPY_COMPLETE finishes and before the copy tail delivers a post-anchor relayed entry whose key is more than ORIGIN_CURSOR_OVERLAP_MS (60 s) below anchor,
  2. The subsequent reconnect sends anchor in SUBSCRIPTION_REQUEST[4] and the sender resumes that origin log by key at anchor - 60 s rather than in append order past anchor, skipping that late-committed entry.

This is the same 60 s overlap heuristic that any reconnect after the tail already relies on (replication/DESIGN.md items 7 and 26, tracked toward closed-floor resume in #922), but recording COPY_START[3] at COPY_COMPLETE opens that reconnect window before the initial copy tail has drained.

Possible direction

Wait to seed the receiver's per-origin cursors from COPY_START[3] until the first post-copy end_txn trailer confirms the initial copy tail pass has drained (or subsume under durable append-order / closed-floor resume in #922).

Review-Deferral: 3a4a5d8c28e26a16 severity=nit category=data-integrity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Fields

    Priority

    P3

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions