Skip to content

Report a resident's full-write head in GET_RECORD answers, so readers keep replicas only of it - #1177

Merged
kriszyp merged 2 commits into
mainfrom
kris/replica-full-write-head
Oct 28, 2026
Merged

kriszyp merged 2 commits into
mainfrom
kris/replica-full-write-head

Conversation

@kriszyp

@kriszyp kriszyp commented Oct 28, 2026 •

Copy link
Copy Markdown
Member
Kris stamps FULL WRITE on a file before his son telegraphs it, while his daughter holds an unstamped relayed copy

⊙ Problem

Core now keeps a resident's GET_RECORD answer as a replica only when the resident certifies that its newest write replaced the whole record (Keep a replica only of a resident's full-write head, and skip writes older than such a head without reading the audit log). Only the resident knows that: a non-resident receives every put and patch as an invalidate. Without this PR, core keeps no replicas at all.

🤔 Should the answer carry the resident's head kind, rather than the stub?

Yes. Tagging each converted put on the invalidate instead changes the audit entry format and must survive relays and copies. The answer field is additive, and readers that don't know it ignore it.

💡 Solution

  • GET_RECORD answers add fullWriteHead, the resident's head log key, only when its row is flagged FULL_WRITE_HEAD. Readers keep a replica only of their stub's own head.
  • No capability gate: older readers ignore the field; older responders omit it, so nothing they answer is kept.
  • No node certifies a copied body, including a relayed one (part of harper#3703).
  • Merge after the core PR.

🤔 Should a copied put an older build logged be treated as a copy?

It has no COPIED_RECORD, so a relay of it during a rolling upgrade can still be certified. Marking it needs the copy-read position tracked in harper#3704, awaiting a decision there.

🔧 Changes

✅ Verification

  • Four-node residencyReplicas.test.mjs, new case: a patch head is not kept; a put head is kept and renews while hot. It fails on main (patch head kept) and without the field (nothing kept). Suite 9/9.
  • residencyReplicaWire.test.mjs: the field appears only for a put head, and copies carry no FULL_WRITE_HEAD. 49 passing with residencyWire; copy suites 125 passing.
  • Three-node star copyProvenanceRelay.test.mjs, with its fixture (resources, config, schema): a relayed copied put is not certified on the next hop and still supersedes a late increment. It fails without the receive mapping.
  • Answer cost, measured by a review run: uncertified answers unchanged; certified ones +23 bytes and about 90 ns.
  • PR QA: After concurrent puts and patches to hot keys, does any non-resident keep serving a body that differs from its residents' once writes stop? — queued @ df9f30f, qa-pr-harper-pro-1177-df9f30f
  • Review: the last round left no surviving finding; the CLI still reports not-converged because 12 earlier leg findings carry no classification record.

🤖 Generated with Claude Code

https://claude.ai/code/session_01F7groUxcrz6wPPXNzUKCnx

🤖 Generated by Claude Code (Claude Opus 5.5); posted via @kriszyp.

Release target: v5.4 — the companion core change (replica tier) is only on main/v5.4
Related PRs: #1092 overlaps (same send and receive paths: keep the copy-walk mask and the copyFrame mapping), #1116 overlaps (copy walk and GET_RECORD responder: shard paths must add fullWriteHead), #1133 overlaps (core pointer: this pin is newer and contains the companion commit), 1 others independent
Complexity: medium

Deferred findings

🤔 defer this PR's major data-integrity finding 937b04141c08bb45 (below-audit-floor fold changes body at unchanged version; certified replica renews stale) to HarperFast/harper#3684 instead of fixing it here?

Late Grok findings

1 surviving (highest major) — Grok reviewed ffc986c
  • major — replication/replicationConnection.ts:655 — the live invalidation stub built by residencyStubOf holds only index fields and leaves out the created-time value. Core's residencyStub (standalone harper/resources/Table.ts:11117-11128) adds the created time. The copy walk adds it as well, by hand (`replication/replicationConnection.ts:7320-7

Not standing: none. Grok ran 13m (13m from launch to report); the fold adjudication (cursor-grok@ffc986c3867e-040b3c86, claude) took 2m.

Late Grok findings

3 surviving (highest nit) — Grok reviewed c70c064
  • nit — unitTests/replication/residencyReplicaWire.test.mjs:155 — Mostly fixed. The answer-field comment at replication/replicationConnection.ts:6121 and the cited test comments at old lines 117 and 268 are gone. The comment "a resident whose head is the stub's own full write" is still in the unit test. It is probably the cited old :157 comment and it repeats th
  • nit — unitTests/replication/residencyReplicaWire.test.mjs:156 — The kept-replica test writes the reader's own stub version into fullWriteHead instead of taking it from the responder. That test therefore can't catch a wrong head computed on the responder side. Other tests cover that: :200-202 checks the head the responder sends for a full write and for a patc
  • nit — replication/replicationConnection.ts:6121 — No Pro test sets a delete head and checks the head key it produces. The author says a delete reaches a non-resident node as a tombstone, not a stub, so that head is never used; this was not verified because core isn't checked out

Not standing: none. Grok ran 12m (13m from launch to report); the fold adjudication (cursor-grok@c70c06488b1b-29e0fef8, claude) took 1m.

Review-Coverage: authored=claude; ran=cursor-composer,gemini,codex,cursor-grok; late=cursor-grok:adjudicated; adjudicated=domain; declined=muse; rounds=11; full=9 @ 23690ed

Review-Attention: deep ~25m (critical: replicationConnection.ts; decisions: ungated-wire-field, accept-rolling-upgrade-certification-gap, repin-as-separate-branch, deferred-937b0414; raised: degraded review) @ 23690ed

… keep replicas only of it

The answer carries fullWriteHead, the responder's head log key when its row is flagged
FULL_WRITE_HEAD (core), and only then. Core keeps a replica only when that key is the stub's head,
so a body whose head is a patch is served but never kept. An older reader ignores the field; an
older responder omits it, so its answers are never kept.

A copy frame, or a relayed entry carrying core's COPIED_RECORD audit flag, applies with copyFrame,
so no node certifies a copied body (part of harper#3703). Relays forward the flag unchanged in the
entry's bytes. The copy walk masks the record-only bit from the audit flags it mirrors, and builds
a replica's projection in core's demotion order (primary key first, since core #3610), so it
encodes as the demoted stub does. Bumps core to the companion change.

Dispatch-Task: harper-replica-row-version-consistency
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F7groUxcrz6wPPXNzUKCnx
@kriszyp kriszyp added this to the v5.4 milestone Oct 28, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request ensures that replicas are only kept when the resident head is a full write, and that copied puts maintain their provenance through relays without being certified. It updates the replication protocol to include fullWriteHead in GET_RECORD_RESPONSE when applicable, filters out FULL_WRITE_HEAD from audit flags, and marks copy frames appropriately. New integration and unit tests are also added to verify these behaviors. The review feedback suggests adding explicit parentheses around a bitwise operation in replicationConnection.ts to improve readability and prevent potential linter warnings.

Comment thread replication/replicationConnection.ts Resolved
Dispatch-Task: pr-maint-harper-pro-1177-repin
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017xoBABrkJLf8fw26t48sNb
@kriszyp
kriszyp marked this pull request as ready for review October 28, 2026 11:23
@kriszyp
kriszyp merged commit 58fd1f6 into main Oct 28, 2026
62 of 64 checks passed
@kriszyp
kriszyp deleted the kris/replica-full-write-head branch October 28, 2026 11:23
kriszyp added a commit that referenced this pull request Oct 28, 2026
Keeps #1177's fullWriteHead in GET_RECORD answers alongside this branch's
residencySlot; the copy walk keeps this branch's copyExtendedType, which
already masks FULL_WRITE_HEAD, so main's copiedFlags is dropped. Core moves
from harper#3527's head to its squash merge b29241296 (adds only #3706, #3708).

Dispatch-Task: pr-maint-harper-pro-1116-rebase
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017xoBA8UtKfgJoZfMDdzpyv
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant