Skip to content

Record a base copy's checkpoint bound in the same transaction as its audit-less rows #1134

Open

Description

@kriszyp

Problem

A base copy writes rows without log entries (isCopyApply). Checkpoint admission must therefore treat a route's certifiedFrom as void when it is at or below the highest version a copy wrote for that origin. Today the copying connection keeps that bound in its own [copyBound, peer] row in __dbis__. It commits the bound before handing a copied row to core's apply loop: a write-ahead that runs about once per origin per copy pass (harper-pro#1092).

Recording the bound in the same transaction as the copied rows would remove that write-ahead entirely. It is not crash-safe today:

  • A rocksdb-js transaction takes its WAL setting from the store it was opened on (src/binding/transaction/transaction_handle.cpp, writeOptions.disableWAL = dbHandle->disableWAL).
  • Table stores open with the WAL off (core/resources/databases.ts, openRocksDatabase); __dbis__ opens with it on.
  • Column families flush independently. After a crash, a table column family can hold a flushed copied row while the __dbis__ memtable that held its bound is gone. Admission would then accept a stale certifiedFrom.

What it would take

One of these:

  • copy transactions written with the WAL on;
  • atomic flush across the database's column families;
  • a bound key stored in each table's own column family, which admission would then read per table.

Each one changes the audit-less copy durability contract, so it is a separate design.

Value

It removes the per-origin write-ahead (one awaited __dbis__ commit per origin per pass) and allows an exact per-origin bound instead of one reserved through the copy start. Low priority: the current design is correct, and its cost is about one commit per origin per pass.

Refs harper-pro#1092, harper-pro#922.

Review-Deferral: d22a669f4b94c44b severity=nit category=performance

Activity

  1. added theissue type on Oct 27, 2026
  2. kriszyp commented on Nov 2, 2026

    @kriszyp
    MemberAuthor

    Another deferral from harper-pro#1092's review belongs here. Each copy-applied row checks whether a committed bound covers its version, and a row above the committed bound awaits the bound commit before it is handed to core. That is cheap in practice: one reservation commit per pass, and a logarithmic number for future-dated rows. But it is a per-row check and a possible await on the copy apply path. Recording the bound in the copy's own transaction (this issue) would remove the await altogether.

    Review-Deferral: d22a669f4b94c44b severity=nit category=performance

    — Claude Opus 5.5

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Fields

    Priority

    P3

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions