This isn't a bug; it's more of a design limitation, so worth filing as a feature request.
Summary
ladybug.replay_replication() reads the full replication log every time it
runs. It does not remember which entries a previous call applied.
This works when materializing an empty graph from the full log. It does not work
well for keeping an existing graph up to date. A second call tries every old
INSERT again. Since INSERTs use CREATE, they fail on duplicate primary keys.
The log also has no status or error columns. After a replay, callers cannot tell
which entries were applied, which failed, or which still need to run without
reading the session's NOTICE messages.
Could replay_replication() support an incremental mode that starts after the
last processed log entry? If the function is intended only for rebuilding an
empty graph, documenting that restriction would also help.
Environment
- PostgreSQL 18.4 (Homebrew), macOS arm64
pg_ladybug main @ e5182153f1d4b588e7fd758d69c1406a6649a79f
liblbug 0.19.1
Reproduction
CREATE DATABASE bugrepro;
\c bugrepro
CREATE EXTENSION pg_ladybug;
CREATE TABLE node_city (
id bigint PRIMARY KEY,
name text NOT NULL
);
SELECT ladybug.register_node('City', 'node_city', 'id', NULL, 'demo');
SELECT ladybug.enable_replication('demo', 'INSERT,UPDATE,DELETE');
INSERT INTO node_city VALUES (1, 'Toronto'), (2, 'Vancouver');
Run both replays in the same session:
SET ladybug.storage_path = '/tmp/bugrepro.lbdb';
SET ladybug.pg_connstr = 'host=/path/to/socket port=5432 dbname=bugrepro user=me';
SELECT * FROM ladybug.cypher(
'CREATE NODE TABLE City(id INT64, name STRING, PRIMARY KEY(id))'
) AS t(ok text);
SELECT ladybug.replay_replication('demo'); -- first run
SELECT ladybug.replay_replication('demo'); -- second run
Result
The first replay returns 2. The second replay returns 0 after attempting
both INSERTs again:
NOTICE: ladybug: replay skipped: CREATE (n:City {id: 1, name: 'Toronto'})
DETAIL: ladybug: query failed: Runtime exception: Found duplicated primary key value 1, ...
NOTICE: ladybug: replay skipped: CREATE (n:City {id: 2, name: 'Vancouver'})
DETAIL: ladybug: query failed: Runtime exception: Found duplicated primary key value 2, ...
NOTICE: ladybug: replayed 0 change(s), skipped 2 (ensure native ladybug node/rel tables exist)
replay_replication
--------------------
0
The log is unchanged:
SELECT count(*) FROM ladybug.replication_log('demo'); -- still 2
A second replay from a new PostgreSQL connection also triggers the backend
crash described in the separate SIGABRT report. Keeping both calls in one
session isolates the incremental replay behavior without that crash.
Why this matters
There is no reliable way to apply only new changes. Duplicate entries and
genuine failures both appear as skipped statements, and the result exists only
in NOTICE output.
In a larger test, the log contained 996 entries. Replay reported 160 applied
and 836 skipped, but it left no durable record of which entries failed.
Possible approach
- Store a per-graph checkpoint, or accept a starting log ID and return the last
processed ID.
- Record the status and error for each entry.
- Allow failed entries to be retried without replaying successful entries.
- Document full-log replay as a rebuild operation if that remains its intended
purpose.
This isn't a bug; it's more of a design limitation, so worth filing as a feature request.
Summary
ladybug.replay_replication()reads the full replication log every time itruns. It does not remember which entries a previous call applied.
This works when materializing an empty graph from the full log. It does not work
well for keeping an existing graph up to date. A second call tries every old
INSERT again. Since INSERTs use
CREATE, they fail on duplicate primary keys.The log also has no status or error columns. After a replay, callers cannot tell
which entries were applied, which failed, or which still need to run without
reading the session's
NOTICEmessages.Could
replay_replication()support an incremental mode that starts after thelast processed log entry? If the function is intended only for rebuilding an
empty graph, documenting that restriction would also help.
Environment
pg_ladybugmain @e5182153f1d4b588e7fd758d69c1406a6649a79fliblbug0.19.1Reproduction
Run both replays in the same session:
Result
The first replay returns
2. The second replay returns0after attemptingboth INSERTs again:
The log is unchanged:
A second replay from a new PostgreSQL connection also triggers the backend
crash described in the separate SIGABRT report. Keeping both calls in one
session isolates the incremental replay behavior without that crash.
Why this matters
There is no reliable way to apply only new changes. Duplicate entries and
genuine failures both appear as skipped statements, and the result exists only
in
NOTICEoutput.In a larger test, the log contained 996 entries. Replay reported 160 applied
and 836 skipped, but it left no durable record of which entries failed.
Possible approach
processed ID.
purpose.