Skip to content

Commit ce0234e

Browse files
committed
Update transaction pipelining docs re: parallelism
Fixes DOC-8591 Specifically, we state that the cost of writes is ~`O(1)` in the number of inserts, which is misleading. The reality is that despite the parallelism introduced by pipelining, there is other work that happens for each write that does not come "for free".
1 parent e68493c commit ce0234e

File tree

2 files changed

+6
-2
lines changed

2 files changed

+6
-2
lines changed

src/current/v25.4/architecture/life-of-a-distributed-transaction.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -116,7 +116,7 @@ The batch evaluator ensures that write operations are valid. Our architecture ma
116116

117117
If the write operation is valid according to the evaluator, the leaseholder sends a provisional acknowledgment to the gateway node's `DistSender`; this lets the `DistSender` begin to send its subsequent `BatchRequests` for this range.
118118

119-
Importantly, this feature is entirely built for transactional optimization (known as [transaction pipelining]({% link {{ page.version.version }}/architecture/transaction-layer.md %}#transaction-pipelining)). There are no issues if an operation passes the evaluator but doesn't end up committing.
119+
Importantly, this feature is entirely built for transactional optimization (known as [transaction pipelining]({% link {{ page.version.version }}/architecture/transaction-layer.md %}#transaction-pipelining)). For important caveats about what pipelining does and does not change in end-to-end latency, see that section. There are no issues if an operation passes the evaluator but doesn't end up committing.
120120

121121
### Reads from the storage layer
122122

src/current/v25.4/architecture/transaction-layer.md

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -352,6 +352,10 @@ COMMIT;
352352

353353
With transaction pipelining, write intents are replicated from leaseholders in parallel, so the waiting all happens at the end, at transaction commit time.
354354

355+
{{site.data.alerts.callout_info}}
356+
Clarification: Transaction pipelining overlaps the Raft consensus work for intent writes across statements. It does not make individual SQL statements free. Each statement must still be planned and evaluated (e.g., index lookups, constraint checks, conflict detection, and waiting on contending writes), and the client submits statements sequentially. Statements that touch the same rows can also create pipeline stalls to preserve read-your-writes ordering. As a result, while the consensus component of write latency can approach O(1) with respect to the number of statements, end-to-end transaction latency can still scale with the number of statements due to SQL evaluation and dependency stalls.
357+
{{site.data.alerts.end}}
358+
355359
At a high level, transaction pipelining works as follows:
356360

357361
1. For each statement, the transaction gateway node communicates with the leaseholders (*L*<sub>1</sub>, *L*<sub>2</sub>, *L*<sub>3</sub>, ..., *L*<sub>i</sub>) for the ranges it wants to write to. Since the primary keys in the table above are UUIDs, the ranges are probably split across multiple leaseholders (this is a good thing, as it decreases [transaction conflicts](#transaction-conflicts)).
@@ -362,7 +366,7 @@ At a high level, transaction pipelining works as follows:
362366

363367
1. When attempting to commit, the transaction gateway node then waits for the write intents to be replicated in parallel to all of the leaseholders' followers. When it receives responses from the leaseholders that the write intents have propagated, it commits the transaction.
364368

365-
In terms of the SQL snippet shown above, all of the waiting for write intents to propagate and be committed happens once, at the very end of the transaction, rather than for each individual write. This means that the cost of multiple writes is not `O(n)` in the number of SQL DML statements; instead, it's `O(1)`.
369+
In terms of the SQL snippet shown above, all of the waiting for write intents to propagate and be committed happens once, at the very end of the transaction, rather than for each individual write. This means the consensus-related waiting is not `O(n)` in the number of SQL DML statements; instead, it approaches `O(1)`. The overall client-observed latency still includes per-statement planning and any pipeline stalls, so it does not, in general, become strictly `O(1)`.
366370

367371
### Parallel Commits
368372

0 commit comments

Comments
 (0)