PS-9822 Improve documentation on MyRocks variables (8.0) #517
Conversation
patrickbirch
changed the title
On branch ps-9822 modified: docs/variables.md
| In database systems, concurrency means that multiple operations can run simultaneously. By default, write operations are handled in order using a single thread queue. The first thread in line becomes the leader and performs the write to both the Write-Ahead Log (WAL) and the memtable, one after the other. This setup limits the number of writes that can happen simultaneously. | ||
|
|
||
| The pipelined write feature allows WAL and memtable writes to happen in parallel. When `rocksdb_enable_pipelined_write` is set to `ON`, RocksDB uses separate threads for writing to the Write-Ahead Log (WAL) and the memtable. A thread waits in the WAL queue and then moves to the memtable queue. In the WAL queue, each thread waits only for earlier WAL writes to finish. They do not wait for memtable updates. | ||
|
|
||
| This setup allows WAL and memtable operations to run in parallel. It improves write throughput and can reduce latency during the prepare phase of a two phase commit. |
There was a problem hiding this comment.
The pipelined write feature in RocksDB is designed to improve concurrent write throughput when the Write-Ahead Log (WAL) is enabled. Normally, all writers share a single write thread queue, and each write proceeds sequentially through WAL and then memtable updates. This serialization limits concurrency.
Pipelined writes change this by allowing WAL and memtable operations from different writers to overlap. Specifically, once a writer finishes its WAL write, the next writer in the queue can begin its own WAL write—even if the previous writer hasn't yet completed its memtable update. This interleaving boosts write throughput and reduces latency.
When rocksdb_enable_pipelined_write = ON, this behavior is enabled, making it particularly beneficial in workloads with multiple concurrent writers and in scenarios like the prepare phase of a two-phase commit.
On branch ps-9822
modified: docs/variables.md