M2.1: Batch WAL writes - #9
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In fsync ALWAYS mode, instead of writing to wal file and fsyncing on each command, batch multiple requests and fsync them together.
In ALWAYS mode, batching emerges naturally from Go’s synchronization: each request acquires w.mu, appends its entry to the shared asyncBuffer, updates UncommittedLSN, and releases the lock. So multiple goroutines quickly accumulate writes into the same buffer without blocking on disk I/O.
The first goroutine that notices no flush is in progress flips isFlushing and signals the flusher via flushChan. The flusher then acquires w.mu (mutually excluding new writers), drains the entire buffer in one go, updates CommittedLSN while holding the sync.Cond lock (CommitSignal.L), and finally calls Broadcast() to wake all waiting goroutines.
Each waiting request, which is blocked on the condition variable until its LSN ≤ CommittedLSN, proceeds only after the batch is durably written. So batching is achieved by many goroutines serializing appends via a mutex, but amortizing a single disk flush across all of them, coordinated through a condition variable for durability acknowledgment.