DRAFT: Convert upsert IO pattern from sender-receiver to fiber based#2019
Draft
DRAFT: Convert upsert IO pattern from sender-receiver to fiber based#2019
Conversation
Introduces a Boost.Fiber scheduler that integrates with AsyncIO's io_uring polling. This enables fibers to yield while waiting for io_uring completions and be resumed when the operation completes. Key components: - CompletionToken: A struct with a magic number that distinguishes fiber completions from sender-receiver operations in the CQE - UringFiberScheduler: A fiber scheduler that polls io_uring during suspend_until, enabling cooperative multitasking with IO - poll_uring_ extension: Detects fiber completions via magic number and schedules the waiting fiber for resumption The scheduler is installed in the rwdb worker thread, preparing the infrastructure for fiberizing the upsert path. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Convert the upsert and copy_trie code paths from callback-based async IO to fiber-based synchronous-style IO. This eliminates the complex `node_receiver_t` callback machinery and sender-receiver node writers in favor of simpler `fiber_read_node()` and `FiberWriteBuffer` patterns. Key changes: - Add `submit_fiber_read` and `submit_fiber_write` to AsyncIO with proper inflight tracking for poll_uring_ to find completions - Add `is_write` field to CompletionToken to track operation type for correct inflight counter decrement on completion - Add `FiberWriteBuffer` class that accumulates writes and yields on flush - Replace `node_writer_fast`/`node_writer_slow` sender-receivers with `fiber_write_buffer_`/`fiber_write_buffer_slow_` - Replace `async_read` callbacks with `fiber_read_node` synchronous calls - Remove `node_receiver_t` template and callback reentrancy handling - Rename `reset_node_writers()` to `init_disk_growth_tracking()` The fiber approach allows stack-based continuation instead of callbacks, making the control flow much easier to follow and maintain. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add fiber context tracking to the buffer pool. When a fiber goes to sleep waiting for memory, track it in a `waiter_` queue. Once a buffer is released, notify the first entry in the queue. The current approach, which is to yield when no memory could be accessed, is not fiber friendly. Yield will starve the scheduler. We need to detach the context for that fiber to be removed from the ready queue.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.