[KV Offload] Carry request provenance through stored events - #52103
Draft
Change72 wants to merge 1 commit into
Draft
[KV Offload] Carry request provenance through stored events#52103Change72 wants to merge 1 commit into
Change72 wants to merge 1 commit into
Conversation
Co-authored-by: OpenAI Codex <noreply@openai.com> Signed-off-by: Change72 <changg@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Carry request provenance through asynchronous KV-offload completions so locally initiated stores and promotions can emit full
BlockStoredevents.The current tracker snapshots full payloads at scheduler lookup/store time and keeps them in a global map. This PR instead keeps a lazy event locator with the request and builds the payload when the raw stored event arrives.
Changes
ReqContext.ReqContextthrough CPU, FS, OBJ, cascade, and promotion completion.extra_keys, including multimodal and prompt-embedding inputs.Scope
This PR does not change:
OffloadKey = (block_hash, group_idx);prepare_storeadmission or batching;Single-key admission, k-way ordering, and #44865 remain separate work.
This PR is based on the KV-group block-granularity fix merged in #51614. Token-mutating paths affected by #49377 keep placeholder payloads until their hash and offload state are safe.
Duplicate-work check
No open PR above carries request provenance through async stored-event completion.
Tests
All validation ran on the workstation against
3d204dfdaa:Model evaluation: not applicable. This changes event metadata and lifecycle only, not model execution or output.
AI assistance
AI assistance was used to trace the event lifecycle, implement the change, and draft tests. The human submitter must review every changed line and these test results before submission.