Skip to content

api: split a request's payload from its envelope - #457

Closed
wseaton wants to merge 4 commits into
llm-d:mainfrom
wseaton:payload-split
Closed

wseaton wants to merge 4 commits into
llm-d:mainfrom
wseaton:payload-split

Conversation

@wseaton

@wseaton wseaton commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

Adds api.SplitPayload, JoinPayload, and AttachPayload, which #452 and #458 use to store a request's payload apart from its envelope. Stacked on #456 (the last two commits are new).

Release note

Adds `api.SplitPayload`, `api.JoinPayload`, and `api.AttachPayload` for storing a request's payload apart from its envelope.

The worker forwards the queued bytes instead of re-encoding a map, and parses the payload for the span's model only when the span is sampled.

Signed-off-by: Will Eaton <weaton@redhat.com>
Producers that know the model set it, so the worker labels its span without parsing the payload. The payload is still parsed for a sampled span when Model is empty.

Signed-off-by: Will Eaton <weaton@redhat.com>
Transports that store the payload apart from the rest of the request read the envelope without scanning the prompt, then attach the payload bytes as stored.

Signed-off-by: Will Eaton <weaton@redhat.com>
Signed-off-by: Will Eaton <weaton@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant