Skip to content

Actually stream the data#31

Merged
mattt merged 6 commits into
huggingface:mainfrom
nathanborror:nathanborror/fix-streaming
Nov 17, 2025
Merged

Actually stream the data#31
mattt merged 6 commits into
huggingface:mainfrom
nathanborror:nathanborror/fix-streaming

Conversation

@nathanborror

Copy link
Copy Markdown
Contributor

Fixes a bug where data that is supposed to be streaming in is stalled until the stream has completed.

@nathanborror

Copy link
Copy Markdown
Contributor Author

I don't think I've fully fixed this yet :/

@mattt

mattt commented Nov 13, 2025

Copy link
Copy Markdown
Collaborator

Thanks for digging into this, @nathanborror!

I recall this working as intended in a previous release, so maybe this was a regression from #13. It could also be a bug in the upstream EventStream package that we use to process SSEs.

@nathanborror

Copy link
Copy Markdown
Contributor Author

I think the main issue is self.data(for: ...) doesn't deliver an "asynchronous sequence of bytes" like self.bytes(for: ...).

@mattt

mattt commented Nov 13, 2025

Copy link
Copy Markdown
Collaborator

I think the main issue is self.data(for: ...) doesn't deliver an "asynchronous sequence of bytes" like self.bytes(for: ...).

Oh yeah, that's exactly the problem. Sorry about that!

Comment thread Sources/AnyLanguageModel/Extensions/URLSession+Extensions.swift Outdated
@mattt mattt force-pushed the nathanborror/fix-streaming branch from a965626 to 3be3707 Compare November 17, 2025 11:36
@mattt

mattt commented Nov 17, 2025

Copy link
Copy Markdown
Collaborator

Thanks again for your help with this, @nathanborror! I just tested this out locally, and responses appear to be streaming as expected.

@mattt mattt merged commit 929712b into huggingface:main Nov 17, 2025
3 checks passed
@mattt

mattt commented Nov 17, 2025

Copy link
Copy Markdown
Collaborator

This is now available in 0.4.4

@nathanborror nathanborror deleted the nathanborror/fix-streaming branch November 19, 2025 05:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants