feat: abstract over stream types on provide and get side #147
Conversation
… notifications/requests for each event type.
This shows how to limit serving content in various ways - by node id - by content hash - throttling - limiting max number of connections
so the other side can know if reconnecting is OK
Also make the whole get fsm generic so it can be used with an arbitrary stream, not just a quinn/iroh RecvStream.
rklaehn
changed the title
We only need async-compression as a dev dep for the compression example!
src/util/stream.rs
Outdated
| /// Send bytes to the stream. This takes a `Bytes` because iroh can directly use them. | ||
| fn send_bytes(&mut self, bytes: Bytes) -> impl Future<Output = io::Result<()>> + Send; | ||
| /// Send that sends a fixed sized buffer. | ||
| fn send<const L: usize>( |
There was a problem hiding this comment.
Why the const L instead of taking &[u8]?
There was a problem hiding this comment.
Since these traits are hopeless, they will never be dynable, I thought I might as well use a const generic since it is convenient. But sure, could change this.
In case of the recv version the code just reads nicer if you return a fixed size slice instead of taking a &mut [u8], and for returning a fixed size slice you don't have to allocate.
src/util/stream.rs
Outdated
| /// Note that this is different from `recv_bytes`, which will return fewer bytes if the stream ends. | ||
| fn recv_bytes_exact(&mut self, len: usize) -> impl Future<Output = io::Result<Bytes>> + Send; | ||
| /// Receive exactly `L` bytes from the stream, directly into a `[u8; L]`. | ||
| fn recv<const L: usize>(&mut self) -> impl Future<Output = io::Result<[u8; L]>> + Send; |
| Ok((value, data.len())) | ||
| } | ||
|
|
||
| async fn read_length_prefixed<T: DeserializeOwned>( |
There was a problem hiding this comment.
Yeah these are great. We should move all this with some cleanup and better docs to an iroh-util crate I think. It will be useful for many people working with iroh send/recv streams even independently of the trait abstraction.
There was a problem hiding this comment.
I am somewhat hesitant to have yet another crate, but I guess maybe we have to do it even before iroh 1.0?
hopefully this makes it more clear what they are for!
Also remove const generics and make the read and write fns a bit more "traditional". Makes usage a bit more inconvenient, but 🤷
|
cargo deny is just because we are pointing to iroh-* main. |
3 participants
Description
This PR introduces abstract versions of iroh::endpoint::{SendStream, RecvStream} and modifies the provide side and get side implementation to use these abstract streams instead of directly using iroh::endpoint streams.
This is necessary for wrapping the streams into a transformation such as compression, see the discussion in n0-computer/sendme#93 .
The compression example shows how streams can be wrapped into compression/decompression to create a derived protocol with a different ALPN that is identical to the blobs protocol except for compression.
Breaking Changes
iroh::endpoint::SendStream and iroh::endpoint::RecvStream are replaced with the traits iroh_blobs::util::SendStream and iroh_blobs::util::RecvStream in the get FSM and in the provider side API.
Notes & open questions
Change checklist