-
Notifications
You must be signed in to change notification settings - Fork 44
[bundles] Add range requests to bundle download API #8183
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
// Note that we'll still stream data in packets which are smaller than this, | ||
// but we won't keep a single connection to Nexus open for longer than a 100 | ||
// MiB download. | ||
const CHUNK_SIZE: u64 = 100 * (1 << 20); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we're happy with this, I'll add something similar to the CLI once this PR merges
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the thought here that at some threshold we're testing our luck with regard to a timeout duration for a single query?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, exactly, 100 MiB seems small enough to not hit our 60 sec timeout. I could change this arbitrarily, but there's a cost with setting up each stream, so I didn't want to make it too small either.
(but this is semi-arbitrary; I can tweak the value if we feel strongly)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looks good to me!
// Note that we'll still stream data in packets which are smaller than this, | ||
// but we won't keep a single connection to Nexus open for longer than a 100 | ||
// MiB download. | ||
const CHUNK_SIZE: u64 = 100 * (1 << 20); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the thought here that at some threshold we're testing our luck with regard to a timeout duration for a single query?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great, thanks!
#8183 plumbed range request support through Nexus for the "download entire support bundle" API. This PR continues in that direction, plumbing range request support for all other "download portions of the bundle" APIs.
This PR adds range request support to the
support_bundle_download
endpoint in the internal and external API.It adds a
RangeRequest
header to both of these endpoints, and forwards this request to the underlying Sled Agent.Additionally, this PR updates the Sled Agent API to identify that range requests are supported. They were supported before this PR, but not encoded into the openapi spec until now.
This PR also adds: