Skip to content
This repository was archived by the owner on Apr 13, 2023. It is now read-only.

proxy: download packages concurrently #33

Open
shoenig opened this issue Sep 8, 2018 · 0 comments
Open

proxy: download packages concurrently #33

shoenig opened this issue Sep 8, 2018 · 0 comments

Comments

@shoenig
Copy link
Contributor

shoenig commented Sep 8, 2018

Currently the proxy has a singe background.Reloader goroutine, which every 30 seconds or so queries the registry for a complete list of modules, and then either ignores (already have it) or downloads each module from the upstream source one at a time. Instead, every iteration when it's time to download modules, spin up a configurable pool of worker threads per-domain which can download more than one module at a time from that upstream domain. That way, even if we want to limit ourselves to downloading from 1 VCS at a time, we're still downloading N modules at a time, 1 from each of the VCSs that appear in the list of modules to keep.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Development

No branches or pull requests

1 participant