Currently, channel tarballs are tightly coupled to the Nix client code, there's two areas to look for:
nix-channel, easy to fix and already went through the bz2 → xz change.
- What I call: path element processing, the process to transform
channel:nixos-26.05 into some proper URL, same happen for Flakes.
First is easy to mitigate by introducing more compression types to try, ideally with an HEAD request first then a proper GET.
Second is slightly harder because it gets used in three ways:
- When search path resolution takes place, it gets expanded. That resolution takes place when many evaluation-relevant paths gets expanded for the interpreter to start and evaluate things.
- Expansion may start even earlier than the interpreter requires it, e.g. auto-completion of installables and various CLI preprocessing (
lookupFileArg)
fetchTree's fetchTarball will rewrite URLs.
1/2 are solveable again by the same technique as nix-channel, i.e. introduce a more complicated logic to resolve "pseudo URL" into URLs and attempt them in order, the slight problem is that it introduces needless latency and this latency is never cached or learned. For anyone who might use Nix to lock historical inputs and relies on bz2 or xz nixexprs, they will always pay this latency from now on.
3 is the worst because fetchTarball logic is generic, the solution is similar, instead of making fetchTarball hit one URL, it can hit many URLs and pseudo URL resolution can return as many candidates as needed.
An alternative way to look at this problem is to use the same techniques that channels have been using: redirections. Communicate the compression type via headers on the leaf and put an URL with a redirect to the most up-to-date compression as long as client software can deal with that Content-Encoding.
Currently, channel tarballs are tightly coupled to the Nix client code, there's two areas to look for:
nix-channel, easy to fix and already went through the bz2 → xz change.channel:nixos-26.05into some proper URL, same happen for Flakes.First is easy to mitigate by introducing more compression types to try, ideally with an HEAD request first then a proper GET.
Second is slightly harder because it gets used in three ways:
lookupFileArg)fetchTree'sfetchTarballwill rewrite URLs.1/2 are solveable again by the same technique as
nix-channel, i.e. introduce a more complicated logic to resolve "pseudo URL" into URLs and attempt them in order, the slight problem is that it introduces needless latency and this latency is never cached or learned. For anyone who might use Nix to lock historical inputs and relies on bz2 or xz nixexprs, they will always pay this latency from now on.3 is the worst because
fetchTarballlogic is generic, the solution is similar, instead of making fetchTarball hit one URL, it can hit many URLs and pseudo URL resolution can return as many candidates as needed.An alternative way to look at this problem is to use the same techniques that channels have been using: redirections. Communicate the compression type via headers on the leaf and put an URL with a redirect to the most up-to-date compression as long as client software can deal with that
Content-Encoding.