Skip to content

Consequences of changing compression types for channel tarballs #1146

Description

@RaitoBezarius

Currently, channel tarballs are tightly coupled to the Nix client code, there's two areas to look for:

  1. nix-channel, easy to fix and already went through the bz2 → xz change.
  2. What I call: path element processing, the process to transform channel:nixos-26.05 into some proper URL, same happen for Flakes.

First is easy to mitigate by introducing more compression types to try, ideally with an HEAD request first then a proper GET.
Second is slightly harder because it gets used in three ways:

  1. When search path resolution takes place, it gets expanded. That resolution takes place when many evaluation-relevant paths gets expanded for the interpreter to start and evaluate things.
  2. Expansion may start even earlier than the interpreter requires it, e.g. auto-completion of installables and various CLI preprocessing (lookupFileArg)
  3. fetchTree's fetchTarball will rewrite URLs.

1/2 are solveable again by the same technique as nix-channel, i.e. introduce a more complicated logic to resolve "pseudo URL" into URLs and attempt them in order, the slight problem is that it introduces needless latency and this latency is never cached or learned. For anyone who might use Nix to lock historical inputs and relies on bz2 or xz nixexprs, they will always pay this latency from now on.

3 is the worst because fetchTarball logic is generic, the solution is similar, instead of making fetchTarball hit one URL, it can hit many URLs and pseudo URL resolution can return as many candidates as needed.

An alternative way to look at this problem is to use the same techniques that channels have been using: redirections. Communicate the compression type via headers on the leaf and put an URL with a redirect to the most up-to-date compression as long as client software can deal with that Content-Encoding.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions