Source path error improvements

[edolstra]

Motivation

Based on #12532 but heavily modified.

We now "mount" the source accessors of flake inputs on top of /nix/store (i.e. the MountedSourceAccessor named storeFS). This allows error messages similar to the lazy-trees branch, i.e. referring to the original source trees (like a Git repo) instead of the store path to which the input was copied.

For example, instead of

error: attribute 'foobar' missing
at /nix/store/ddzfiipzqlrh3gnprmqbadnsnrxsmc9i-source/machine/configuration.nix:209:7:
   208|
   209|       pkgs.foobar
      |       ^
   210|     ];

you now get

error: attribute 'foobar' missing
at /home/eelco/Misc/eelco-configurations/machine/configuration.nix:209:7:
   208|
   209|       pkgs.foobar
      |       ^
   210|     ];

Context

---

Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

[Mic92]

Cool. Could you write a changelog message for this? This is a major UX improvement.

[roberth]

Why is this a hack?

[Ericson2314]

Yeah where is FilteringSourceAccessor even being used anymore? I only see it used with GitExportIgnoreSourceAccessor

[roberth]

Why is this a hack?

[roberth]

Not a bug:

Suggested change

	state.allowPath(storePath); // FIXME: should just whitelist the entire virtual store
	state.allowPath(storePath); // TODO: we could simply whitelist the entire virtual store, because all of that is created by the evaluator and its allowed external sources

[roberth]

In fact, we don't even need to copy the inputs to the store at all, so
this gets us very close to lazy trees. We just need to know the store
path so that requires hashing the entire input, which isn't lazy. But
the next step will be to use a virtual store path that gets rewritten
to the actual store path only when needed.

This could use some clarification so that others can follow.
We have multiple operations or optimizations that may be called "lazy":

• lazy copying: copy as little to the store as possible
• lazy evaluation and lazy locking: completely ignore an input if possible (already somewhat possible, and improving with Make nix flake metadata|update|lock lazy #12432, Reuse input lock files #7730)
• lazy outPath: use input metadata without fetching anything (e.g. (fetchTree foo).rev) probably already provided by call-flake.nix.
• fetch as little as possible (we don't call this laziness, but someone might, and that'd be confusing)
• incremental fetching: e.g. Git submodules and LFS only on access
• lazy hashing: hash as little as possible, avoiding read operations from local cache

What you're suggesting as the next step is to provide lazy copying.
Lazy hashing and incremental fetching would be part of #11367 or #6530.

I hope I'm not adding to the confusion.

[Ericson2314]

I suggest we do this in conjunction with #12531

[Ericson2314]

I changed #12531 as discussed in the meeting, and then rebased this on top in https://github.com/Ericson2314/nix/tree/source-path-errors