Feature Request: Importing Newer GHC backends?

[edgeofeverywhere]

Hello there!

My sincerest thanks for developing hix! It's genuinely amongst the most feature-complete devops solutions I've seen that leverage nix (for any language), and I'm surprised more people aren't using it!

From my use of it far, management of projects compiling code for standard host platforms enumerated in haskell.packages is seamless using hix for current GHCs (I especially appreciate the streamlining of cabal management with the declarative interface to hpack!). With that said, I would like to request a feature that would vastly simplify me finalizing an ideal pipeline for a project I'm working on.

My question is if you might be able to expose a toplevel option (ideally a configurable attribute set) for building the alternate cross-compilation backends of GHC built using Hadrian + ghc.nix or by simply importing the derivation ci artifacts (for the new wasm backend) over from ghc-wasm-meta, to be used within an env in lieu of the default flavours of GHC fetched from haskell.packages. Primarily, this is because I'm interested in using the new GHC LLVM/Emscripten-based backends for Java/Webassembly in lieu of the more mature (but ancestral) ghcJs tooling meant to target Reflex-dom and Obsidian that nixpkgs already has pkgsCross-based solvers baked-in for. There are a few reasons for this, but it's largely owing to many of my experiments getting liquid packages to compile within that context being largely fruitless owing to dependency conflicts that (from a day or two of me messing around with overrides) seem incommensurable between the LH versions committed to hackage around the release of GHC 8.10.7 and the obsidian backend.

Hopefully accomodating this doesn't require excessive codebase changes, since this functionality would be a great help to me, unless of course, you have word that the new backends will be mainlined to nixpkgs in the near future.

[tek]

Wow, glad to hear it's working so well for you, thanks for the feedback ☺️

Have you already attempted to build a package set on top of such a GHC outside of hix?

I would imagine that inlining some parts of nixpkgs' GHC derivation/package set machinery and tweaking the Hadrian build to produce the proper cross compiler would be a good way to start.
The result of that could be exposed from envs.dev.ghc.ghc or envs.dev.ghc.pkgsCross to integrate it into the build.

In nixpkgs I see two cross systems, wasi32 and wasm32-unknown-none, that might be useful as well for inspiration.

I'm happy to brainstorm this idea, but I don't have a clear picture of what this would require. I'll try to find some time to look at ghc-wasm-meta.

[edgeofeverywhere]

Don't worry, there's no need to rush! I can likely import from derivation and overlay my way into a solution that fits over hix specifically in my case if spend a bit of time on it. I may need to shoot you a few pms about it (let me know the best place) if I can't follow a thread or figure out a signature in a reasonable amount of time. I just figured I'd bring it up since despite the "new" cross-platform backends being almost two years old at this point (granted, that's nanoseconds relative to most of haskell's paradigm shifts), there's still no great way to access them for those haskellers bound to the nix ecosystem (who want to manage more complicated stacks) without needing to rehash a lot of old boilerplate along the way, since none of the providers of the major ghc infrastructures for nix have been willing to take the time to work out the kinks with them yet.

[TerrorJack]

Note that ghc-wasm-meta works by pulling binary artifacts and repackaging them as nix derivations.

Now, it's also possible to build GHC from source in nix, specifying any GHC revision with wasm backend support as build input. @MangoIV worked on it briefly, see https://gitlab.haskell.org/bgamari/ghcs-nix/-/merge_requests/33. This still requires pulling in wasi-sdk binary artifacts, but at least you can see in the linked MR, most of ghc derivation's build logic can be somehow reused with a bit of hacks on top. If you want to go even further, it's easy to build wasi-sdk from source as a native derivation as well.

The real challenge is, none of the above works with nixpkgs cross compilation infrastructure (the pkgsCross.xxx stuff) at all. Every nix derivation mentioned above is a non-cross native derivation. There exist pkgsCross.wasi32, but we cannot use the stdenv provided by it per se, we need our custom wasi-sdk fork for the time being. And my nix-fu is not sufficient to understand all the eldrich horrors regarding nixpkgs cross compilation internals.

So, regarding what can be done here: I literally know nothing about hix, how it works or if it works with cross targets at all. I'm just dropping quick conclusion here: if you're okay with adding some new nix derivations to make wasm32-wasi-ghc etc available, yes, that's easy and I can offer some help. If you want pkgsCross to work with ghc wasm backend at all, hats off to you and you're completely on your own