mistral-rs: fix cuda support

[GaetanLepage]

Description of changes

cc @SomeoneSerge

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[GaetanLepage]

Result of nixpkgs-review pr 343254 run on aarch64-linux 1

1 package built:

• mistral-rs

[GaetanLepage]

Result of nixpkgs-review pr 343254 run on x86_64-linux 1

1 package built:

• mistral-rs

[Aleksanaa]

Can we use autoPatchelfHook with runtimeDependencies (Since there are only this two executables)?

https://nixos.org/manual/nixpkgs/unstable/#setup-hook-autopatchelfhook

[GaetanLepage]

Thanks for the tip. I went and try it and it doesn't work weirdly...
I'm pushing the changes.

[GaetanLepage]

The ldd output is the same, but I guess that the rpath is not about that.

[GaetanLepage]

This is the result of patchelf --print-rpath for the two variants.

[Aleksanaa]

Suggested change

	runtimeDependencies = lib.optionals cudaSupport [
	cudaPackages.libcublas
	cudaPackages.libcurand
	];

Try this. I guess it's because autoPatchelfHook overrides rpath (--set-rpath) instead of prepending/appending (--add-rpath) it.

nixpkgs/pkgs/build-support/setup-hooks/auto-patchelf.py

Lines 303 to 307 in 122d20b

	if rpath:
	print("setting RPATH to:", rpath_str)
	subprocess.run(
	["patchelf", "--set-rpath", rpath_str, path.as_posix()] + extra_args,
	check=True)

But you already have these two in buildInputs, and it seems unneeded to add them to runtimeDependencies again, see torch as an example:

nixpkgs/pkgs/development/python-modules/torch/bin.nix

Lines 45 to 77 in 122d20b

	nativeBuildInputs = lib.optionals stdenv.isLinux [
	addDriverRunpath
	autoPatchelfHook
	autoAddDriverRunpath
	];
	buildInputs = lib.optionals stdenv.isLinux (
	with cudaPackages;
	[
	# $out/${sitePackages}/nvfuser/_C*.so wants libnvToolsExt.so.1 but torch/lib only ships
	# libnvToolsExt-$hash.so.1
	cuda_nvtx
	cuda_cudart
	cuda_cupti
	cuda_nvrtc
	cudnn
	libcublas
	libcufft
	libcurand
	libcusolver
	libcusparse
	nccl
	]
	);
	autoPatchelfIgnoreMissingDeps = lib.optionals stdenv.isLinux [
	# This is the hardware-dependent userspace driver that comes from
	# nvidia_x11 package. It must be deployed at runtime in
	# /run/opengl-driver/lib or pointed at by LD_LIBRARY_PATH variable, rather
	# than pinned in runpath
	"libcuda.so.1"
	];

[Aleksanaa]

I guess we have to add something like extraRuntimeDependencies to autoPatchelfHook then.

[Aleksanaa]

Also try this:

nixpkgs/pkgs/development/python-modules/torch/bin.nix

Lines 71 to 77 in 122d20b

	autoPatchelfIgnoreMissingDeps = lib.optionals stdenv.isLinux [
	# This is the hardware-dependent userspace driver that comes from
	# nvidia_x11 package. It must be deployed at runtime in
	# /run/opengl-driver/lib or pointed at by LD_LIBRARY_PATH variable, rather
	# than pinned in runpath
	"libcuda.so.1"
	];

I guess runtimeDependencies is still not necessary, unless cuda libraries are loaded by dlopen

[GaetanLepage]

Thank you for your suggestions !
I tried removing the

runtimeDependencies = lib.optionals cudaSupport [
  cudaPackages.libcublas
  cudaPackages.libcurand
];

block but it doesn't work better.
Those libs are loaded at runtime (dlopen I guess) and thus are not seen as required by patchelf`.

[GaetanLepage]

I guess runtimeDependencies is still not necessary, unless cuda libraries are loaded by dlopen

This works for the cuda lib but it still complains about cublas...

[SomeoneSerge]

I guess we have to add something like extraRuntimeDependencies to autoPatchelfHook then.

appendRunpathsArray exists, used for adding $ORIGIN in a few places

[Aleksanaa]

@GaetanLepage Could you fix this to a usable state? I suppose it's at least better than nothing

[GaetanLepage]

@GaetanLepage Could you fix this to a usable state? I suppose it's at least better than nothing

Sure. Maybe let's merge the update first and then I'll rebase this branch.
#352064

[Aleksanaa]

Done

[GaetanLepage]

Currently fails to build with cudaSupport enabled:

error[E0063]: missing field `seed_value` in initializer of `cuda_backend::device::CudaDevice`
   --> /build/cargo-vendor-dir/candle-core-0.7.2/src/cuda_backend/device.rs:203:12
    |
203 |         Ok(Self {
    |            ^^^^ missing `seed_value`