Add `find_nvidia_dynamic_library` to find the DSO location without loading

[leofang]

It’d be nice for path finder to gain a capability find_nvidia_dynamic_library, that only returns the DSO location that the path finder would load, but does not actually load it. cuFile and nvJitLink could use this.

[rwgk]

I floated this idea a few days ago:

This could be implemented in a few lines by calling load_nvidia_dynamic_library() in a separate process. This approach guarantees that load_nvidia_dynamic_library() and find_nvidia_dynamic_library() are 100% consistent. All major parts of the functionality needed exist already (in particular https://github.com/NVIDIA/cuda-python/blob/a93b760e01a16c7f80e7b802095a1d4d069c001e/cuda_bindings/tests/spawned_process_runner.py).

[kkraus14]

If a different library loaded in the current process had some RPATH set that influenced where it loaded something from, wouldn't that not be captured in a separate process?

[leofang]

Yes I've been thinking about this too. One notable example is how conda Python finds shared libraries via RPATH. nvmath-python currently also uses RPATH. Another potential divergence is if there is another library that already loads what we need, and pathfinder would jus reuse it.

I think find_nvidia_dynamic_library should present the solution that we would get without being influenced by other DSOs. So in a sense a potential divergence is inevitable.