Convert remaining JLL stdlibs to LazyLibraries

[IanButterworth]

Builds on #58405 (which I rebased, so the merge base here is master for now)

TODO:

• Convert the new Zstd_jll from [deps] enable zstd support #58344
• Convert LLD_jll? No, it's an executable not a lib
• Convert p7zip_jll? No, it's an executable not a lib
• Blocked by --trim failure trimming: no way to ccall into a dynamically-computed library name #57707 Instead it copies libzstd over for testing with a const path
• Fix linux RR test failure - release Fix _ossl_modules_path() returning wrong path to ossl-modules JuliaWeb/OpenSSL.jl#40

[topolarity]

Unfortunately the --trim failure is expected until we resolve #57707:

Core.MethodError(f=Base.Libc.Libdl.var"#dlsym"(), args=("/cache/julia-buildkite-plugin/depots/53de1ea5-6dd3-4008-b976-161c9f34b0ad/artifacts/4db1e58d71ac6bbb35ecc832033264c630d5d3b3/lib/libzstd.so", :ZSTD_versionString), world=0x00000000000097b7)

(LinearAlgebra is already broken for --trim on master for the same reason)

We do have a plan to fix it, but I don't know when it's going to happen since @Keno would like to revamp our ccall semantics (#57931) around the same time

[IanButterworth]

Ah ok.
In its current state, along with JuliaSparse/SparseArrays.jl#626 this PR gives

macOS M2 compared to results in #58436

julia> @time using Plots
  0.850111 seconds (1.29 M allocations: 74.658 MiB, 6.23% gc time, 2.34% compilation time)

Linux AMD Ryzen 9 5950X

julia> @time using Plots
  0.757390 seconds (1.38 M allocations: 77.337 MiB, 4.01% gc time, 3.59% compilation time)

And basic startup appears to be faster (if that makes sense?)
master vs. PR

% hyperfine './julia --start=no -e "1+1"' '/Users/ian/.julia/juliaup/julia-nightly/bin/julia --start=no -e "1+1"'
Benchmark 1: ./julia --start=no -e "1+1"
  Time (mean ± σ):      89.3 ms ±   3.6 ms    [User: 58.6 ms, System: 29.0 ms]
  Range (min … max):    85.4 ms …  98.2 ms    30 runs

Benchmark 2: /Users/ian/.julia/juliaup/julia-nightly/bin/julia --start=no -e "1+1"
  Time (mean ± σ):      99.2 ms ±   2.1 ms    [User: 67.8 ms, System: 29.7 ms]
  Range (min … max):    97.0 ms … 107.3 ms    27 runs

Summary
  ./julia --start=no -e "1+1" ran
    1.11 ± 0.05 times faster than /Users/ian/.julia/juliaup/julia-nightly/bin/julia --start=no -e "1+1"

[IanButterworth]

Should/could cglobal be taught how to handle a LazyLibrary?

[staticfloat]

It could, the most annoying thing is just that cglobal() is an intrinsic, not a function, so to add an overload for that involves writing C code, not just Julia code.

[topolarity]

Can / should we add a test that checks the dependency list against the ldd (or DT_NEEDED entries) of the .so's?

[vtjnash]

BinaryBuilder should already audit that, except for a few know broken libraries

[topolarity]

BinaryBuilder should already audit that, except for a few know broken libraries

Right, but it would be good to make sure that we haven't fallen out-of-sync with BinaryBuilder

[staticfloat]

it would be good to make sure that we haven't fallen out-of-sync with BinaryBuilder

I agree, I added a test that can be run on macOS or Linux. I (or someone else feeling brave) can make one for Windows as well. It already catches plenty of minor issues:

┌ Warning: Dependency mismatch
│   jll = "SuiteSparse_jll"
│   library = "libbtf"
│   missing_deps = ""
│   extraneous_deps = "libsuitesparseconfig, libamd, libcolamd"
└ @ Main ~/src/julia/test/stdlib_dependencies.jl:169

┌ Warning: Dependency mismatch
│   jll = "SuiteSparse_jll"
│   library = "libldl"
│   missing_deps = ""
│   extraneous_deps = "libsuitesparseconfig"
└ @ Main ~/src/julia/test/stdlib_dependencies.jl:169

┌ Warning: Dependency mismatch
│   jll = "SuiteSparse_jll"
│   library = "libspqr"
│   missing_deps = "libstdc++, libgcc_s"
│   extraneous_deps = "libamd, libcolamd"
└ @ Main ~/src/julia/test/stdlib_dependencies.jl:169

┌ Warning: Dependency mismatch
│   jll = "SuiteSparse_jll"
│   library = "libumfpack"
│   missing_deps = ""
│   extraneous_deps = "libcolamd"
└ @ Main ~/src/julia/test/stdlib_dependencies.jl:169

┌ Warning: Dependency mismatch
│   jll = "MPFR_jll"
│   library = "libmpfr"
│   missing_deps = ""
│   extraneous_deps = "libgmpxx"
└ @ Main ~/src/julia/test/stdlib_dependencies.jl:169
Test Summary:               | Pass  Fail  Total  Time
Stdlib JLL dependency check |   29     5     34  3.9s
RNG of the outermost testset: Random.Xoshiro(0x1329e9815a60ab0e, 0x784c9414423a89d0, 0x1a3aea28fda405f6, 0x8c1dfb6d4dda4f7b, 0xd4081888363c92a9)
ERROR: LoadError: Some tests did not pass: 29 passed, 5 failed, 0 errored, 0 broken.
in expression starting at /home/sabae/src/julia/test/stdlib_dependencies.jl:132

Note that there are some slight differences between macOS and Linux, so you'll have to add some conditionals for those things.

[IanButterworth]

Nice

[vtjnash]

Blocked by --trim failure

That should not actually be blocking. The required code dependencies should be vendored into the test directory anyways, rather than relying on changeable state elsewhere, since otherwise we can lose test coverage simply if these package files change, which should not be true of test code.

[staticfloat]

The required code dependencies should be vendored into the test directory anyways, rather than relying on changeable state elsewhere, since otherwise we can lose test coverage simply if these package files change, which should not be true of test code.

I don't quite follow what you mean, can you be more explicit?

[IanButterworth]

I haven't figured out why, but load time seems to have regressed on this since #58444 (comment) (0.85s)

julia> @time using Plots
  0.969735 seconds (1.35 M allocations: 76.738 MiB, 5.62% gc time, 1.59% compilation time)

Lots of big changes on master since. We need visible key performance metrics like load time on every PR, before merge.