Skip to content

Commit 9576a06

Browse files
authored
sysroot: fix missing symlinks for blas and lapack [IO-60] (#25)
To support cross-compiling from x86_64 to aarch64 debian targets - we're using some scripts from the chromium project for building sysroots directly from debian (`.deb`) packages. A caveat of using these scripts is they rely on very low level plumbing commands (i.e. `dpkg-deb`) to handle unpacking and installing the package contents (libs and headers), into "jail" (the sysroot). These commands don't handle many details of the full install process. In the case of blas and lapack - they don't handle the post install step of creating symlinks at `/usr/lib/${TRIPLE}/libblas.so`. This is important because this location is on the linkers default search path. The reason these symlinks are not part of the package by default is because blas and lapack are virtual packages, there are several different "concrete" packages that can satisfy the dependency (i.e. `libblas-dev`, `libopenblas-dev`, etc..). It's up the the debian alternatives system (& system administrators) to handle the detail of which concrete library the `libblas` and `liblapack` symlinks actually point to. I explored a couple of routes to try to do this in a less janky way then manually creating the links, but this ended up being a rabbit hole. Ideally we revisit the way we're building these sysroots in a more robust way. In the meantime we can now remove the hardcoded linker paths in libraries that depend on `blas` and `lapack`: https://github.com/swift-nav/rules_swiftnav/blob/d10814e9b8f3d3b948e3ea9c1aed9c80c2fd74cc/third_party/suitesparse.BUILD#L25. Trying to handle all the different conditions (os, arch, sysroot or no sysroot, etc..) was becoming untenable, and just didn't work at all in some cases. ## Testing I've tested the resulting sysroot build locally to verify that this mechanism of creating the links works.
1 parent 9e03c6b commit 9576a06

File tree

1 file changed

+24
-40
lines changed

1 file changed

+24
-40
lines changed

sysroot/sysroot-creator.sh

Lines changed: 24 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -361,45 +361,29 @@ StripChecksumsFromPackageList() {
361361
HacksAndPatches() {
362362
Banner "Misc Hacks & Patches"
363363

364-
# Remove an unnecessary dependency on qtchooser.
365-
# rm "${INSTALL_ROOT}/usr/lib/${TRIPLE}/qt-default/qtchooser/default.conf"
366-
367-
# libxcomposite1 is missing a symbols file.
368-
# cp "${SCRIPT_DIR}/libxcomposite1-symbols" \
369-
# "${INSTALL_ROOT}/debian/libxcomposite1/DEBIAN/symbols"
370-
371-
# __GLIBC_MINOR__ is used as a feature test macro. Replace it with the
372-
# earliest supported version of glibc (2.26, obtained from the oldest glibc
373-
# version in //chrome/installer/linux/debian/dist_packag_versions.json and
374-
# //chrome/installer/linux/rpm/dist_package_provides.json).
375-
local usr_include="${INSTALL_ROOT}/usr/include"
376-
# local features_h="${usr_include}/features.h"
377-
# sed -i 's|\(#define\s\+__GLIBC_MINOR__\)|\1 26 //|' "${features_h}"
378-
379-
# fcntl64() was introduced in glibc 2.28. Make sure to use fcntl() instead.
380-
# local fcntl_h="${INSTALL_ROOT}/usr/include/fcntl.h"
381-
# sed -i '{N; s/#ifndef __USE_FILE_OFFSET64\(\nextern int fcntl\)/#if 1\1/}' \
382-
# "${fcntl_h}"
383-
384-
# Do not use pthread_cond_clockwait as it was introduced in glibc 2.30.
385-
local cppconfig_h="${usr_include}/${TRIPLE}/c++/10/bits/c++config.h"
386-
sed -i 's|\(#define\s\+_GLIBCXX_USE_PTHREAD_COND_CLOCKWAIT\)|// \1|' \
387-
"${cppconfig_h}"
388-
389-
# This is for chrome's ./build/linux/pkg-config-wrapper
390-
# which overwrites PKG_CONFIG_LIBDIR internally
391-
SubBanner "Move pkgconfig scripts"
392-
mkdir -p ${INSTALL_ROOT}/usr/lib/pkgconfig
393-
mv ${INSTALL_ROOT}/usr/lib/${TRIPLE}/pkgconfig/* \
394-
${INSTALL_ROOT}/usr/lib/pkgconfig
395-
396-
# Avoid requiring unsupported glibc versions.
397-
"${SCRIPT_DIR}/reversion_glibc.py" \
398-
"${INSTALL_ROOT}/lib/${TRIPLE}/libc.so.6"
399-
"${SCRIPT_DIR}/reversion_glibc.py" \
400-
"${INSTALL_ROOT}/lib/${TRIPLE}/libm.so.6"
401-
# "${SCRIPT_DIR}/reversion_glibc.py" \
402-
# "${INSTALL_ROOT}/lib/${TRIPLE}/libcrypt.so.1"
364+
# In debian blas and lapack are virtual packages.
365+
#
366+
# As such - the alternatives system is responsible for ensuring that a
367+
# libblas.so and liblapack.so are available in /usr/lib/${TRIPLE} -
368+
# i.e. - they are in the linkers default search path.
369+
#
370+
# The implementation we're using here - libblas-dev, and liblapack-dev
371+
# only install the libraries to /usr/lib/${TRIPLE}/blas and /usr/lib/${TRIPLE}/lapack
372+
# which are not on the linkers default search paths.
373+
#
374+
# Typically the symlink creation is handled by their respective post install
375+
# scripts - but since we're using dpkg-deb directly - we need to do it ourselves.
376+
#
377+
# Otherwise we have to hardcode the install locations in the build system.
378+
# Dealing with the possible combinations of arch, os, whether we're building
379+
# with a sysroot or not, etc... is too much to keep straight. So we'll deal
380+
# with it here.
381+
cd ${INSTALL_ROOT}/usr/lib/${TRIPLE}
382+
ln -s ./blas/libblas.so libblas.so
383+
ln -s ./blas/libblas.a libblas.a
384+
ln -s ./lapack/liblapack.so liblapack.so
385+
ln -s ./lapack/liblapack.a liblapack.a
386+
cd -
403387
}
404388

405389
InstallIntoSysroot() {
@@ -528,7 +512,7 @@ BuildSysroot() {
528512
local files_and_sha256sums="$(cat ${package_file})"
529513
StripChecksumsFromPackageList "$package_file"
530514
InstallIntoSysroot ${files_and_sha256sums}
531-
# HacksAndPatches
515+
HacksAndPatches
532516
# CleanupJailSymlinks
533517
# VerifyLibraryDeps
534518
CreateTarBall

0 commit comments

Comments
 (0)