[SR-2100] Add support for multi-lib systems

[swift-ci]

Previous ID	SR-2100
Radar	rdar://problem/28109107
Original Reporter	jdfergason (JIRA User)
Type	New Feature

Attachment: Download

Additional Detail from JIRA

Votes	2
Component/s	Compiler, Foundation, LLDB for Swift, Package Manager, Source Tooling, Standard Library, XCTest
Labels	New Feature, Linux
Assignee	None
Priority	Medium

md5: 8786cef65def4eeae0eea5d2859d7870

Issue Description:

On some linux distributions (i.e., the Redhat / CentOS / Fedora family of distros) there is a requirement to use /usr/lib64 instead of /usr/lib on 64-bit machines. The "lib" directory is hard-coded throughout the source code.

I think I have all the places tracked down and have a patch that simply hard-codes it to lib64. It would be nice to make this more flexible and allow the libdir to be specified to build-script.

[modocache]

Good catch, thanks. I'd be happy to accept a patch to swift-corelibs-xctest, if it contains any hardcoded paths to /usr/lib. I imagine it would be challenging to modify this across the entire set of Swift repositories – do you have a plan here?

[swift-ci]

Comment by Jeremy Fergason (JIRA)

I have a patch that modifies it for the fedora RPMs I am building but I don't think it's in a fit state for the repos. It's one of those things that needs to be done all at once. There were no changes necessary to XCTest except for the location of the Std Lib, Foundation and the module path.

[ddunbar]

Could you show your patch in its current form, just so we have an idea of the places that need changing?

[swift-ci]

Comment by Felipe Bugno (JIRA)

I did successfully created a patch for that on Slackware 14.2 multilib. Attached you guys have it.

It was designed to be applied on top of 4.0-RELEASE, and as such, requires that below to test:

https://github.com/apple/swift/archive/swift-4.0-RELEASE/swift-4.0-RELEASE.tar.gz
https://github.com/apple/swift-corelibs-foundation/archive/swift-4.0-RELEASE/swift-corelibs-foundation-4.0-RELEASE.tar.gz
https://github.com/apple/swift-corelibs-libdispatch/archive/swift-4.0-RELEASE/swift-corelibs-libdispatch-4.0-RELEASE.tar.gz
https://github.com/apple/swift-corelibs-xctest/archive/swift-4.0-RELEASE/swift-corelibs-xctest-4.0-RELEASE.tar.gz
https://github.com/apple/swift-llvm/archive/swift-4.0-RELEASE/swift-llvm-4.0-RELEASE.tar.gz
https://github.com/apple/swift-clang/archive/swift-4.0-RELEASE/swift-clang-4.0-RELEASE.tar.gz
https://github.com/apple/swift-lldb/archive/swift-4.0-RELEASE/swift-lldb-4.0-RELEASE.tar.gz
https://github.com/apple/swift-cmark/archive/swift-4.0-RELEASE/swift-cmark-4.0-RELEASE.tar.gz
https://github.com/apple/swift-package-manager/archive/swift-4.0-RELEASE/swift-package-manager-4.0-RELEASE.tar.gz
https://github.com/apple/swift-llbuild/archive/swift-4.0-RELEASE/swift-llbuild-4.0-RELEASE.tar.gz
https://github.com/apple/swift-compiler-rt/archive/swift-4.0-RELEASE/swift-compiler-rt-4.0-RELEASE.tar.gz
https://github.com/apple/swift-integration-tests/archive/swift-4.0-RELEASE/swift-integration-tests-4.0-RELEASE.tar.gz

After decompressing everything inside some folder, and moving the patch inside that folder, just rename everything to the default Swift build structure:

mv swift-clang-swift-4.0-RELEASE clang
mv swift-cmark-swift-4.0-RELEASE cmark
mv swift-corelibs-foundation-swift-4.0-RELEASE swift-corelibs-foundation
mv swift-corelibs-libdispatch-swift-4.0-RELEASE swift-corelibs-libdispatch
mv swift-corelibs-xctest-swift-4.0-RELEASE swift-corelibs-xctest
mv swift-llbuild-swift-4.0-RELEASE llbuild
mv swift-lldb-swift-4.0-RELEASE lldb
mv swift-llvm-swift-4.0-RELEASE llvm
mv swift-package-manager-swift-4.0-RELEASE swiftpm
mv swift-compiler-rt-swift-4.0-RELEASE compiler-rt
mv swift-swift-4.0-RELEASE swift
mv swift-integration-tests-swift-4.0-RELEASE swift-integration-tests

And then, apply the patch with:

patch -p1 < ./multilib.patch

That's it.

Now... a observation:

Under FHS 3.0 standard ( http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.pdf ), on lib64 all 64bits must go. But, Ubuntu/Debian do not follow the standard, instead using a lib32 and lib structure.

To solve that, normally experienced Linux package maintainers define a LIBDIR variable on the build system to be the result of the following command captured: "gcc -print-multi-os-directory".

So, on my Slackware system, i have

felipe@CapEnt$ gcc -print-multi-os-directory -m32
../lib
felipe@CapEnt$ gcc -print-multi-os-directory -m64
../lib64

And on my Ubuntu 17.10 system, i have

felipe@KOS-MOS$ gcc -print-multi-os-directory -m32
../lib32
felipe@KOS-MOS$ gcc -print-multi-os-directory -m64
../lib

My patch is quite brittle, and don't do that. As such, it breaks on Ubuntu/Debian systems. We need to implement on top of that patch a proper detection like that above. Doing that, Swift can become pretty much "Linux agnostic", and as such, could be compiled and packaged to any distribution.

For curiosity, that below is the build command that i use:

utils/build-script -R -l -b -p --xctest --foundation -i --tvos 1 --watchos 1 \
--no-swift-stdlib-assertions --build-swift-static-stdlib \
--build-swift-static-sdk-overlay --build-swift-stdlib-unittest-extra --skip-test-lldb \
--install-destdir=$PKG --install-prefix=usr --install-swift \
--install-lldb --install-foundation --install-llbuild --install-swiftpm --install-xctest --install-libdispatch \
--swift-install-components='autolink-driver;compiler;clang-builtin-headers;stdlib;swift-remote-mirror;sdk-overlay;license;sourcekit-inproc' \
--reconfigure

Tks!