Skip to content

ripgrep fails to build with MUSL on Linux since the nightly release on 2019-03-15 #59411

New issue
New issue
Closed
#59886
Closed
ripgrep fails to build with MUSL on Linux since the nightly release on 2019-03-15#59411
#59886
Assignees
pnkfelix
Labels
A-linkageArea: linking into static, shared libraries and binariesC-bugCategory: This is a bug.P-highHigh priorityT-compilerRelevant to the compiler team, which will review and decide on the PR/issue.regression-from-stable-to-betaPerformance or correctness regression from stable to beta.regression-from-stable-to-nightlyPerformance or correctness regression from stable to nightly.
@BurntSushi

Description

@BurntSushi
BurntSushi
opened on Mar 25, 2019
Member

The last successful build was with the Rust nightly released on 2019-03-14 (bc44841): https://travis-ci.org/BurntSushi/ripgrep/jobs/506631861

The first failing build was with the Rust nightly released on 2019-03-15 (52e8856): https://travis-ci.org/BurntSushi/ripgrep/jobs/507121811

There weren't any changes on my end. (Last commit to ripgrep master was 2019-03-03.)

Here's the pertinent error:

  = note: /usr/bin/ld: elf.o: access beyond end of merged section (3688)
          /usr/bin/ld: elf.o: access beyond end of merged section (2343)
          /usr/bin/ld: elf.o: access beyond end of merged section (2627)
          /usr/bin/ld: elf.o: access beyond end of merged section (3203)
          /usr/bin/ld: elf.o: access beyond end of merged section (2765)
          /usr/bin/ld: elf.o: access beyond end of merged section (1902)
          /usr/bin/ld: elf.o: access beyond end of merged section (3186)
          /usr/bin/ld: elf.o: access beyond end of merged section (3953)
          /usr/bin/ld: elf.o: access beyond end of merged section (2724)
          /usr/bin/ld: elf.o: access beyond end of merged section (3971)
          /usr/bin/ld: elf.o: access beyond end of merged section (3393)
          /usr/bin/ld: elf.o: access beyond end of merged section (3965)
          /usr/bin/ld: elf.o: access beyond end of merged section (2006)
          /usr/bin/ld: elf.o: access beyond end of merged section (3529)
          /usr/bin/ld: elf.o: access beyond end of merged section (2612)
          /usr/bin/ld: elf.o: access beyond end of merged section (2543)
          /usr/bin/ld: elf.o: access beyond end of merged section (3427)
          /usr/bin/ld: elf.o: access beyond end of merged section (2429)
          /usr/bin/ld: elf.o: access beyond end of merged section (3119)
          /usr/bin/ld: elf.o: access beyond end of merged section (2494)
          /usr/bin/ld: elf.o: access beyond end of merged section (2895)
          /usr/bin/ld: elf.o: access beyond end of merged section (3151)
          /usr/bin/ld: elf.o: access beyond end of merged section (3936)
          /usr/bin/ld: elf.o: access beyond end of merged section (3876)
          /usr/bin/ld: elf.o: access beyond end of merged section (2221)
          /usr/bin/ld: elf.o: access beyond end of merged section (4090)
          /usr/bin/ld: elf.o: access beyond end of merged section (3652)
          /usr/bin/ld: elf.o: access beyond end of merged section (3028)
          /usr/bin/ld: elf.o: access beyond end of merged section (2734)
          /usr/bin/ld: elf.o: access beyond end of merged section (3928)
          /usr/bin/ld: elf.o: access beyond end of merged section (2422)
          /usr/bin/ld: elf.o: access beyond end of merged section (2021)
          /usr/bin/ld: elf.o: access beyond end of merged section (4111)
          /usr/bin/ld: elf.o: access beyond end of merged section (1866)
          /usr/bin/ld: elf.o: access beyond end of merged section (1944)
          /usr/bin/ld: elf.o: access beyond end of merged section (2179)
          /usr/bin/ld: elf.o: access beyond end of merged section (2028)
          /usr/bin/ld: elf.o: access beyond end of merged section (3919)
          /usr/bin/ld: elf.o: access beyond end of merged section (2294)
          /usr/bin/ld: elf.o: access beyond end of merged section (2793)
          /usr/bin/ld: elf.o: access beyond end of merged section (2317)
          /usr/bin/ld: elf.o: access beyond end of merged section (2939)
          /usr/bin/ld: elf.o: access beyond end of merged section (2371)
          /usr/bin/ld: elf.o: access beyond end of merged section (2468)
          /usr/bin/ld: elf.o: access beyond end of merged section (4122)
          /usr/bin/ld: elf.o: access beyond end of merged section (3248)
          /usr/bin/ld: elf.o: access beyond end of merged section (4054)
          /usr/bin/ld: elf.o: access beyond end of merged section (3546)
          /usr/bin/ld: elf.o: access beyond end of merged section (1959)
          /usr/bin/ld: elf.o: access beyond end of merged section (3994)
          /usr/bin/ld: elf.o: access beyond end of merged section (1954)
          /usr/bin/ld: elf.o: access beyond end of merged section (4148)
          /usr/bin/ld: elf.o: access beyond end of merged section (4079)
          /usr/bin/ld: elf.o: access beyond end of merged section (3584)
          /usr/bin/ld: elf.o: access beyond end of merged section (2186)
          /usr/bin/ld: elf.o: access beyond end of merged section (2715)
          /usr/bin/ld: elf.o: access beyond end of merged section (3829)
          /usr/bin/ld: elf.o: access beyond end of merged section (3401)
          /usr/bin/ld: elf.o: access beyond end of merged section (3668)
          /usr/bin/ld: elf.o: access beyond end of merged section (3898)
          /usr/bin/ld: elf.o: access beyond end of merged section (1926)
          /usr/bin/ld: elf.o: access beyond end of merged section (2049)
          /usr/bin/ld: elf.o: access beyond end of merged section (2271)
          /usr/bin/ld: elf.o: access beyond end of merged section (3297)
          /usr/bin/ld: elf.o: access beyond end of merged section (3539)
          /usr/bin/ld: elf.o: access beyond end of merged section (3615)
          /usr/bin/ld: elf.o: access beyond end of merged section (3195)
          /usr/bin/ld: elf.o: access beyond end of merged section (2824)
          /usr/bin/ld: elf.o: access beyond end of merged section (1988)
          /usr/bin/ld: elf.o: access beyond end of merged section (2460)
          /usr/bin/ld: elf.o: access beyond end of merged section (3257)
          /usr/bin/ld: elf.o: access beyond end of merged section (3570)
          /usr/bin/ld: elf.o: access beyond end of merged section (2968)
          /usr/bin/ld: elf.o: access beyond end of merged section (2255)
          /usr/bin/ld: elf.o: access beyond end of merged section (2041)
          /usr/bin/ld: elf.o: access beyond end of merged section (2857)
          /usr/bin/ld: elf.o: access beyond end of merged section (4098)
          /usr/bin/ld: elf.o: access beyond end of merged section (1977)
          /usr/bin/ld: elf.o: access beyond end of merged section (4127)
          /usr/bin/ld: elf.o: access beyond end of merged section (3240)
          /usr/bin/ld: elf.o: access beyond end of merged section (2603)
          /usr/bin/ld: elf.o: access beyond end of merged section (2618)
          /usr/bin/ld: elf.o: access beyond end of merged section (3625)
          /usr/bin/ld: elf.o: access beyond end of merged section (3652)
          /usr/bin/ld: elf.o: access beyond end of merged section (4061)
          /usr/bin/ld: elf.o: access beyond end of merged section (3041)
          /usr/bin/ld: elf.o: access beyond end of merged section (3660)
          /usr/bin/ld: elf.o: access beyond end of merged section (2396)
          /usr/bin/ld: elf.o: access beyond end of merged section (3332)
          /usr/bin/ld: elf.o: access beyond end of merged section (2988)
          /usr/bin/ld: elf.o: access beyond end of merged section (1996)
          /usr/bin/ld: elf.o: access beyond end of merged section (2382)
          /usr/bin/ld: elf.o: access beyond end of merged section (3866)
          /usr/bin/ld: elf.o: access beyond end of merged section (2124)
          /usr/bin/ld: elf.o: access beyond end of merged section (3051)
          /usr/bin/ld: elf.o: access beyond end of merged section (3041)
          /usr/bin/ld: elf.o: access beyond end of merged section (3232)
          /usr/bin/ld: elf.o: access beyond end of merged section (2124)
          /usr/bin/ld: elf.o: access beyond end of merged section (2583)
          /usr/bin/ld: elf.o: access beyond end of merged section (4273)
          /usr/bin/ld: elf.o: access beyond end of merged section (3036)
          /usr/bin/ld: elf.o: access beyond end of merged section (2124)
          /usr/bin/ld: elf.o: access beyond end of merged section (4003)
          /usr/bin/ld: elf.o: access beyond end of merged section (3478)
          /usr/bin/ld: elf.o: access beyond end of merged section (3158)
          /usr/bin/ld: elf.o: access beyond end of merged section (3320)
          /usr/bin/ld: elf.o: access beyond end of merged section (4148)
          /usr/bin/ld: elf.o: access beyond end of merged section (2203)
          /usr/bin/ld: elf.o: access beyond end of merged section (3883)
          /usr/bin/ld: elf.o: access beyond end of merged section (3501)
          /usr/bin/ld: elf.o: access beyond end of merged section (3320)
          /usr/bin/ld: elf.o: access beyond end of merged section (4148)
          /usr/bin/ld: elf.o: access beyond end of merged section (2203)
          /usr/bin/ld: elf.o: access beyond end of merged section (4203)
          /usr/bin/ld: elf.o: access beyond end of merged section (1888)
          /usr/bin/ld: elf.o: access beyond end of merged section (2228)
          /usr/bin/ld: elf.o: access beyond end of merged section (2124)
          /usr/bin/ld: elf.o: access beyond end of merged section (2197)
          /usr/bin/ld: elf.o: access beyond end of merged section (4251)
          /usr/bin/ld: elf.o: access beyond end of merged section (4203)
          /usr/bin/ld: elf.o: access beyond end of merged section (2203)
          /usr/bin/ld: elf.o: access beyond end of merged section (3320)
          /usr/bin/ld: elf.o: access beyond end of merged section (4148)
          /usr/bin/ld: elf.o: access beyond end of merged section (2203)
          /usr/bin/ld: elf.o: access beyond end of merged section (3909)
          /usr/bin/ld: elf.o: access beyond end of merged section (3062)
          /usr/bin/ld: elf.o: access beyond end of merged section (2265)
          /usr/bin/ld: elf.o: access beyond end of merged section (3221)
          /usr/bin/ld: elf.o: access beyond end of merged section (2801)
          /usr/bin/ld: elf.o: access beyond end of merged section (2067)
          /usr/bin/ld: elf.o: access beyond end of merged section (3284)
          /usr/bin/ld: elf.o: access beyond end of merged section (3978)
          /usr/bin/ld: elf.o: access beyond end of merged section (3016)
          /usr/bin/ld: elf.o: access beyond end of merged section (2404)
          /usr/bin/ld: elf.o: access beyond end of merged section (4135)
          /usr/bin/ld: elf.o: access beyond end of merged section (2774)
          /usr/bin/ld: elf.o: access beyond end of merged section (2325)
          /usr/bin/ld: elf.o: access beyond end of merged section (2302)
          /usr/bin/ld: elf.o: access beyond end of merged section (3845)
          /usr/bin/ld: elf.o: access beyond end of merged section (4068)
          /usr/bin/ld: elf.o: access beyond end of merged section (2952)
          /usr/bin/ld: elf.o: access beyond end of merged section (4021)
          /usr/bin/ld: elf.o: access beyond end of merged section (2162)
          /usr/bin/ld: elf.o: access beyond end of merged section (3354)
          /usr/bin/ld: elf.o: access beyond end of merged section (2012)
          /usr/bin/ld: elf.o: access beyond end of merged section (3683)
          /usr/bin/ld: elf.o: access beyond end of merged section (2963)
          /usr/bin/ld: elf.o: access beyond end of merged section (2963)
          /usr/bin/ld: elf.o: access beyond end of merged section (2460)
          /usr/bin/ld: elf.o: access beyond end of merged section (3041)
          /usr/bin/ld: elf.o: access beyond end of merged section (3067)
          /usr/bin/ld: elf.o: access beyond end of merged section (3441)
          /usr/bin/ld: elf.o: access beyond end of merged section (2073)
          /usr/bin/ld: elf.o: access beyond end of merged section (3461)
          /usr/bin/ld: elf.o: access beyond end of merged section (2531)
          /usr/bin/ld: /home/travis/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libbacktrace_sys-76fed535951a06b4.rlib(elf.o)(.debug_info+0x1af8): reloc against `.text.elf_add': error 2
          /usr/bin/ld: final link failed: Nonrepresentable section on output
          collect2: error: ld returned 1 exit status

Unfortunately, I can't seem to reproduce this on my local system. One possible difference between my local system (Archlinux) and the Ubuntu CI on Travis is that Ubuntu might be using an older version of musl-gcc via the musl-tools package. I have musl 1.1.21 locally. According to the logs in the link above, CI is using musl 0.9.15 (for both the last successful build and the first failing build).

Activity

jonas-schievink
added
A-linkageArea: linking into static, shared libraries and binaries
I-nominated
regression-from-stable-to-nightlyPerformance or correctness regression from stable to nightly.
T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.
C-bugCategory: This is a bug.
on Mar 25, 2019
svenstaro

svenstaro commented on Mar 26, 2019

@svenstaro
svenstaro
on Mar 26, 2019

Exactly the same problem with exactly the same setup here: https://travis-ci.org/svenstaro/miniserve/jobs/510971585

svenstaro

svenstaro commented on Mar 26, 2019

@svenstaro
svenstaro
on Mar 26, 2019

Works fine with dist: xenial: https://travis-ci.org/svenstaro/miniserve/builds/511554891

jannic

jannic commented on Mar 26, 2019

@jannic
jannic
on Mar 26, 2019
Contributor

I get the same error message for cargo +nightly run --target=x86_64-unknown-linux-musl on a new hello-world project (as generated by cargo new).
This is on debian jessie. (Don't ask. Need to build for old servers...)
Reproducible locally, so if I can help debugging this, please tell me where to look.

jannic

jannic commented on Mar 26, 2019

@jannic
jannic
on Mar 26, 2019
Contributor

It looks like it depends on the version of binutils: With 2.28-5 from debian/stretch I can build the binary successfully, with 2.25-5+deb8u1 from debian/jessie I get the mentioned error.

mati865

mati865 commented on Mar 27, 2019

@mati865
mati865
on Mar 27, 2019 · edited by mati865
Member

I was afraid something like that can happen when tinkering around #58575 but I was able to compile ripgrep on Alpine (native musl host) and Ubuntu (with musl target).

I'll see if I can reproduce it.

sidenote: Ubuntu 14.04 becomes EOL next month.

mati865
mentioned this on Mar 27, 2019
mati865

mati865 commented on Mar 27, 2019

@mati865
mati865
on Mar 27, 2019
Member

I was able to reproduce it in Ubuntu 14.04 container and verified that #59468 fixes it.

BurntSushi

BurntSushi commented on Mar 27, 2019

@BurntSushi
BurntSushi
on Mar 27, 2019
MemberAuthor

@mati865 Thank you!!

mati865

mati865 commented on Mar 27, 2019

@mati865
mati865
on Mar 27, 2019
Member

Sorry for breaking it, when I was testing dozen of different distros I didn't even think of running it on distro with so old binutils.

BurntSushi

BurntSushi commented on Mar 27, 2019

@BurntSushi
BurntSushi
on Mar 27, 2019
MemberAuthor

Yeah unfortunately Ubuntu 14.04 is the default Linux VM on Travis. :-(

23 remaining items

mati865

mati865 commented on Apr 10, 2019

@mati865
mati865
on Apr 10, 2019 · edited by mati865
Member

😕
Ubuntu 14.04 uses binutils 2.24 and 16.04 uses binutils 2.26.

  • Enable compressed debug section feature for x86/x86_64 pe-coff.

Musl is ELF just like glibc, PE/COFF is for MinGW.

alexcrichton

alexcrichton commented on Apr 10, 2019

@alexcrichton
alexcrichton
on Apr 10, 2019
Member

Ah oops I was looking at glibc versions rather than binutils versions. That being said I think it's ironically the same culprit:

Changes in 2.26:

...

* New command-line option for ELF targets to compress DWARF debug
  sections, --compress-debug-sections=[none|zlib|zlib-gnu|zlib-gabi].
mati865

mati865 commented on Apr 10, 2019

@mati865
mati865
on Apr 10, 2019 · edited by mati865
Member

As I said earlier I tried switching musl-toolchain.sh to use binutils 2.25.1 and I'm quite confident it was picked because I've got errors saying -Wa,-mrelax-relocations=no is invaild.
I'll try tomorrow if Rust didn't accidentally pick binutils from system instead of musl-toolchain but it seems unlikely.

You can ping me on Discord BTW.

alexcrichton

alexcrichton commented on Apr 10, 2019

@alexcrichton
alexcrichton
on Apr 10, 2019
Member

@mati865 this isn't an aspect of binutils or such to build the toolchain, it has to do with the binaries we're producing using features that aren't supported by older toolchains. It doesn't matter what we build the musl toolchain with so much as how we build it.

Using nightly-2019-03-15 compiling hello world works, using 2019-03-16 fails. Inspecting gets us:

$ llvm-dwarfdump `rustc +nightly-2019-03-15 --print sysroot`/lib/rustlib/x86_64-unknown-linux-musl/lib/libbacktrace_sys-* > /dev/null
error: /home/alex/.rustup/toolchains/nightly-2019-03-15-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libbacktrace_sys-c3c54ad8d35f7169.rlib(rust.metadata.bin): The end of the file was unexpectedly encountered

$ llvm-dwarfdump /home/alex/.rustup/toolchains/nightly-2019-03-16-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libbacktrace_sys-76fed535951a06b4.rlib > /dev/null
...
error: failed to decompress '.debug_ranges', zlib is not available
error: failed to decompress '.debug_line', zlib is not available
error: failed to decompress '.debug_str', zlib is not available
error: failed to decompress '.debug_info', zlib is not available
error: failed to decompress '.debug_abbrev', zlib is not available
error: failed to decompress '.debug_loc', zlib is not available
error: failed to decompress '.debug_aranges', zlib is not available
error: failed to decompress '.debug_ranges', zlib is not available
error: failed to decompress '.debug_line', zlib is not available
error: failed to decompress '.debug_str', zlib is not available
error: failed to decompress '.debug_info', zlib is not available
error: failed to decompress '.debug_abbrev', zlib is not available
error: failed to decompress '.debug_loc', zlib is not available
error: failed to decompress '.debug_ranges', zlib is not available
error: failed to decompress '.debug_line', zlib is not available
error: failed to decompress '.debug_str', zlib is not available
error: failed to decompress '.debug_info', zlib is not available
error: failed to decompress '.debug_abbrev', zlib is not available
error: failed to decompress '.debug_loc', zlib is not available
error: failed to decompress '.debug_aranges', zlib is not available
error: failed to decompress '.debug_line', zlib is not available
error: failed to decompress '.debug_str', zlib is not available
error: failed to decompress '.debug_info', zlib is not available
error: failed to decompress '.debug_abbrev', zlib is not available
error: failed to decompress '.debug_loc', zlib is not available
error: failed to decompress '.debug_aranges', zlib is not available
error: failed to decompress '.debug_ranges', zlib is not available
error: failed to decompress '.debug_line', zlib is not available
error: failed to decompress '.debug_str', zlib is not available
...

This seems highly likely to be related to compression of debuginfo at least, if not more aspects of the binaries.

mati865

mati865 commented on Apr 10, 2019

@mati865
mati865
on Apr 10, 2019
Member

That looks clearer.

I assumed it would use binutils from musl-toolchain because we point Rust to use GCC from it:

rust/src/ci/docker/dist-x86_64-musl/Dockerfile

Line 44 in 451343e

CC_x86_64_unknown_linux_musl=x86_64-linux-musl-gcc \

I can try to point it to musl-toolchain binutils 2.25.1 or disable compression, whatever you like and open PR tomorrow (or reply here if it doesn't help).

sfackler
added
regression-from-stable-to-betaPerformance or correctness regression from stable to beta.
on Apr 11, 2019
sfackler

sfackler commented on Apr 11, 2019

@sfackler
sfackler
on Apr 11, 2019
Member

This has slipped to beta now I believe.

eldruin
added a commit that references this issue on Apr 11, 2019

Update CI to xenial to work around rust-lang/rust#59411

ae6ee75
mati865
mentioned this on Apr 11, 2019
mati865

mati865 commented on Apr 11, 2019

@mati865
mati865
on Apr 11, 2019
Member

/usr/local/bin/x86_64-linux-musl-gcc calls

/usr/local/bin/../lib/gcc/x86_64-linux-musl/6.3.0/../../../../x86_64-linux-musl/bin/as -v --64 -o /tmp/ccdt44wi.o /tmp/ccKflwB5.s
GNU assembler version 2.27 (x86_64-linux-musl) using BFD version (GNU Binutils) 2.27

so my downgrade to binutils 2.25.1 should have fixed this. Probably Rust's build systems at some point calls gcc to build musl libs.

Anyway disabling debug sections compression fixed the issue:

root@d49f02ca0f13:/foo# cargo build --target x86_64-unknown-linux-musl 
   Compiling foo v0.1.0 (/foo)
    Finished dev [unoptimized + debuginfo] target(s) in 0.25s
root@d49f02ca0f13:/foo# target/x86_64-unknown-linux-musl/debug/foo 
Hello, world!
root@d49f02ca0f13:/foo# cargo build --target x86_64-unknown-linux-musl --release
   Compiling foo v0.1.0 (/foo)
    Finished release [optimized] target(s) in 0.17s

@alexcrichton thank you for help!

bors
closed this as completedin #59886on Apr 12, 2019
javierpedreira
mentioned this on Apr 16, 2019
BurntSushi
mentioned this on Apr 24, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

A-linkageArea: linking into static, shared libraries and binariesC-bugCategory: This is a bug.P-highHigh priorityT-compilerRelevant to the compiler team, which will review and decide on the PR/issue.regression-from-stable-to-betaPerformance or correctness regression from stable to beta.regression-from-stable-to-nightlyPerformance or correctness regression from stable to nightly.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

    Development

    Participants

    @svenstaro@alexcrichton@pnkfelix@jannic@BurntSushi

    Issue actions
      ripgrep fails to build with MUSL on Linux since the nightly release on 2019-03-15 · Issue #59411 · rust-lang/rust