Skip to content

Latest master toolchains are broken when linking LLVM: "__morestack: invalid needed version 2" #81554

Closed
@RalfJung

Description

@RalfJung
Member

When I install the latest master toolchains via rustup-toolchain-install-master and try to build Miri with them, linking fails:

error: linking with `cc` failed: exit code: 1
  |
  [...]
  = note: /usr/bin/ld: /home/r/.rustup/toolchains/miri/lib/rustlib/x86_64-unknown-linux-gnu/lib/libLLVM-11-rust-1.51.0-nightly.so: __morestack: invalid needed version 2
          /usr/bin/ld: /home/r/.rustup/toolchains/miri/lib/rustlib/x86_64-unknown-linux-gnu/lib/libLLVM-11-rust-1.51.0-nightly.so: error adding symbols: bad value
          collect2: error: ld returned 1 exit status

I did some bisection, and it looks like the problem started happening with #81489.

Cc @rust-lang/infra

Activity

RalfJung

RalfJung commented on Jan 30, 2021

@RalfJung
MemberAuthor

The error is actually already mentioned in #81489. However, I am building Miri with cargo, so I don't know how I'd "use LLD instead".

changed the title [-]Latest master toolchains are broken[/-] [+]Latest master toolchains are broken when linking LLVM: "__morestack: invalid needed version 2"[/+] on Jan 30, 2021
Mark-Simulacrum

Mark-Simulacrum commented on Jan 30, 2021

@Mark-Simulacrum
Member

Hm, we probably need to revert that PR. I did not realize we'd need lld downstream too, I thought that was only on our builder.

cc @nikic @cuviper

added
A-LLVMArea: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues.
P-highHigh priority
T-infraRelevant to the infrastructure team, which will review and decide on the PR/issue.
on Jan 30, 2021
cuviper

cuviper commented on Jan 30, 2021

@cuviper
Member

Yes, a revert seems appropriate. We're going to have to figure out where that's actually coming from. The only change I saw between 10 and 11 by grepping morestack was the removal of llgo, but maybe there's some bad remnant.

nikic

nikic commented on Jan 30, 2021

@nikic
Contributor

Yeah, I thought this issue would be limited to the rustc build, but it makes sense that it can also affect project linking against rustc shared objects.

I've looked a bit into what the root cause is, and what I have so far as that the symbol comes from https://github.com/llvm/llvm-project/blob/b10d445307a0f3c7e5522836b4331090aacaf349/llvm/lib/ExecutionEngine/RuntimeDyld/RTDyldMemoryManager.cpp#L217 and is present as a weak symbol both when compiling with clang-10 and clang-11. The only difference I see from objdump -T is this:

; clang-10
0000000000000000  w   D  *UND*  0000000000000000  LLVM_11     __morestack
; clang-11
0000000000000000  w   D  *UND*  0000000000000000  Base        __morestack

So apparently __morestack was treated as a symbol of the LLVM_11 library, while now it's treated as a symbol of Base -- I'm not sure what that actually means, but presumably that refers to the compiler runtime libraries. My guess is that previously it just ended up being nulled out as a missing weak symbol, but now it's actually being found and linking fails because ... reasons. I'm not good at linkers :)

Edit: readelf -Ws shows these as:

; clang-10
   315: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND __morestack
; clang-11
   315: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND __morestack@@<corrupt>
RalfJung

RalfJung commented on Jan 30, 2021

@RalfJung
MemberAuthor

@nikic can you prepare a revert PR?

nikic

nikic commented on Jan 30, 2021

@nikic
Contributor

@RalfJung Done in #81565.

If nobody has an idea on how to avoid the linker error, two more options would be to comment out the symbol in our llvm-project fork, or to avoid upgrading clang altogether and just build both Python 2 and Python 3 for that image.

nagisa

nagisa commented on Jan 30, 2021

@nagisa
Member

The issue is probably going to be in how exactly the objects that comprise libLLVM.so are compiled, not linked. I suspect lld just ends up ignoring the malformed object or otherwise not reporting the issue, but the object and the relevant symbols ultimately end up in the so anyway.

Alas, reasons these obscure failures can occur are many – I've seen them occur from things as trivial as forgetting to -fPIC.

The question is: is there perhaps a way to disable ExecutionEngine entirely? Intuitively nothing we do should depend on that code. And if not, I'm comfortable patching out __morestack in our fork as we get closer to Feb 11 and continue having trouble pinpointing exact cause of this problem.

added a commit that references this issue on Jan 30, 2021

Auto merge of rust-lang#81565 - nikic:revert, r=nagisa

04caa63
nikic

nikic commented on Jan 30, 2021

@nikic
Contributor

The question is: is there perhaps a way to disable ExecutionEngine entirely? Intuitively nothing we do should depend on that code.

We normally don't link ExecutionEngine, but dist-x86_64-linux in particular builds LLVM with ThinLTO, in which case we link libLLVM.so as shared, instead of statically linking individual components into rustc_llvm.

12 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-LLVMArea: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues.P-highHigh priorityT-infraRelevant to the infrastructure team, which will review and decide on the PR/issue.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      Participants

      @cuviper@nikic@RalfJung@matthiaskrgr@nagisa

      Issue actions

        Latest master toolchains are broken when linking LLVM: "__morestack: invalid needed version 2" · Issue #81554 · rust-lang/rust