Skip to content

Tools for dumping information about .rlibs #25820

New issue
New issue
Open
Open
Tools for dumping information about .rlibs#25820
Labels
C-feature-requestCategory: A feature request, i.e: not implemented / a PR.E-hardCall for participation: Hard difficulty. Experience needed to fix: A lot.T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.T-dev-toolsRelevant to the dev-tools subteam, which will review and decide on the PR/issue.
@novocaine

Description

@novocaine
novocaine
opened on May 27, 2015

It would be great if there was a set of tools for dumping information about a given .rlib. This could be used to debug linkage and export issues.

Something like nm and ldd.

I note that nm does appear to partly work on a .rlib, but the output is a bit raw.

Where is the best place to start for understanding the .rlib format as it stands? Is it a good idea to build this on rustc::metadata?

See also http://stackoverflow.com/questions/24603672/rust-library-for-inspecting-rlib-binaries

Activity

lifthrasiir

lifthrasiir commented on May 27, 2015

@lifthrasiir
lifthrasiir
on May 27, 2015
Contributor

cc me

For inspecting metadata this tool might be a good starting point. I have a version of that for post-reform metadata, I'll update the script soon.

novocaine

novocaine commented on May 27, 2015

@novocaine
novocaine
on May 27, 2015
Author

👍

alexcrichton
added
T-tools
on May 27, 2015
alexcrichton

alexcrichton commented on May 27, 2015

@alexcrichton
alexcrichton
on May 27, 2015
Member

All rlibs are actually just static archives (e.g. *.a files) so tools like nm and ar will work on them naturally assuming they ignore the non-object files located within (e.g. metadata and bytecode). As @lifthrasiir mentioned, however, I'd love to have a tool to inspect the metadata in more detail to, for example, dump a module structure, type signatures, etc. The metadata format is a huge black box and is indeed all centralized around rustc::metadata. Note that a tool which does lots and lots with metadata will likely want to be tightly integrated to rustdoc as well as rustdoc already has tons of code to inspect metadata and reconstruct ASTs and/or module hierarchies.

novocaine

novocaine commented on May 27, 2015

@novocaine
novocaine
on May 27, 2015
Author

Would you expect ldd to work?

Otool on os x does not

alexcrichton

alexcrichton commented on May 27, 2015

@alexcrichton
alexcrichton
on May 27, 2015
Member

I think that ldd and otool only work on actual elf files like dynamic libraries and executables, so in that sense no I wouldn't expect them to work.

novocaine

novocaine commented on May 28, 2015

@novocaine
novocaine
on May 28, 2015
Author

(sorry about the 20 questions, but I can't find any resources on this stuff)

How do .rlibs link against 'normal' dynamic libraries provided to rustc via -l ? Is there an ELF image inside the ar somewhere that's dynamically linked against them? Or does rust embed dynamic loader code e.g dlopen?

alexcrichton

alexcrichton commented on May 28, 2015

@alexcrichton
alexcrichton
on May 28, 2015
Member

but I can't find any resources on this stuff

That... may be because none exist :(. I'm more than willing to answer questions though!

How do .rlibs link against 'normal' dynamic libraries provided to rustc via -l ?

No actual linkage occurs, the compiler emits information into the rlib metadata saying what dynamic native libraries should be linked in later, but they're not actually linked until the rlib is linked into a dylib or executable. If the rlib is later used to assemble a staticlib the compiler just prints out what native libraries are needed to link everything.

Is there an ELF image inside the ar somewhere that's dynamically linked against them?

Nah the only ELF file inside rlibs is the object file that the compiler generates (and possibly the object files of native static libraries linked against).

Or does rust embed dynamic loader code e.g dlopen?

No the compiler doesn't embed any code like this in this respect.

novocaine

novocaine commented on May 28, 2015

@novocaine
novocaine
on May 28, 2015
Author

Thanks, Alex :)

novocaine
mentioned this on Jun 14, 2015
alexcrichton
removed
T-tools
on Jan 27, 2016
rillian
mentioned this on Feb 7, 2016
steveklabnik
added
A-tools
on Mar 11, 2016
vvakame
mentioned this on Mar 8, 2017
steveklabnik
added
T-tools
on Mar 24, 2017

6 remaining items

m4b

m4b commented on Jun 30, 2017

@m4b
m4b
on Jun 30, 2017 · edited by m4b
Contributor

I see. Is this something that's still actively desired, or not really? And what did you mean above about tightly integrated with rustdoc? The tool would use it as a lib to pretty print the metadata, or something else?

alexcrichton

alexcrichton commented on Jun 30, 2017

@alexcrichton
alexcrichton
on Jun 30, 2017
Member

With not really much activity for >2 years I wouldn't necessarily classify this as "actively desired". The effort required to stabilize something like this seems quite high, but adding debugging utilities on nightly perhaps may not be so hard.

m4b

m4b commented on Jun 30, 2017

@m4b
m4b
on Jun 30, 2017
Contributor

Ok.

While you're here, do you know offhand what the contents of bytecode.deflate are? I've asked on the channel but no one seems to know, and I can't seem to find it documented anywhere.

alexcrichton

alexcrichton commented on Jun 30, 2017

@alexcrichton
alexcrichton
on Jun 30, 2017
Member

Yes it's just compressed LLVM bytecode. The actual format is a bit wonky with some headers and maybe a footer, you'd have to look around in trans for the exact format.

m4b

m4b commented on Jun 30, 2017

@m4b
m4b
on Jun 30, 2017
Contributor

Cool, thanks so much for your time and answers :)

Let me know if there's something similar to this w.r.t. binary formats, etc., that might need some help/tooling, would be interested in helping.

Mark-Simulacrum
added
C-feature-requestCategory: A feature request, i.e: not implemented / a PR.
T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.
on Jul 22, 2017
ruuda

ruuda commented on Dec 6, 2017

@ruuda
ruuda
on Dec 6, 2017
Contributor

I am trying to obtain the stdlib bitcode, so I can use llvm-link to produce a self-contained bitcode program. Being able to extract it from the rlibs would be nice, because those are bundled with Rust, and I wouldn’t have to compile everything from source. However, I have been unable to decompress the bytecode.deflate file, which makes sense if it contains Rust-specific headers. (It definitely starts with RUST_OBJECT.) Is there a simple way to strip these headers so I can inflate the data?

sekineh
mentioned this on Jul 26, 2018
steveklabnik

steveklabnik commented on Sep 17, 2019

@steveklabnik
steveklabnik
on Sep 17, 2019
Member

Triage; not aware of anyone having implemented this, which, given the fact that all of this is unstable, isn't that surprising. It would be nice to have, but it's unclear that it's a good idea to keep this issue open.

Mart-Bogdan

Mart-Bogdan commented on Mar 5, 2020

@Mart-Bogdan
Mart-Bogdan
on Mar 5, 2020

This os really needed feature, as currently IDEs (Idea and rls/rust-analyzer) have to expand all macros and analyze source code of dependecies.

It would be nice if they would be able to extract info about types and signarures from rlib in sone json format.

AFAIK Currently there are no way to provide autocompletion for proc-macro generated code. Only for regular macros, as IDEs are expanding them by themselves, but it looks like overkill for already compiled libraries.

jeffy1009
mentioned this on Jan 25, 2022
bjorn3

bjorn3 commented on May 15, 2022

@bjorn3
bjorn3
on May 15, 2022
Member

Current rustc has -Zls which dumps a tiny bit of information:

$ rustc -Zls $(rustc --print sysroot)/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-*.rlib
Crate info:
name std-977ece543e1e3d2f
hash 493ec6fd8120daa9 stable_crate_id StableCrateId(15100501329461213700)
proc_macro false
=External Dependencies=
1 core-0e3656b1fda5fd7b hash bf602c611c93048d host_hash None kind Explicit
2 compiler_builtins-16d69221f10b0282 hash 98516d7f9ee4dc1f host_hash None kind Explicit
3 rustc_std_workspace_core-2a6a2797f7a73818 hash 7669287c749f7e23 host_hash None kind Explicit
4 alloc-89782a6344bc3ddf hash 1998205beffe24d7 host_hash None kind Explicit
5 libc-e534461cfee9dab8 hash a1e6883f7e6924a3 host_hash None kind Explicit
6 unwind-cb0068802dd8f031 hash 9e9ab65c1f206b57 host_hash None kind Explicit
7 cfg_if-aa03de290f9594ce hash b325cc00df81012f host_hash None kind Explicit
8 miniz_oxide-1c5c08d77aa4ee1f hash ab361b9594d8cdef host_hash None kind Explicit
9 rustc_std_workspace_alloc-3ad551729ddf5bdf hash d33ff096cbf61200 host_hash None kind Explicit
10 adler-43c97e136c6f66b3 hash 43903e2b4d3800a9 host_hash None kind Explicit
11 hashbrown-626bd4749ba5679b hash 2af9a104d611783a host_hash None kind Explicit
12 std_detect-9961a029e2fb69de hash 48c6d697101a743a host_hash None kind Explicit
13 rustc_demangle-5543e955d2b2e107 hash 3db3e4e8af49176c host_hash None kind Explicit
14 addr2line-2cd7f06709609788 hash 119ed746aeeccd16 host_hash None kind Explicit
15 gimli-05bd833c6cc845b5 hash 835369855f9bdf9b host_hash None kind Explicit
16 object-93491bde8b3642ba hash 59cb584eda23ec10 host_hash None kind Explicit
17 memchr-d338f5690e3fda2f hash 84134cd8ef0ca3e9 host_hash None kind Explicit
18 panic_unwind-1141b5caaf7b2195 hash ec463f279262c141 host_hash None kind Implicit

I also wrote https://github.com/bjorn3/rust_read_rlib at some point, but it would need to be updated to latest rustc.

SamB

SamB commented on Sep 15, 2024

@SamB
SamB
on Sep 15, 2024

It seriously vexes me that things like this can't be allowed on stable because somebody would inevitably come to rely on the exact format of the output. :-(

Specifically, the part where we can't inspect the rust-specific metadata.

The fact that such functionality would be nightly-only probably isn't helping to motivate anyone to implement it.

jieyouxu
added
E-hardCall for participation: Hard difficulty. Experience needed to fix: A lot.
on Mar 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    C-feature-requestCategory: A feature request, i.e: not implemented / a PR.E-hardCall for participation: Hard difficulty. Experience needed to fix: A lot.T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.T-dev-toolsRelevant to the dev-tools subteam, which will review and decide on the PR/issue.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @SamB@steveklabnik@novocaine@alexcrichton@lifthrasiir

        Issue actions
          Tools for dumping information about .rlibs · Issue #25820 · rust-lang/rust