Skip to content

Conversation

@mayeut
Copy link
Contributor

@mayeut mayeut commented Aug 24, 2025

This allows a speed-up of all QEMU targets by using a statically built clang that runs with the native architecture of the runner rather than calling gcc through QEMU.

CI should finish in roughly 6 minutes compared to 24 minutes in master.

This PR is mostly used as a PoC for this trick and see wether it would see adoption, what are the limitations and how it could be improved.

Clang used in this PR is built & published in the following repo: https://github.com/mayeut/static-clang-images

@jcfr
Copy link
Contributor

jcfr commented Aug 24, 2025

CI should finish in roughly 6 minutes compared to 24 minutes in master.

This is an outstanding improvement 🎉

@henryiii
Copy link
Contributor

henryiii commented Nov 10, 2025

What's the status of this, is it safe to merge? I see "PoC" in the description, but it seems to be working? (Same with cmake)

@mara004
Copy link

mara004 commented Nov 10, 2025

That's very interesting indeed!
I once thought, couldn't we have a compiler that escapes the container's emulation and runs on the host arch, while being pre-configured for cross-compilation to the target arch?, but put it aside as a wild dream.
But if I understood this correctly, this is actually possible, and shown here?
I'd be really intrigued if that approach could be used more widely across the Python ecosystem, or if there are any drawbacks that make adoption difficult?

@mayeut
Copy link
Contributor Author

mayeut commented Nov 10, 2025

What's the status of this, is it safe to merge?

yes I think it is since we run the ninja test suite and, since the PR was first introduced, the manylinux images have moved to using the same trick using clang meaning I'm committed to keep clang up-to-date.

I see the PoC in the description, but it seems to be working?

It's a working proof-of-concept from the Python ecosystem point of view.
I'm almost sure using clang is not an issue at all when building executables and I don't think it would be an issue with Python extensions (although not tested).
If it's acceptable as-is by other maintainers, I'd like to reference this PR (and probably the one in CMake as well) as an example for this trick.
If it proves useful enough then the install-static-clang.sh script will most likely end-up in manylinux itself.

I'd be really intrigued if that approach could be used more widely across the Python ecosystem, or if there are any drawbacks that make adoption difficult?

There are definitely some differences, I mostly encountered some issues with the linker either here, in scikit-build/cmake-python-distributions#632 or manylinux. Those were easy enough to fix in any case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants