Skip to content

Conversation

pytorchbot
Copy link
Collaborator

Note: This is an attempt to cherry-pick Mergen's RPATH fix from #13254 onto main. Fixes #14421. Original description below.

Problem: The _portable_lib.so Python extension built on CI couldn't find PyTorch libraries when installed locally because it had hardcoded absolute paths from
the CI build environment.

Error:
ImportError: dlopen(.../_portable_lib.cpython-311-darwin.so, 0x0002): Library not loaded: @rpath/libtorch_python.dylib
Referenced from:
.../executorch/extension/pybindings/_portable_lib.cpython-311-darwin.so Reason: tried:
'/Users/runner/work/_temp/.../torch/lib/libtorch_python.dylib' (no such file)

Root Cause: The CMake build was linking to PyTorch libraries using absolute paths from the build environment, without setting proper relative RPATHs for
runtime library resolution.

Solution: Added platform-specific relative RPATH settings to the portable_lib target in /Users/mnachin/executorch/CMakeLists.txt (lines 657-669):

  • macOS: Uses @loader_path/../../../torch/lib to find PyTorch libraries relative to the .so file location
  • Linux: Uses $ORIGIN/../../../torch/lib for the same purpose
  • Sets both BUILD_RPATH and INSTALL_RPATH to ensure consistency

Impact: This allows the wheel-packaged _portable_lib.so to find PyTorch libraries regardless of the installation location, fixing the runtime linking issue
when using ExecutorTorch wheels built on CI.

Note: The same fix may be needed for _training_lib if it experiences similar issues.

Test Plan:

  # Build the wheel locally
  python setup.py bdist_wheel

  # create fresh conda env
  conda create -yn executorch_test_11 python=3.11.0 && conda activate executorch_test_11

  # install
  pip install ./dist/executorch-*.whl

  # Verify
  python -c "from executorch.extension.pybindings._portable_lib import _load_for_executorch; print('Success!')"

Copy link

pytorch-bot bot commented Sep 19, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14442

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 1 Cancelled Job

As of commit c6f46e3 with merge base e0dda90 (image):

NEW FAILURE - The following job has failed:

CANCELLED JOB - The following job was cancelled. Please retry:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 19, 2025
**Note: This is an attempt to cherry-pick Mergen's RPATH fix from #13254
onto main. Fixes #14421.
Original description below.**

Problem: The _portable_lib.so Python extension built on CI couldn't find
PyTorch libraries when installed locally because it had hardcoded
absolute paths from
the CI build environment.

Error:
ImportError: dlopen(.../_portable_lib.cpython-311-darwin.so, 0x0002):
Library not loaded: @rpath/libtorch_python.dylib
Referenced from:
.../executorch/extension/pybindings/_portable_lib.cpython-311-darwin.so
Reason: tried:
'/Users/runner/work/_temp/.../torch/lib/libtorch_python.dylib' (no such
file)

Root Cause: The CMake build was linking to PyTorch libraries using
absolute paths from the build environment, without setting proper
relative RPATHs for
runtime library resolution.

Solution: Added platform-specific relative RPATH settings to the
portable_lib target in /Users/mnachin/executorch/CMakeLists.txt (lines
657-669):

- macOS: Uses @loader_path/../../../torch/lib to find PyTorch libraries
relative to the .so file location
- Linux: Uses $ORIGIN/../../../torch/lib for the same purpose
- Sets both BUILD_RPATH and INSTALL_RPATH to ensure consistency

Impact: This allows the wheel-packaged _portable_lib.so to find PyTorch
libraries regardless of the installation location, fixing the runtime
linking issue
when using ExecutorTorch wheels built on CI.

Note: The same fix may be needed for _training_lib if it experiences
similar issues.

Test Plan:

```
  # Build the wheel locally
  python setup.py bdist_wheel

  # create fresh conda env
  conda create -yn executorch_test_11 python=3.11.0 && conda activate executorch_test_11

  # install
  pip install ./dist/executorch-*.whl

  # Verify
  python -c "from executorch.extension.pybindings._portable_lib import _load_for_executorch; print('Success!')"
```

Co-authored-by: Mergen Nachin <[email protected]>
(cherry picked from commit 641e737)
@abhinaykukkadapu abhinaykukkadapu force-pushed the cherry-pick-14422-by-pytorch_bot_bot_ branch from d862f88 to c6f46e3 Compare September 23, 2025 00:52
@abhinaykukkadapu abhinaykukkadapu merged commit 9c172fe into release/1.0 Sep 23, 2025
291 of 293 checks passed
@abhinaykukkadapu abhinaykukkadapu deleted the cherry-pick-14422-by-pytorch_bot_bot_ branch September 23, 2025 16:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants