Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add C++ build support for use with LibTorch #819

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

shaltielshmid
Copy link

@shaltielshmid shaltielshmid commented Feb 9, 2024

1] Added a CMakeLists.txt file equivalent to setup.py for compiling in C++
2] Added a header file with export, so the linker in Visual C++ (on windows) will work.
3] Added a preprocessor directive when building in Python so the python headers and binding occur (I added it for the Python build since that requires adding the directive just in the setup.py, whereas for C++ would need to be added in every project that references the project).

I've tested out using the C++ compiled library both both for window & linux with CUDA 12.1 and LibTorch 2.1.0, and it works great! I also managed to build the python library with the adjusted code for windows.

Happy to make any changes, as you see fit.

If this PR is accepted, I'd be happy to add similar changes for the other sub-packages here (LayerNorm, FusedDense, etc.) .

Instructions for building it for C++:

mkdir build
cd build
cmake .. -DLIBTORCH_PATH=/path/to/libtorch-shared-with-deps/libtorch
cmake --build . --config Release # or make -j N

@shaltielshmid shaltielshmid marked this pull request as draft February 9, 2024 10:52
@shaltielshmid shaltielshmid marked this pull request as ready for review February 10, 2024 16:46
@shaltielshmid
Copy link
Author

@tridao Using the built C++ binaries I added a wrapper for TorchSharp for all the TorchSharp users to have access to your brilliant library! You can view it here: https://github.com/shaltielshmid/TorchSharp.FlashAttention

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant