Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce binary size #1952

Merged
merged 1 commit into from
Mar 11, 2025
Merged

Reduce binary size #1952

merged 1 commit into from
Mar 11, 2025

Conversation

awni
Copy link
Member

@awni awni commented Mar 10, 2025

Reduce the binary size by moving the kernel dispatches to an outer level and adding back unsafe_weak_copy:

Binary size built with:

cmake .. \                                
  -DCMAKE_BUILD_TYPE=MinSizeRel \
  -DBUILD_SHARED_LIBS=ON \
  -DMLX_BUILD_CPU=ON \
  -DMLX_BUILD_SAFETENSORS=ON \
  -DMLX_BUILD_GGUF=OFF \
  -DMLX_METAL_JIT=ON

Pre: 35MB
Post: 11MB (potentially shrink more by updating a few more of the Primitives with the new pattern)

Benchmarks

These are some pretty lightweight CPU benchmarks to try and get at overhead. For anything larger the time differences aren't visible.

Bench Pre Post
MNIST 0.325 (s) 0.324 (s)
MNiST 10 0.525 (s) 0.543 (s)
No Compile MNIST 0.366 (s) 0.366 (s)
No Compile MNIST 10 0.750 (s) 0.818 (s)
Linear Regression 4027.99455 (it/s) 4659.03894 (it/s)
Logistic Regression 4056.80979 (it/s) 4079.13215 (it/s)

@awni awni marked this pull request as draft March 10, 2025 21:26
@awni awni force-pushed the reduce_binary_size branch from 1388a6f to 45f2582 Compare March 10, 2025 23:50
@awni awni force-pushed the reduce_binary_size branch from 45f2582 to c2d2c43 Compare March 11, 2025 02:37
@awni awni marked this pull request as ready for review March 11, 2025 02:49
@awni awni requested review from barronalex and angeloskath March 11, 2025 02:51
Copy link
Member

@angeloskath angeloskath left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@awni awni merged commit 736a340 into main Mar 11, 2025
5 checks passed
@awni awni deleted the reduce_binary_size branch March 11, 2025 13:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants