MLX Roadmap?

[vade]

Hello

Since the announcement of MLX I've been excited and watching on the sidelines. I'm curious if there has been any public discussion of a roadmap for MLX?

I'm aware of some general efforts that seem quite useful

• Swift compatibility
• Quantization (recently merged)
• Expanding out supported layers and model types

All of this is awesome, but im left wondering a few things.

• Who is MLX directly aimed at? IE:

  • For researchers implementing new approaches within python wanting to take advantage of Apple Silicon?
  • For developers looking take advantage of Apple silicon within applications or services using python or other languages?

• What language is the focus for MLX?

  • Given that MLX like most Pythonic libraries actually implements the most important code in C++, there is a C++ backend.
  • That backend however does not have a clearly documented API, nor are 'useful' abstractions easily composed with it.
  • Swift compatibility is being worked on, but to what end? Is Swift meant to be more like the C++ layer, where developers lose out on helpful higher level abstractions and composability, or is the goal of Swift to match some of Pythons higher level MLX Api?

I ask these questions from the perspective of an Application Developer building native apps for macOS and iOS.

MLX seems very promising, but its unclear to me if I should invest my time learning MLX python and C++ API without understanding the overall goal of MLX writ large.

Thanks! I know this is early days and abstract, but I think having a public roadmap helps in a lot of ways. It can help guide contributions, help frame questions and answers and provide framing for how to accept PRs.

Thank you again

[dougdew64]

The MLX README offers this:

MLX is designed by machine learning researchers for machine learning researchers. The framework is intended to be user-friendly, but still efficient to train and deploy models. The design of the framework itself is also conceptually simple. We intend to make it easy for researchers to extend and improve MLX with the goal of quickly exploring new ideas.

From that statement, I could imagine that having only the higher level python API would be sufficient to meet the needs of the target audience of MLX. In other words, it's not clear that either a Swift API or a higher level C++ API are necessary to serve the needs of machine learning researchers.

After thinking about all of this for a bit, it seems entirely reasonable that machine learning researchers would be the only users of MLX. The machine learning researchers would use MLX and the python API to train and deploy models. App developers would then consume those models by using Swift and the CoreML / MPS stack.

Rather than implement higher level C++ MLX APIs or Swift MLX APIs, perhaps a better use of time and effort is to ensure that MLX can export models for convenient consumption in the CoreML / MPS stack.

Rather than compete with the CoreML / MPS stack, MLX would complement the CoreML / MPS stack.

[dougdew64]

Possibilities like that are the primary reason why I am so enthusiastic about this Apple-platform-specific MLX repo.

[vade]

https://developer.apple.com/library/archive/documentation/IDEs/Conceptual/CustomClassDisplay_in_QuickLook/CH01-quick_look_for_custom_objects/CH01-quick_look_for_custom_objects.html

[dougdew64]

I am ready for a new kind of book. One that integrates with a debugger.

[vade]

Can we keep this somewhat on topic please?

[dougdew64]

Thank you very much for sharing that url @vade !!!