Implement Jang format

[wsantos]

Is your feature request related to a problem? Please describe.
I'd like to run models ising Jang format for speed and quality

Describe the solution you'd like
Being able to run it and use it in opencode is the goal

Describe alternatives you've considered
n/a we can use it in MLX Studio today

Additional context
Check: https://huggingface.co/JANGQ-AI/Nemotron-Cascade-2-30B-A3B-JANG_2L

[czyrichard]

+1. I'd like to share some context from my own experience running MoE models on Apple Silicon.

I've been running Qwen models locally and noticed that standard MLX quantization at 4 bit produces noticeably degraded output. After digging into it, I learned that MLX applies uniform bit-width across all layers, which is fatal for MoE/hybrid architectures. Attention layers are only ~1-12% of parameters but control all of the model's coherence — when they get crushed to the same low bits as expert layers, the model essentially breaks.

JANG_Q solves this elegantly by giving attention layers 5-8 bit while compressing expert layers to 2-4 bit, at only ~0.3 extra bits on average. The quality difference is night and day — models that are unusable at standard MLX 4 bit become perfectly coherent with JANG_Q at similar sizes.

oMLX is hands down the best MLX inference server out there — I'd hate to have to switch to MLX Studio just for JANG_Q support. Having this in oMLX would make it the complete solution for local MoE inference on Mac.

[wsantos]

+1. I'd like to share some context from my own experience running MoE models on Apple Silicon.

I've been running Qwen models locally and noticed that standard MLX quantization at 4 bit produces noticeably degraded output. After digging into it, I learned that MLX applies uniform bit-width across all layers, which is fatal for MoE/hybrid architectures. Attention layers are only ~1-12% of parameters but control all of the model's coherence — when they get crushed to the same low bits as expert layers, the model essentially breaks.

JANG_Q solves this elegantly by giving attention layers 5-8 bit while compressing expert layers to 2-4 bit, at only ~0.3 extra bits on average. The quality difference is night and day — models that are unusable at standard MLX 4 bit become perfectly coherent with JANG_Q at similar sizes.

oMLX is hands down the best MLX inference server out there — I'd hate to have to switch to MLX Studio just for JANG_Q support. Having this in oMLX would make it the complete solution for local MoE inference on Mac.

I can use it on chat, no problem, but on agentic workflow, none beats oMLX right now. Were you able to use MLX Studio with opencode?

[AlexTzk]

Implemented and raised a PR here

[czyrichard]

+1. I'd like to share some context from my own experience running MoE models on Apple Silicon.
I've been running Qwen models locally and noticed that standard MLX quantization at 4 bit produces noticeably degraded output. After digging into it, I learned that MLX applies uniform bit-width across all layers, which is fatal for MoE/hybrid architectures. Attention layers are only ~1-12% of parameters but control all of the model's coherence — when they get crushed to the same low bits as expert layers, the model essentially breaks.
JANG_Q solves this elegantly by giving attention layers 5-8 bit while compressing expert layers to 2-4 bit, at only ~0.3 extra bits on average. The quality difference is night and day — models that are unusable at standard MLX 4 bit become perfectly coherent with JANG_Q at similar sizes.
oMLX is hands down the best MLX inference server out there — I'd hate to have to switch to MLX Studio just for JANG_Q support. Having this in oMLX would make it the complete solution for local MoE inference on Mac.

I can use it on chat, no problem, but on agentic workflow, none beats oMLX right now. Were you able to use MLX Studio with opencode?

I don't use opencode, but I just asked Claude Code to write a Python script that benchmarks models via their OpenAI-compatible API endpoints. I tested the same MMLU benchmark against models served by both MLX Studio and oMLX. Here are the results:

Model	Served by	Accuracy (%)	Correct/Total	Time (min)	Speed (s/q)
Qwen3.5-VL-35B-A3B-JANG_4K-CRACK	MLX Studio	79.12	11110/14042	91.2	0.39
Qwen3.5-35B-A3B-Uncensored (mxfp4)	oMLX	81.77	11482/14042	36.3	0.15
Qwen3.5-35B-A3B-Uncensored (mxfp8)	oMLX	82.45	11578/14042	44.5	0.19

As you can see, MLX Studio took 91 minutes to complete the same benchmark that oMLX finished in 36–45 minutes — roughly 2–3x slower. The speed per question was 0.39s vs 0.15–0.19s. oMLX is significantly faster for API-driven workflows.

[d4mer]

This is a duplicate of https://github.com/jundot/omlx/issues/325

[jundot]

Closing this since it's being tracked in #364.