feat: add LiquidAI LFM2.x parser support

[necaris]

Implement LfmToolParser for the Liquid.AI models.

Why is this needed?

Fixes #85, adding support for a popular model series.

AI assistance disclosure

Initial implementation generated by Claude, review and additional tests from Codex, and manual touch-ups and refactorings in concert with Gemini.

By submitting this PR I confirm I can explain the intent, risk, and behavior of every non-generated change in this PR. For any generated / boilerplate / scaffolded sections, I've identified them above and can describe how I verified them.

Test plan

• new tests/test_lfm_tool_parser.py
• updated tests/test_native_tool_format.py to mark LfmToolParser as non-native
• updated tests/test_tool_call_streaming_parity.py to add a streaming-parity fixture for the LFM parser

Checklist

• Tests pass locally (python3 -m pytest tests/ -x)
• Lint passes (ruff check && ruff format --check)
• Self-validated with python3 -m scripts.pr_validate.pr_validate <PR#> — see CONTRIBUTING.md (opt out heavy steps with PR_VALIDATE_NO_DEEPSEEK=1 PR_VALIDATE_NO_STRESS=1 if you don't have the hardware/keys)
• If new tests touch a critical code path (parser / scheduler / security), I've spot-checked that they fail when the corresponding production line is broken (see SOP §Step 3)
• Updated README/docs if applicable
• No breaking changes to existing API

NOTE: Leaving in draft state till benchmarks can be run.

[raullenchai]

Thanks for tackling LFM — the parser core is well thought out (AST-based, positional-arg rejection, balanced-bracket detection with quoted-string awareness, streaming hold + flush_held_content at stream end), and the AI-assistance disclosure is appreciated. Two blockers must be fixed before this can merge, plus a few observations.

Blockers

1. aliases.json is not valid JSON — import vllm_mlx will crash

Reproducing from the PR branch:

$ python3 -c "import json; json.load(open('vllm_mlx/aliases.json'))"
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 779 column 3

The lfm2.5-1b entry is missing its closing } and trailing comma. The current shape is:

  "lfm2.5-1b": {
    "hf_path": "mlx-community/LFM2.5-1.2B-Instruct-4bit",
    ...
    "supports_spec_decode": false
  "diffusion-gemma-26b-4bit": {   ← syntax error here

This means the [x] Tests pass locally (python3 -m pytest tests/ -x) checkbox can't have been ticked truthfully — aliases.json is loaded at module import time, so the very first from vllm_mlx import … in any test crashes. Could you run the test suite locally end-to-end (no -x, full collection) to catch import-time failures like this one before re-pushing?

2. Alias names violate the project's <family>-<size>-<quant> SOP

Both new aliases ship without an explicit quant suffix:

  "lfm2-24b": { "hf_path": "lmstudio-community/LFM2-24B-A2B-MLX-4bit", ... },
  "lfm2.5-1b": { "hf_path": "mlx-community/LFM2.5-1.2B-Instruct-4bit", ... },

Per the project-local naming SOP (see CLAUDE.md in this repo):

Every alias key in vllm_mlx/aliases.json MUST follow <family>-<size>-<quant>. … An alias whose hf_path quant doesn't match the alias suffix … silently swaps weights on operators.

lfm2-24b is MoE (is_moe: true) with A2B active experts and a 4-bit checkpoint, and lfm2.5-1b is a 4-bit dense checkpoint. Please rename:

• lfm2-24b → lfm2-24b-a2b-4bit (matches HF naming LFM2-24B-A2B-MLX-4bit)
• lfm2.5-1b → lfm2.5-1b-4bit

Why this matters: operators pin rapid-mlx serve <alias> in startup scripts. If a future PR offers an 8-bit variant under the same family and someone retargets the bare lfm2-24b alias to it, every pinned deployment silently doubles its VRAM and may OOM in production. The suffix makes the pin explicit and forces a documented migration when the quant tier changes. This rule cost us a revert on PR #558 (diffusion-gemma-26b bare-alias swap), so I'm tight on enforcing it.

Significant concerns (not strict blockers)

3. Author-acknowledged WIP — convert to Draft until benched

The PR description leads with:

"Work in progress: I don't have the hardware to run the full benchmarks, so the README and benchmark updates are not done."

Two ways forward, either is fine:

• (A) Mark as Draft until you can borrow time on a 32GB+ Mac. I'm happy to do the bench pass on my M3 Ultra (256GB) and post results once Blockers 1+2 are fixed — drop me a ping. The Model Onboarding SOP wants suffix_decoding_tier set and at least a smoke run of the canonical tool-call prompts before going green.
• (B) Keep as ready-for-review with suffix_decoding_tier: "unknown" (which you already have — good) and supports_dflash: false, but please call out in the PR body that those fields stay unknown/false until benched, so a future onboarding sweep knows to fill them in.

Either way: the title should not be feat: if the work is still WIP — the convention here is feat(wip): or just Draft state. (Squash-merge keeps the prefix in the commit message.)

4. Postprocessor → parser layering inversion (NIT)

vllm_mlx/service/postprocessor.py now hard-imports LFM_CALL_START from a specific parser to extend its "plausible markup" pre-check:

from ..tool_parsers.lfm_tool_parser import LFM_CALL_START
…
_has_plausible_markup = bool(_fallback_text) and (
    "<" in _fallback_text
    or "{" in _fallback_text
    or "[Calling" in _fallback_text
    or LFM_CALL_START.search(_fallback_text) is not None
)

This grows linearly with every new parser format we add — eventually the postprocessor sprouts an import-and-marker line per parser. Cleaner shape: each ToolParser subclass exposes a quick_marker_present(text) -> bool classmethod, and the postprocessor iterates over registered parsers. Out of scope for this PR (the current shape is already established with <, {, [Calling), but flagging so we can clean it up across all parsers in one go.

5. extract_tool_calls_streaming loose end-marker (NIT)

Both AutoToolParser (your changes) and LfmToolParser triggered extraction on a bare ] in the delta, then re-trigger on every subsequent ]. The _streaming_tools_emitted flag correctly fixes the "re-emit corrupts arguments" case (good catch), but every chatty response containing ] (markdown lists, JSON output, even code) now runs the full extractor and fails — adding latency per ]-containing delta on non-tool responses.

Lighter shape: gate the bare-] trigger on LFM_CALL_START.search(current_text) being non-None first. That keeps the extra extractor call off the prose path entirely.

Not blocking — performance impact is per-delta-with-], not per-token — but worth tightening in a follow-up.

What's good

• AST-only argument parsing (no eval 👍), explicit rejection of positional args, and bracket-balanced extraction with string/escape awareness — exactly right for this format.
• _safe_content_prefix holding partial [name(… until balance arrives is the correct streaming shape, and the flush_held_content hook landing where it does means the abstract-parser contract is honored.
• 337 LOC test file + the streaming-parity fixture + the native-format flag update is good coverage.
• Honest AI-assistance disclosure and Codex review attribution — appreciated.

Summary

• Step 0 (does this solve a real product problem): ✅ LFM2 is a popular small-model line and #85 is a real request.
• Supply-chain audit: ✅ Clean — no new deps, parser uses ast from stdlib only, no network calls.
• Action:
  • Fix Blocker 1 (JSON syntax) and re-run pytest tests/ -k tool_parser locally to confirm imports work.
  • Fix Blocker 2 (rename to lfm2-24b-a2b-4bit and lfm2.5-1b-4bit).
  • Decide on #3 — convert to Draft and let me bench, or stay ready-for-review with explicit "unknown until benched" call-out.
  • #4 and #5 are NITs — happy to land follow-ups myself after merge.

Thanks again for the well-structured parser implementation — once Blockers 1 + 2 are fixed I can take it from here on bench data.

[necaris]

@raullenchai thank you for the thorough review! I've converted to draft and will fix 1 & 2 before pinging you for help with the benchmarks!

[necaris]

1. aliases.json is not valid JSON — import vllm_mlx will crash

Thanks for the catch! Looks like when I merged from main this got overwritten -- I didn't rerun the tests after that, so it was missed. Fixed and rerun:

=========================================== 4833 passed, 19 skipped, 155 deselected, 6 xfailed in 60.09s (0:01:00) ===========================================

2. Alias names violate the project's -- SOP

I had not known this, thank you for educating me! Fixed and rebased into the main change.

Bench data

I got a colleague to run benchmarks on their M4 Max -- this is the snippet from the scorecard.md. I'm not sure what I need to extract and add to the PR but I'll dig further... really just saying I've got this.

Generated: 2026-06-12T14:08:04

Model	Decode TPS	Cold TTFT	Cached TTFT	Tool %	Score	Status
lfm2-24b-a2b-4bit	148.9	51ms	88ms	100%	385.5	OK
lfm2.5-1b-4bit	413.8	119ms	60ms	0%	682.4	OK

[necaris]

@raullenchai I've addressed #1, #2, #3, and #5 and added a TODO: comment for #4 to be cleaned up in a future pass. Unfortunately I'm not able to complete the benchmarking with all the models -- although I have confirmed with a colleague's machine that the benchmarks do at least run with the LFM models.

I'll leave it as draft until the benchmarking can be done, but I think it's at a point where you could pick up the benchmark runs?

[raullenchai]

Hi @necaris — just checking in! The structural side of this PR looks ready: parser at 333 LOC, full test coverage (test_lfm_tool_parser.py + streaming-parity fixture in test_tool_call_streaming_parity.py), alias + auto-config wired up, AI-assistance disclosure is great.

You noted you wanted to leave it in draft "till benchmarks can be run" — is there anything blocking that on your end? If it's hardware, we can run the live-server gate on our M3 here (the parser unit tests cover the structural correctness; what'd remain is a smoke run against an actual mlx-community/LFM2-* model to confirm tool-call extraction works end-to-end on real model output).

Specifically, if you can either:

1. Mark it "Ready for review" and we'll take it from here — we'll boot the LFM2 model, run a short tool-call smoke, and report back, OR
2. Let us know which LFM2 variant you've been testing against, and we'll target that same one,

…we can land this without making you babysit it any longer.

No rush either way — just wanted to make sure it doesn't sit in draft purgatory.

[necaris]

@raullenchai I'll mark it ready for review, then -- I was under the impression benchmark results needed to be updated including other models that I don't have the hardware for, but if that's not the case I'm very confident this is correct!

[raullenchai]

Triage assessment (deferring final review + merge to @raullenchai):

Structural state of this PR looks solid based on the prior review thread:

• Blockers 1 (JSON syntax) and 2 (alias naming SOP <family>-<size>-<quant>) addressed — lfm2-24b-a2b-4bit and lfm2.5-1b-4bit now match the HF quant suffix.
• Test suite reported green locally (4833 passed, 19 skipped) after the rebase.
• AST-only argument parsing, no eval, balanced-bracket extraction with string/escape awareness — correct shape for this format.
• 333 LOC parser + 337 LOC test file + streaming-parity fixture is appropriate coverage.
• Bench data from a colleague's M4 Max (lfm2-24b-a2b-4bit: 148.9 TPS / 100% tool %, lfm2.5-1b-4bit: 413.8 TPS) is in the thread.
• NIT fix: Prevent server crash from malformed response_format schemas #4 (postprocessor → parser layering) has a TODO marker for a future cleanup pass — fine.

Why I'm not merging from triage:

• First external contribution to this repo, so per our policy auto-merge is reserved for trusted contributors with prior merges here.
• The pending smoke run @raullenchai promised (mlx-community/LFM2-* end-to-end tool-call against the live server) hasn't been recorded yet on this PR.
• pr_validate scorecard isn't visible on this PR thread; would want that in writing before merge.

@raullenchai — next action is yours: run the LFM2 live-server tool-call smoke + pr_validate, then merge if both clear. Nothing else looks like it needs another revision round from @necaris.

@necaris — thanks for the patience and for cleanly addressing the SOP feedback. Nothing requested from you right now; this is just queued behind one operator smoke run.

[necaris]

Anything I can do to push this forward?

[TomLucidor]

@necaris first thing that comes to my mind is to make sure the changes apply to v0.10.2 (or whatever is in development), and make sure everything still works. And then the "smoke tests" and maybe some other tests?
Found out the discrepancy here, and 3 versions probably have enough changes to warrant a double-check https://github.com/necaris/Rapid-MLX/tags

[TomLucidor]

Thanks @necaris and hopefully this + Hermes agent can be tested to show that reasoning + tool selection are functional

Check the following for my current report using AI to auto-test this feature

Live-server test report: LfmToolParser (PR #552) — LFM2.5-8B-A1B, 4-bit and 8-bit

Tester: independent (not the PR author). Offering the live-server smoke data identified as the merge blocker in the triage comment.

Environment

Item	Value
Hardware	Apple M3 Max, 48 GB
OS	macOS 26.5.1 (arm64)
Branch tested	pr-552 (133d540) rebased onto current main → builds as rapid-mlx 0.10.2
Install	clean venv (uv, Python 3.12.11), pip install -e '.[embeddings]'
Models	LiquidAI/LFM2.5-8B-A1B-MLX-4bit and -8bit (official LiquidAI quants)
Serve flags	--tool-call-parser lfm --enable-auto-tool-choice unless noted
Determinism controls	temperature=0; persistent prefix cache (~/.cache/rapid-mlx/prefix_cache) cleared before restart-stability runs

Rebase note: 3 small, purely additive conflicts (vllm_mlx/aliases.json, tests/test_tool_call_streaming_parity.py, scripts/audit_tool_parser_coverage.py). tests/test_lfm_tool_parser.py: 23/23 pass post-rebase.

⚠️ Recommend rebasing before merge. On the PR's original base (builds as 0.7.32): streaming emitted no tool-call deltas, the multi-turn roundtrip produced spurious repeated calls, and <think> content leaked into streamed delta.content. All three are clean on the rebased build — the fixes come from main, so the PR just needs the rebase to benefit.

Headline verdict

The parser works. In every test where the model emitted its native markup (<|tool_call_start|>[func(args)]<|tool_call_end|> or bare [func(args)]), LfmToolParser converted it into correct OpenAI-style tool_calls — names, arguments, single and multiple calls, streaming and non-streaming. When the model repeated markup, the parser faithfully produced N identical calls (see "Observations" for a dedup suggestion).

Every failure below traces to the model not emitting markup, which further isolated to the auto-injected _TOOL_USE_SYSTEM_SUFFIX (service/helpers.py:1473, injected at routes/chat.py:1875 whenever tools are present and a parser is active) — outside this PR's scope, filed separately as #[ISSUE-NUMBER].

Results matrix

"sysmsg" = a short counter-priming system message: "You are a function-calling assistant. Think step by step in your thinking block, then emit the tool call in your native format. Your reasoning process is welcome."

4-bit (LiquidAI/LFM2.5-8B-A1B-MLX-4bit)

Test	Result	Detail
Explicit call, bare request	❌ 5/5 fail	Deterministic across 5 cold restarts (cache cleared): model emits a fabricated tool result — Tool returned: {"status": "success", ...} — verbatim one of the phrases the injected suffix names as forbidden. No markup produced; nothing for the parser to parse.
Explicit call, + sysmsg	✅ 5/5	Single call, {"x": "diagnose"}, empty content — cold cache
Explicit multi-tool, + sysmsg	✅ 3/3	calculator_add, {"a": 17, "b": 25}
Judgment multi-tool ("use the correct tool"), + sysmsg	❌ 0/3	Deterministic narration: 17 + 25 = 42\n\nTool: calculator_add with arguments {...} — right tool identified, no markup emitted
Control: same judgment payload, bare mlx_lm (no server, no suffix)	✅	Perfect `…<
Streaming explicit, + sysmsg	✅	1 tool_call delta chunk, clean content, finish=tool_calls
Multi-turn roundtrip (assistant tool_calls + tool result in history), + sysmsg	✅	Text answer FINAL diagnose, no spurious calls

4-bit summary: model and quant are fine — the bare-mlx_lm control proves the official 4-bit produces flawless native markup for the exact payload that fails through the server. The injected suffix suppresses autonomous (judgment-based) emission even with the counter-priming message, and elicits the named-phrase hallucination without it.

8-bit (LiquidAI/LFM2.5-8B-A1B-MLX-8bit)

Test	Result	Detail
Judgment multi-tool, + sysmsg	✅ 3/3	Clean single calculator_add calls — the cell that fails 0/3 on 4-bit
Judgment multi-tool, bare	⚠️ 3/3 matched, quality issues	Run 1: 3 duplicate identical calls + answer text (42) leaked into content; runs 2–3 clean
Explicit call, bare	⚠️ 1/3	Two runs narrated ("The user's request to call the echo function…") instead of emitting markup
Explicit call, + sysmsg	⬜ not tested	(Implied by the pattern; happy to run on request)
Streaming judgment, + sysmsg	✅	1 tool_call delta, clean content, finish=tool_calls

8-bit summary: with the counter-priming system message, green across everything tested, including the autonomous-selection case that defeats the 4-bit. Without it, the suffix still destabilizes output (narration, duplication, content leakage) — just in different places than the 4-bit.

Cross-cutting observations

1. Suffix interaction (filed as #[ISSUE-NUMBER]). The decisive A/B: byte-identical judgment payload → bare mlx_lm emits perfect markup; served with the injected suffix → deterministic narration (3/3). There is currently no opt-out flag for the injection. Effects are capability/quantization-dependent: 4-bit shows judgment suppression and named-phrase hallucination; 8-bit shows intermittent explicit-call narration and call duplication. A substantive user system message partially (4-bit) or fully (8-bit) stabilizes it.
2. Duplicate-call handling. When the model repeats identical markup in one response (observed on both quants), the parser emits N identical tool_calls. Faithful, but agent frameworks then execute twice or trip on tool_call_id bookkeeping. Worth considering an opt-in dedup of byte-identical consecutive calls, or a documented recommendation that callers dedup.
3. --enable-auto-tool-choice is effectively required. Content→tool_calls extraction is gated on it (service/helpers.py:2724), and without it, emitted markup is left unparsed in content. Meanwhile the failure mode with tools but without useful config looks like a model defect rather than a config gap. A docs line ("LFM models require --enable-auto-tool-choice") — or auto-enabling via the alias profile — would prevent misdiagnosis; I initially lost time to exactly this.
4. Auto-config wiring works. The \blfm|\bliquid pattern in model_auto_config.py correctly wires LiquidAI/LFM2.5-8B-A1B-*, which is absent from aliases.json — nice touch, since the alias table only carries the 24B and 1.2B entries.

Suggested smoke recipe for the maintainer's merge gate

To avoid false negatives from the (out-of-scope) suffix interaction:

• Use the 8-bit variant, or include a substantive system message with the 4-bit
• Explicit invocation phrasing ("Call X with a=1") is reliable on both quants with a system message; autonomous phrasing ("use the correct tool") is only reliable on 8-bit
• temperature=0 alone does not guarantee cross-restart determinism unless the persistent prefix cache is cleared — cached generations replay the first roll

Repro: minimal passing case (either quant)

rapid-mlx serve LiquidAI/LFM2.5-8B-A1B-MLX-8bit --host 127.0.0.1 --port 8215 \
  --tool-call-parser lfm --enable-auto-tool-choice &

curl -s http://127.0.0.1:8215/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "LiquidAI/LFM2.5-8B-A1B-MLX-8bit",
  "stream": false, "temperature": 0, "max_tokens": 1024,
  "messages": [
    {"role": "system", "content": "You are a function-calling assistant. Think step by step in your thinking block, then emit the tool call in your native format. Your reasoning process is welcome."},
    {"role": "user", "content": "Add 17 and 25 using the correct tool. Do not answer normally."}
  ],
  "tools": [
    {"type":"function","function":{"name":"echo","description":"echo text","parameters":{"type":"object","properties":{"x":{"type":"string"}},"required":["x"]}}},
    {"type":"function","function":{"name":"calculator_add","description":"Add two integers","parameters":{"type":"object","properties":{"a":{"type":"integer"},"b":{"type":"integer"}},"required":["a","b"]}}}
  ]
}' | jq '.choices[0].message.tool_calls'

Expected: single calculator_add call with {"a": 17, "b": 25}.

Repro: the out-of-scope suffix failure (4-bit, for the linked issue)

Same server on the 4-bit model, same payload — returns narration in content ("17 + 25 = 42 … Tool: calculator_add with arguments …") with tool_calls: null, 3/3 at temp=0. The identical payload through bare mlx_lm.generate (no server) yields <|tool_call_start|>[calculator_add(a=17, b=25)]<|tool_call_end|>.

Bottom line

• Parser: correct and ready. Merge-blocking smoke evidence is above; I found no case where valid markup was mis-parsed.
• Please rebase before merge — the streaming/roundtrip wins come from main.
• The remaining LFM end-to-end issues are pre-existing serving-layer behavior (suffix injection), tracked separately in #[ISSUE-NUMBER], and shouldn't hold this PR.

Full harness (payload files + runner script) available if useful.

[necaris]

@TomLucidor thanks for the comments! I've rebased this up to the latest main -- as I recall what was holding it up was running the full set of smoke tests / benchmarks, which I don't have the hardware to do.

[TomLucidor]

@necaris there is always a smaller model to test tool calling (ideally 8B-A1B, 1.2B-Instruct, 350M or 230M as model of last resort) since 24B-A2B is for LFM2 not the newer LFM2.5, also Hermes agent or OpenCode as long-context test targets kinda would make sense even for 1.2B-Instruct with 4GB for 128K or 3GB or 64K (8B-A1B would need 18GB for 128K or 14GB for 64K).

[raullenchai]

Thanks for keeping this alive @necaris, and thanks @TomLucidor for pitching in. Quick triage update to unblock the mechanical side:

1. Still needs a rebase. Even after your 2026-07-08 rebase, GitHub currently reports this branch as CONFLICTING against main (main moves fast — several parser/alias-adjacent PRs have landed since). Could you rebase feat/lfm2.5-parser onto the latest main once more and force-push? I'm intentionally not rebasing the fork for you. The likely conflict surface is vllm_mlx/aliases.json and vllm_mlx/tool_parsers/__init__.py / auto_tool_parser.py, which see frequent edits.

2. Good news on the mirror / smoke-test blocker. The long-standing "I don't have the hardware to run the models" blocker is smaller than it looks — the exact repos your aliases target are already live on the mlx-community mirror:

• mlx-community/LFM2.5-8B-A1B-MLX-4bit ✓
• mlx-community/LFM2.5-1.2B-Instruct-4bit ✓
• plus small siblings ideal for a cheap tool-calling smoke test: mlx-community/LFM2.5-350M-bf16, mlx-community/LFM2-1.2B-4bit.

So the parser can be exercised end-to-end on real weights. A single LFM2.5-350M / LFM2-1.2B tool-calling round-trip (not a full benchmark sweep) would be enough to demonstrate the wire format matches your parser.

3. Final merge decision is @raullenchai's call. Per his earlier note on this thread, adding a new model family (LFM2.x) and the final review/merge are reserved for him — so I'm leaving this open rather than merging, even though the structure looks solid. Once it's rebased clean and there's one real tool-calling round-trip logged, it's in good shape for his sign-off.

Rebase + one smoke-test round-trip → and this should be ready. Thanks again!

[TomLucidor]

@raullenchai what is the standard way to get Hermes agent or some other agent to standard stress-test rapid-mlx with multi-round tool calling + tool choice?

[necaris]

@raullenchai noticed #1076 fully supersedes this, closing in favor of that 😁

[raullenchai]

Landed via #1076 (squash 83bd917c) with your commit cherry-picked — thank you @necaris 🙏. Rebased onto current main (zero conflicts), the stale aggregated.json timestamp bump dropped, and the lfm MATRIX_EXEMPT placeholder upgraded to a documented exemption referencing #85. Parser verified against real LFM2.5-1.2B output (<|tool_call_start|>[func(kw=val)]<|tool_call_end|> → correct extraction, prose preserved, no markup leak); streaming + \blfm/\bliquid word-boundary checked; full parser/config suites green on CI. Closes #85.

[raullenchai]

LiquidAI LFM2.x tool-call parser support landed in #1076 and shipped in rapid-mlx 0.10.7 (PyPI + Homebrew). LFM2.x aliases now route to the dedicated lfm parser. Thanks for the request.