Skip to content

Expose Harmony/GPT-OSS tool parser in serve CLI#4

Merged
krystophny merged 1 commit intomainfrom
feature/expose-harmony-tool-parser
Mar 24, 2026
Merged

Expose Harmony/GPT-OSS tool parser in serve CLI#4
krystophny merged 1 commit intomainfrom
feature/expose-harmony-tool-parser

Conversation

@krystophny
Copy link
Copy Markdown
Collaborator

@krystophny krystophny commented Mar 24, 2026

Summary

  • expose harmony and gpt-oss as valid --tool-call-parser values for vllm-mlx serve
  • factor CLI parser construction so the flag surface is unit-testable
  • add regression tests proving the serve CLI accepts the parser choices already registered in code

Why this is independently deployable

  • CLI-only surface fix for already-registered parsers
  • no batching, cache, scheduler, or API protocol change
  • immediately useful for local GPT-OSS/Harmony tool-use without launcher-side hacks

Related upstream context

In upstream waybarrios/vllm-mlx, the parser building blocks already exist:

The gap this PR fixes is narrower: those parsers were registered, but the serve CLI did not expose them consistently via --tool-call-parser.

Related ongoing work around tool-choice / tool parsing behavior upstream:

This PR does not try to solve those behavioral questions. It only exposes the parser selections cleanly.

Validation

  • PYTHONPATH=/Users/ert/code/vllm-mlx /Users/ert/code/.venv/bin/python -m pytest tests/test_server.py -q
  • python3 -m compileall vllm_mlx

What could still improve

  • end-to-end HTTP tests covering --tool-call-parser harmony and --tool-call-parser gpt-oss
  • matrix tests covering parser selection plus tool_choice=auto/none/required

@krystophny krystophny merged commit 53c3df8 into main Mar 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant