Skip to content

feat(cli): add jarvis model convert for local HF→MLX/GGUF conversion#542

Open
gilbert-barajas wants to merge 1 commit into
open-jarvis:mainfrom
gilbert-barajas:feat/model-convert-251
Open

feat(cli): add jarvis model convert for local HF→MLX/GGUF conversion#542
gilbert-barajas wants to merge 1 commit into
open-jarvis:mainfrom
gilbert-barajas:feat/model-convert-251

Conversation

@gilbert-barajas

Copy link
Copy Markdown
Contributor

Summary

Adds jarvis model convert <hf_repo> — downloads raw HuggingFace weights and converts them to the active engine's format, closing the gap for models that exist on HF but ship no pre-built GGUF or MLX artifact. Today jarvis model pull only fetches pre-converted artifacts (Ollama /api/pull, a specific .gguf, or a pre-quantized mlx_repo); if a model has none, there's no path forward. This wires up the well-tested conversion tools — mlx_lm.convert for Apple MLX and convert_hf_to_gguf.py + llama-quantize for llama.cpp.

Closes #251.

What it does

jarvis model convert <hf_repo> [--engine ENGINE] [--quantize q4_k_m] [-q/--mlx-4bit] [--output DIR] [--force]
  • MLX (--engine mlx): mlx_lm.convert(...), with -q for 4-bit. Guarded so a missing inference-mlx extra prints an install hint instead of a stack trace.
  • GGUF (--engine llamacpp): shells convert_hf_to_gguf.py, then llama-quantize when --quantize is given. Tools located via $LLAMA_CPP_DIR or PATH, with graceful "not found" hints (mirroring how pull's hf_download handles a missing huggingface-cli).
  • Engine resolution mirrors pull (--engineconfig.engine.defaultmlx). Non-convertible engines (ollama/vllm/sglang) print an informational note pointing at pull (exit 0).
  • Refuses to overwrite a non-empty --output without --force. On success, prints the artifact path and the exact command to serve it.

It's a subcommand on the existing model group in cli/model.py, following pull's conventions (Click, rich, sys.exit(1) on error).

Testing

tests/cli/test_model_convert.py — 7 CliRunner tests with converters fully mocked (mlx_lm.convert, subprocess.run, hf_download), so nothing downloads or shells in the default lane:

  • mlx + -q calls mlx_lm.convert(quantize=True)
  • missing inference-mlx → install hint, exit 1
  • engine-default resolution (no --engine → config default)
  • gguf shells the converter + llama-quantize
  • missing tool → install hint, exit 1
  • non-empty --output refusal + --force override
  • ollama → informational, exit 0

Gates run locally with the pinned toolchain:

  • uv run ruff check src/ tests/ — clean
  • uv run ruff format --check src/ tests/ — clean
  • uv run pytest tests/cli/test_model_convert.py -v — 7 passed

Out of scope (happy to follow up)

Auto-registering the converted model into the catalog so model list surfaces it without the engine running — left out to keep this focused; glad to add it in a follow-up if useful.

🤖 Generated with Claude Code

Closes open-jarvis#251. Adds a `convert` subcommand to `jarvis model` that downloads raw
HuggingFace weights and converts them to the active engine's format — filling
the gap for models that exist on HF but ship no pre-built GGUF/MLX artifact
(which `jarvis model pull` can't handle).

- MLX path: `mlx_lm.convert(...)`, guarded so a missing `inference-mlx` extra
  fails with an install hint instead of a stack trace.
- GGUF/llamacpp path: shells `convert_hf_to_gguf.py` then `llama-quantize`,
  located via `$LLAMA_CPP_DIR` or PATH, with graceful "tool not found" hints.
- Engine resolution mirrors `pull` (`--engine` → `config.engine.default`);
  flags `--quantize`, `-q/--mlx-4bit`, `--output`, `--force`. Non-convertible
  engines (ollama/vllm/sglang) print an informational note pointing at `pull`.
- On success, prints the artifact path and the exact command to serve it.

Tests: tests/cli/test_model_convert.py — 7 CliRunner tests with mocked
converters (mlx_lm.convert / subprocess / hf_download) covering both engine
paths, the missing-extra and missing-tool errors, engine-default resolution,
the force/refusal guard, and the informational path. `ruff check` and
`ruff format --check` clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@josephdhand568-cpu josephdhand568-cpu left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Open Jarvis

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

[Feature Request] Weight conversion between inference server locally

3 participants