feat(cli): add jarvis model convert for local HF→MLX/GGUF conversion#542
Open
gilbert-barajas wants to merge 1 commit into
Open
feat(cli): add jarvis model convert for local HF→MLX/GGUF conversion#542gilbert-barajas wants to merge 1 commit into
gilbert-barajas wants to merge 1 commit into
Conversation
Closes open-jarvis#251. Adds a `convert` subcommand to `jarvis model` that downloads raw HuggingFace weights and converts them to the active engine's format — filling the gap for models that exist on HF but ship no pre-built GGUF/MLX artifact (which `jarvis model pull` can't handle). - MLX path: `mlx_lm.convert(...)`, guarded so a missing `inference-mlx` extra fails with an install hint instead of a stack trace. - GGUF/llamacpp path: shells `convert_hf_to_gguf.py` then `llama-quantize`, located via `$LLAMA_CPP_DIR` or PATH, with graceful "tool not found" hints. - Engine resolution mirrors `pull` (`--engine` → `config.engine.default`); flags `--quantize`, `-q/--mlx-4bit`, `--output`, `--force`. Non-convertible engines (ollama/vllm/sglang) print an informational note pointing at `pull`. - On success, prints the artifact path and the exact command to serve it. Tests: tests/cli/test_model_convert.py — 7 CliRunner tests with mocked converters (mlx_lm.convert / subprocess / hf_download) covering both engine paths, the missing-extra and missing-tool errors, engine-default resolution, the force/refusal guard, and the informational path. `ruff check` and `ruff format --check` clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
jarvis model convert <hf_repo>— downloads raw HuggingFace weights and converts them to the active engine's format, closing the gap for models that exist on HF but ship no pre-built GGUF or MLX artifact. Todayjarvis model pullonly fetches pre-converted artifacts (Ollama/api/pull, a specific.gguf, or a pre-quantizedmlx_repo); if a model has none, there's no path forward. This wires up the well-tested conversion tools —mlx_lm.convertfor Apple MLX andconvert_hf_to_gguf.py+llama-quantizefor llama.cpp.Closes #251.
What it does
--engine mlx):mlx_lm.convert(...), with-qfor 4-bit. Guarded so a missinginference-mlxextra prints an install hint instead of a stack trace.--engine llamacpp): shellsconvert_hf_to_gguf.py, thenllama-quantizewhen--quantizeis given. Tools located via$LLAMA_CPP_DIRorPATH, with graceful "not found" hints (mirroring howpull'shf_downloadhandles a missinghuggingface-cli).pull(--engine→config.engine.default→mlx). Non-convertible engines (ollama/vllm/sglang) print an informational note pointing atpull(exit 0).--outputwithout--force. On success, prints the artifact path and the exact command to serve it.It's a subcommand on the existing
modelgroup incli/model.py, followingpull's conventions (Click,rich,sys.exit(1)on error).Testing
tests/cli/test_model_convert.py— 7CliRunnertests with converters fully mocked (mlx_lm.convert,subprocess.run,hf_download), so nothing downloads or shells in the default lane:-qcallsmlx_lm.convert(quantize=True)inference-mlx→ install hint, exit 1--engine→ config default)llama-quantize--outputrefusal +--forceoverrideGates run locally with the pinned toolchain:
uv run ruff check src/ tests/— cleanuv run ruff format --check src/ tests/— cleanuv run pytest tests/cli/test_model_convert.py -v— 7 passedOut of scope (happy to follow up)
Auto-registering the converted model into the catalog so
model listsurfaces it without the engine running — left out to keep this focused; glad to add it in a follow-up if useful.🤖 Generated with Claude Code