Skip to content

fix: change default Ollama model to qwen2.5:7b (coder variant breaks tool calls)#9

Merged
morganlinton merged 1 commit into
GetSmallAI:mainfrom
fubak:fix/ollama-default-model-tool-calls
Jun 29, 2026
Merged

fix: change default Ollama model to qwen2.5:7b (coder variant breaks tool calls)#9
morganlinton merged 1 commit into
GetSmallAI:mainfrom
fubak:fix/ollama-default-model-tool-calls

Conversation

@fubak

@fubak fubak commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Problem

The default Ollama model qwen2.5-coder:7b does not use Ollama's structured tool_calls API. When SmallHarness sends tool definitions with a request, the coder variant emits tool invocations as raw JSON text inside the content field:

{"name": "shell", "arguments": {"command": "ls"}}

SmallHarness reads tool calls from the tool_calls array in the API response, not from response text. These invocations are never executed — the session displays them as prose and stalls, with no error message to explain why.

This is the first thing a new Ollama user hits, and there's no indication of what's wrong.

Fix

Change the default from qwen2.5-coder:7b to qwen2.5:7b.

The base model uses the <tool_call> XML template that Ollama correctly translates into structured tool_calls responses. Tool calls from the base model are received and executed as expected in both interactive and one-shot (--print) modes.

Users who specifically want the coder fine-tune can set modelOverride in agent.config.json. The coder variant is a fine coding model — it just cannot reliably invoke tools, which is a prerequisite for an interactive coding agent.

Test

Added defaults_ollama_to_base_qwen_not_coder_variant which asserts the default is qwen2.5:7b and explicitly rejects qwen2.5-coder:7b with a comment explaining why.

All 384 tests pass.

Verified with

Ollama 0.30.8, both qwen2.5:7b and qwen2.5-coder:7b pulled locally. The base model calls tools correctly; the coder variant emits raw JSON text.

qwen2.5-coder:7b does not use Ollama's structured tool_calls API. When
SmallHarness sends tool definitions with a request, the coder variant
emits tool invocations as raw JSON text inside the content field:

  {"name": "shell", "arguments": {"command": "..."}}

SmallHarness parses tool calls from the API-level tool_calls array, not
from response text, so these calls are never executed. The session
displays the JSON as prose and stalls.

qwen2.5:7b (the base model) uses the same <tool_call> XML template that
Ollama translates into structured tool_calls responses. Tool calls from
this model are received correctly and executed as expected.

The coder variant is otherwise a reasonable default for coding tasks, but
tool-call reliability is a prerequisite for an interactive coding agent.
Users who specifically want the coder fine-tune can set modelOverride in
agent.config.json.

Verified with Ollama 0.30.8, qwen2.5:7b and qwen2.5-coder:7b.
Copilot AI review requested due to automatic review settings June 15, 2026 02:23

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Updates Ollama’s default model selection to prefer the Qwen2.5 base model due to structured tool call support, and adds a regression test to lock in the behavior.

Changes:

  • Switch default Ollama model from qwen2.5-coder:7b to qwen2.5:7b.
  • Add a unit test asserting Ollama defaults to the base variant (and not the coder variant).
  • Document the rationale for the default change in code comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/backends.rs
Comment on lines +242 to +245
// qwen2.5:7b (base) must be the default, not qwen2.5-coder:7b.
// The coder variant emits tool calls as raw JSON text rather than
// via Ollama's structured tool_calls API, so SmallHarness cannot
// execute them in interactive or one-shot mode.
Comment thread src/backends.rs
Comment on lines +247 to +251
assert_eq!(model, "qwen2.5:7b");
assert_ne!(
model, "qwen2.5-coder:7b",
"coder variant does not support structured tool_calls"
);

@bennewell35 bennewell35 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed the diff and ran the upstream-style local checks in a disposable Docker Rust environment.

Validation run:

docker run --rm -v /tmp/smallharness-pr9:/work -w /work rust:latest bash -c 'rustup component add rustfmt clippy >/dev/null && rustc --version && cargo --version && cargo fmt --all -- --check && cargo clippy --all-targets -- -D warnings && cargo test'\n```\n\nResult:\n- `rustc 1.96.0` / `cargo 1.96.0`\n- `cargo fmt --all -- --check` passed\n- `cargo clippy --all-targets -- -D warnings` passed\n- `cargo test` passed: 381 passed, 0 failed\n- The new `defaults_ollama_to_base_qwen_not_coder_variant` test passed as part of the suite\n\nI did not live-test Ollama/model tool-call behavior on my machine, so this is code/test validation rather than independent model-runtime validation. The patch is narrow and the regression test covers the intended default-model contract.

@morganlinton morganlinton merged commit 2672593 into GetSmallAI:main Jun 29, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants