Files
llmgw/internal
Ilker ManapandClaude Opus 5 9fc098fd5b Start llama.cpp with the model's own chat template
Tool calling did not work, and it looked like the model's fault. Asked to
use a tool, qwen2.5-coder produced exactly the right call —
{"name": "list_files", "arguments": {"path": "."}} — as plain reply text,
with tool_calls null and finish_reason "stop". Every OpenAI-compatible
client, opencode among them, sees a model that ignores the tools it was
given.

The cause is that llama-server falls back to a generic chat template unless
--jinja is passed, and the generic one neither formats the tool definitions
the way the model was trained to read them nor parses what comes back into
tool_calls. It affects ordinary chat too, for the same reason.

A profile can still pass --no-jinja, because a model whose bundled template
is broken has to be serviceable without editing the adapter.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 21:20:55 +02:00
..