serve: tool calls declared, slots that hold the context, longer answers
CI / test (push) Failing after 2m41s
CI / test (push) Failing after 2m41s
A --context-length 32768 --ctx-pool 32768 model got llama.cpp's default four slots of 8192, and refused what the gateway admitted; --parallel is now what the pool holds of the context. max_output was 1024 for every model, which cut an editor's agent off mid-file; it is now a quarter of the context, from 1024 to 8192. --tools writes capabilities.tools, which the Continue configuration now reads, as it reads max_output. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
df6d243ee7
commit
481523ca46
8 files changed
+111
-8
No files matched your search
@@ -184,6 +184,15 @@ asks for, send the results back. Whether a model calls tools well is the
|
||||
model's business, not the gateway's; ask your operator which of the served
|
||||
models are set up for it.
|
||||
|
||||
**Editors' agents need such a model.** Continue's Agent mode, Cline and
|
||||
Codex act only through tool calls: with a model whose calls come back as
|
||||
text — `<tool_call>{...}` or a JSON block in the answer, `tool_calls` empty —
|
||||
they chat but never read a file or run a command. Qwen2.5-Coder is one;
|
||||
Qwen3-Coder is not. The configuration the portal writes for Continue says
|
||||
`tool_use` only for a model whose catalog entry says tools, and gives
|
||||
Continue the model's context and longest answer, so it never sends a request
|
||||
the gateway refuses.
|
||||
|
||||
```python
|
||||
import json
|
||||
|
||||
|
||||
Reference in new issue
Block a user