The catalog now declares whether a model supports tools, which makes it a
claim. This is the claim that can be checked, and the one worth checking:
the gateway forwards a tools array to any model, and an engine that does
not return structured tool_calls looks exactly like a model that ignores
its tools. Every OpenAI-compatible client then behaves as though the model
is broken.
It separates the two diagnoses, which have the same symptom and different
fixes. The model ignoring its tools is fixed by choosing another model. The
model making the call and the engine handing it back as prose is fixed by
changing engine build, and no larger model will help — which is what
today's investigation established only after eliminating the gateway, the
model size, two chat templates and tool_choice in turn.
Against the live engine it says:
the model called the tool and the engine did not parse it.
{"name": "get_weather", "arguments": {"city": "Istanbul"}}
This is the engine, not the model.
engine-check asks whether an engine still accepts our flags; this asks
whether it behaves. Flags were the failure we had seen, so they were the
failure we checked for. Non-zero exit, so a rollout can be gated on it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
