The accounting layer has always exempted a model that costs nothing, with a
comment saying it is "what lets a key with no budget left still be served a
cheap fallback". That described a caller nobody had written: the budget was
checked once, against the requested model, and a refusal was the end of it,
so the exemption could never fire.
A caller that has run out is now offered the first model in its fallbacks it
can still afford. The allowlist still applies — running out of money is not
a way to reach a model a key was never allowed — and a model with no
fallback is refused exactly as before, or a budget would stop meaning
anything. Only a spent budget falls back: a rate limit is answered by
waiting a second, and moving the request elsewhere would spend someone
else's capacity to avoid a delay nobody asked to avoid.
Which uncovered a header that has never worked. x-llmgw-served-model was set
in finish, and finish runs after the proxy has written the response — so the
one mechanism for telling a client that something else answered has been
announcing it to nobody since it was written. Nothing failed, because the
only thing asserting it was a comment. It is now set before dispatch, for
the capacity fallback as well as this one.
Finding that took bisecting a test I had written wrong: it polled for a 429
that never came, because the budget had run out and the fallback had served
the request — the feature working, reported as the feature failing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
