Correct what the status section says is missing
CI / test (push) Successful in 1m39s

Two of the four items were already built. A model falls through to its
fallbacks on no capacity as well as on a spent budget, and the per-request
routing profile header exists — it is ignored under authentication on purpose,
because a header that picks a routing profile is a header that promotes a
client into the top priority band.

Checked against the code rather than against memory, which is how both got
onto the list in the first place.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5 committed 2026-09-18 13:49:53 +02:00
1 parent 21a502035f
commit cf20570ff1
1 file changed
+8 -2
+8 -2
View File
@@ -37,8 +37,14 @@ control plane places replicas — including ones larger than a single machine
models load when someone asks for them and stop when nobody does, and keys,
budgets and rate limits decide who may use any of it.
Before v1.0.0: routing profiles selectable per request, spillover to a fallback
pool, tracing, and reference documentation. One thing is worth saying plainly:
Before v1.0.0: tracing, reference documentation, and filtering on `ctl nodes`
and `ctl replicas` for fleets too large to read a whole table of. Two items
that were on this list are not gaps: a model falls through to its fallbacks
when there is no capacity as well as when a budget is spent, and routing
profiles are chosen by the key rather than by a request header on purpose —
a header would let any client promote itself into the top priority band.
One thing is worth saying plainly:
multi-node replicas are proven on a simulated cluster and on real hardware
without a fabric. The InfiniBand, NVML and NCCL paths are written against the
formats and failure modes those tools actually have, but they have not yet run