Correct what the status section says is missing
CI / test (push) Successful in 1m39s

Two of the four items were already built. A model falls through to its
fallbacks on no capacity as well as on a spent budget, and the per-request
routing profile header exists — it is ignored under authentication on purpose,
because a header that picks a routing profile is a header that promotes a
client into the top priority band.

Checked against the code rather than against memory, which is how both got
onto the list in the first place.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5 committed 2026-09-18 13:49:53 +02:00
1 parent 21a502035f
commit cf20570ff1
1 file changed
+8 -2
+8 -2
View File
@@ -37,8 +37,14 @@ control plane places replicas — including ones larger than a single machine
models load when someone asks for them and stop when nobody does, and keys, models load when someone asks for them and stop when nobody does, and keys,
budgets and rate limits decide who may use any of it. budgets and rate limits decide who may use any of it.
Before v1.0.0: routing profiles selectable per request, spillover to a fallback Before v1.0.0: tracing, reference documentation, and filtering on `ctl nodes`
pool, tracing, and reference documentation. One thing is worth saying plainly: and `ctl replicas` for fleets too large to read a whole table of. Two items
that were on this list are not gaps: a model falls through to its fallbacks
when there is no capacity as well as when a budget is spent, and routing
profiles are chosen by the key rather than by a request header on purpose —
a header would let any client promote itself into the top priority band.
One thing is worth saying plainly:
multi-node replicas are proven on a simulated cluster and on real hardware multi-node replicas are proven on a simulated cluster and on real hardware
without a fabric. The InfiniBand, NVML and NCCL paths are written against the without a fabric. The InfiniBand, NVML and NCCL paths are written against the
formats and failure modes those tools actually have, but they have not yet run formats and failure modes those tools actually have, but they have not yet run