llmgw ctl model gpus MODEL N [--profile P] [--in-place],
POST /api/v1/models/{name}/gpus and the console's GPUs…: how many cards of
one node each replica uses. Tensor parallel for vLLM, SGLang and
TensorRT-LLM, held to the model's KV heads (modelplan.SplitsHeads, now one
rule for the planner and this); the layers split for llama.cpp, its old
--tensor-split dropped. Refused for a count no node has, for cards that
would not hold the weights and the context, and for engines on one card.
The replicas start again beside the old ones; with no room beside them the
command says so, and --in-place stops each first.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ctl model context MODEL TOKENS [--slots N], POST /api/v1/models/{name}/context
and the console's Context length…: the catalog, the engine's arguments and
the KV memory change together, refused past the trained context or what the
cards hold, then the replicas restart one by one.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Found running it through the cluster:
- Strata's server starts its engine in a process group of its own, so the
agent saw a GPU replica on the CPU and its 22 GB as somebody else's
memory. A process the shard started, at any remove, is now the shard's.
- An adopted shard's kept compute was sent at once and never again, so a
wrong one read again on adoption never reached the control plane: what
is sent is sent again when it changes, and a GPU shard kept as the CPU
is read again on adoption.
- /metrics/prometheus is llama.cpp's names without the KV cache; vLLM's,
with it, are /metrics?format=prometheus. The router now sees running 1,
waiting 2 under load.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Strata (github.com/Niko1221/Strata) keeps a mixture-of-experts model's
experts in RAM and the ones used most in VRAM; on cuda01's RTX 6000 it
serves Qwen3.8-Flash-Next 125B at 70-87 tokens a second to one client.
- engines/strata.go: the adapter. The agent starts llmgw strata-serve,
which runs Strata's setup the way its desktop app does
(STRATA_CONTROL_INSTALL: installs nothing, writes only under the
replica's and the node's directories) on the fetched GGUF files, points
the configuration it writes at the installed engine, and execs Strata's
server. The key goes in STRATA_API_KEY, never on a command line.
- A profile's args go to setup: --family, --size (setup's --model, which
is the source everywhere else), --context, --parallel, ... Flags that
move files or pick the engine or GPU are refused.
- provision: the source pinned by digest in a Python environment, and the
release's prebuilt engine unpacked beside it, checked against the digest
GitHub publishes: CUDA 13 (7.5+, driver 580) or CUDA 12.9 (6.0-8.9,
driver 575). Python builds can now carry release assets.
- Metrics are vLLM's names under /metrics/prometheus.
- The catalog wants one card and a family, and a directory, not one .gguf.
Opt-in: not in DefaultEngines.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mirror add-peer, run on the mirror's machine, sets up a mirror on
another over ssh, kept in step with this one: llmgw there if it has
none, mirror init --peer, this mirror told of it and allowed to be read
by it, and mirror.peers in the cluster's settings. The setup screens
offer one on either database machine, as server init --second-mirror.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
mirror.peers names the mirrors kept in step with mirror.api. The
mirror's management calls go to the first that can be reached, and
each mirror is asked every fifteen seconds whether it answers: a shard
is sent its model's and adapters' address on one that does while the
mirror the catalog names is down.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Each mirror reads its peers every thirty seconds and copies the
versions a peer holds whole that it lacks, keeping a copy only if its
checksum is the peer's. A removal is kept with its time (.removed.json)
and listed for peers: a version removed while a peer was down is removed
there when it is back, not copied back from it; one added again after
its removal stands on both. Neither mirror leads.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
From the second review.
- An output service reads the answer with the values restore put back
taken out again (guard.Subject.Restored), as an input service reads the
request redacted.
- With n above 1, the service reads the first choice once it is whole,
not at whichever choice finished first.
- Restored values go into an answer asked for as JSON (response_format)
escaped for it, as into tool-call arguments.
- The embedding failure is logged once a minute through an outage, not
once.
- A card's own lane count and fastest generation come from the kernel
(max_link_width, max_link_speed): the driver's width.max is what card
and slot allow together, so an x16 card in an x8 slot read x8 of x8 and
was never said.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Revoking a server's tools takes them from every level beneath too, and a
level left naming no tool stays closed ([]) while any level above or
beneath names tools: an organization's revoke left its teams the copies
their first grants took, and going back to no list opened the level to
what was above it. A closed level is shown on the page and in the
console with an action to open it (store.OpenTools). A key's owner's
list is among the levels above a key.
- The flag check matches short flags as written again (-m, -hf, -mu were
let through), takes a long flag's beginning for it only for the engines
that parse with argparse (vLLM, SGLang, ComfyUI: --li is --listen), so
llama.cpp's own --chat-template is not refused, and refuses llama.cpp's
short spellings (-hfr, -hff, -mm, -jf, …).
- A stored model with an adapter by Hugging Face id can be changed again:
what it already had is not asked about anew.
- The catalog reads a Hugging Face id as the agent does (engines.IsHubID).
- The page's form cannot move llmgw:admin to another address.
- server init skips the memory check only where the gateway already runs,
not wherever a half-finished first run left server.yaml.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From review.
- /profile asked before signing in keeps its query for ten minutes, and
the sign-in (password or OIDC) returns to it: VS Code's sign-in state
came back lost, and the confirmation came every time. Only /profile is
ever returned to.
- The extension keeps an https gateway at https when a link names it by
plain http (a proxy ending TLS unannounced), so the key is never sent in
the clear. The install command says --force, since dev builds share a
version and VS Code kept the old one.
- Every frame of every install recording is checked for a key, a
password or a private key, not only the page.
- The admin API document again matches the API: f94d902's
MCPServerRequest.clear was not in it.
- CHANGELOG for this round's fixes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From review. An adapter given by its id is downloaded by vLLM itself, and
a TensorRT-LLM model by its id by TensorRT-LLM, past the node's check and
not held to safetensors: a model with either is refused when it is added,
unless it says allow_pickle, with how to mirror its safetensors instead.
vLLM's own model by id stays allowed: the agent gives it --load-format
safetensors. A test holds every preset to its own pickle check.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From review.
- A service two policies name is asked the stricter way of the two, both
directions and block over flag, rather than as the first one says.
- A service answering null where it says unsafe gave no answer, not safe.
- With restore: true, the kept answer had the request's redacted values in
it beside the redacted request; they are taken out again before it is
kept.
- guardrails.md says what fail: open lets through, and that of n answers a
service reads the first.
- The similar cache's embedding failures are logged once a minute, not per
request; the page's test reads the gateway's local MCP handler under its
lock; org/name:Q4_K_M is not taken for a Hugging Face id to read as
safetensors.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From review. A server's description, timeout and rate can be emptied from
the console and the page (MCPServerRequest.Clear), where an empty field
used to leave them as they were. An address with a user and password in
it is refused (adminapi.CheckMCPAddress): the credential belongs in a
header, sealed, and the address is shown and audited as written. The
console accepts llmgw:admin and the page lets it be changed, address
fixed. A grant to a key that is not there is refused rather than said
done.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
At the user's asking: we know what a card's link could be, so a card
linked slower is reported. Each agent reads, apart from the GPU query so
that a driver not knowing a field costs nothing else, the fastest PCIe
generation card and slot allow together, the card's own, and the lanes in
use and the card's most (nvidia-smi; NVML in the nvml build). The current
generation is not compared: it drops while a card idles. A card on fewer
lanes, or a slot of an older generation, is a node warning with the GB/s
it carries against what it could — on the dashboard, the console and in
llmgw doctor, which points at lspci's LnkCap and LnkSta. MIG instances are
said of their card. GPU proto fields 23-26.
The PoC's Quadro RTX 6000 reads Gen3 of Gen3, x16 of x16, idle at Gen1:
nothing to say.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From review. A MIG instance is given its share of the card's bandwidth,
not the whole card's. NVLink between a layout's cards means all of them
joined to each other, the largest such set on one ready node, not a count
of cards with any peer: two bridged pairs are not four on NVLink. GH200 is
matched before H200. An FP8 checkpoint is listed for cards of 8.0 and up,
which vLLM runs with Marlin; 8.9 is the agent's bound for --quantization
fp8. NVFP4 repositories tagged fp4 are sized at four bits.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- server init run again on a machine already set up is not refused its
memory (the docs fix a certificate with --host so), and judges count
only when one will be installed (judgeComes: not without a mirror, nor
from a bundle without it).
- A server's sealed secrets that it no longer reads are removed when it is
put or deleted; replacing a credential replaces every header it was
sent, rather than adding one beside the old.
- llmgw.gatewayUrl is a machine setting and restricted in untrusted
workspaces: a cloned repository's settings cannot send a sign-in to
another gateway.
- The bandwidth buffer is at least eight times the CPU's last cache and at
most a quarter of free memory, else nothing is measured until there is
room; a registration carries the figure measured so far.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The similar-question cache embeds a question as its project would: a
project kept in the cluster has external fallbacks refused, and an
embedding model outside the cluster is not asked at all. The vectors are
compared outside the cache's lock. The question's bound is in characters.
- The restorer puts values back in streamed tool calls' arguments (escaped
for the JSON they are), completions' text and reasoning, each held across
events on its own, and releases a held tail with the finish event. A
whole answer is walked as JSON, so a value with a quote no longer breaks
tool-call arguments.
- An outside guardrail service reads the user's turn as redacted, and a
streamed answer once rather than at the finish chunk and again at the
end. guardrails.md says it can stop a stream only under buffer_all.
- A replica's smoothed time to first token is forgotten after five minutes
without an answer: one found slow and then given nothing was never tried
again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From review. A mirror set already on the node is looked at again, not
trusted for having been fetched once; Pickled cuts a query or a fragment
from any source, hf: too; .pt2 is a pickle. The flag check reads a flag as
the engine does: underscores as dashes and any beginning of a refused flag
as that flag, since argparse takes --trust-remote for --trust-remote-code;
SGLang refuses --load-format, which would undo the agent's safetensors.
TripoSR's preset says allow_pickle: without it the preset could not be
added and the presets pass failed on it each time. A shared ComfyUI
environment's hidden entry takes its first member's allow_pickle, as it
takes its source.
The GenAI trace attributes made temperature and top_p typed fields of the
request's parse, so "0.7" as a string was a 400; they are read loosely,
as they are read only for the trace.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A tool is seen where every level above that names tools names it (store
Effective.AllowsTool). The console and /admin/mcp wrote a level's list as
given, so a team that saw its organization's hr.* through having no list
was given jira.* and then saw neither; and revoking a level's last server
left [] in the store, closed to every tool, shown nowhere.
store.GrantTools and RevokeTools now do both screens' work: a level's first
list starts from the nearest list above it; taking a server away leaves the
others, and a level left naming nothing goes back to no list, or stays
closed when a level above names tools, so it opens to nothing it was kept
from. GrantTools says which levels above still keep the granted tools out
of reach, and both screens say so. The page shows closed levels and member
scopes by name, and offers no revoke for a * grant it would not change. The
console calls the store instead of keys limits with a list built from what
it had loaded, which overwrote changes made since. Found in review.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With restore: true and the answer cache on, alice's "write to
alice@example.com" and bob's "write to bob@example.com" redact to the
same request, and bob was given the answer kept for alice, her address put
back in it. A request with restore values is now never looked up in the
cache nor kept in it.
The similar-question cache compared the last message's text and blanked
its parts: two pictures asked about in the same words were one question.
A last message with any part that is not text is now reused only when the
request is the same.
Found in review; both tests fail without the fix.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
/admin/mcp, for administrators only, since a server's credential reaches
whatever it guards: each server's state, who has been granted its tools,
its rate, the headers it is sent by name with the sealed ones marked, and
its address; adding one with a bearer token or another header kept in the
store; testing it with its credential and listing its tools; granting them
to a team, project, org, person or key, written into that level's limits
beside its other grants and its rate and budget; revoking; changing;
replacing the credential; turning off and on; removing; and how a person
points an agent at the gateway's /mcp with a key of their own. English and
Turkish. The secret's field is named so the audit line redacts it.
The admin API's server merge and test are package functions now,
adminapi.MCPServerFrom and adminapi.TestMCPServer, shared by both.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A screen beside Knowledge: each server's state, who has been granted its
tools (read from the store's limits), its rate, the names of the headers
it is sent but never their values, and its address. A adds one, with a
bearer token or another header sent once and kept sealed. On a row: test
it and list its tools; grant them to a team, project, org, person or key,
merged with what else that level is granted, since keys limits --tools
writes the list whole; revoke; change; replace the credential; turn off or
on; remove. The screen also says how an agent is pointed at /mcp with a
person's own key.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Seven VMs on one host restarted at once read a third less each than any
gets alone. An agent now waits 10 s to 2 min, at random, before measuring,
and measures again every hour, keeping the best.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
For each layout modelplan.Options offers here — one card, tensor parallel
across a node's cards, llama.cpp's layer split, a pipeline across nodes on
a fabric — modelplan.GPUTokensPerSecond gives one reply's speed: the bytes
a token reads over each card's bandwidth, by its model name (a table, SXM
and PCIe apart), plus two all-reduces a layer for tensor parallel (8 us on
NVLink, 25 us and up on PCIe), turns between cards for a layer split, and
a hop per node over InfiniBand or RoCE for a pipeline. The explorer and a
repository's files show the fastest, with the layout; mixed classes are
judged class by class, FP8 on 8.9 cards only.
Calibrated on cuda01's Quadro RTX 6000 (672 GB/s): qwen3-vl-8b Q8, 8.7 GB,
62.1 tok/s measured, 59 estimated; Qwen3-Coder-30B-A3B Q4_K_M, 18.6 GB,
144 measured, which says a model of experts reads 1.6 times its active
share (ActiveShare now does, on the CPU too). Tensor parallel and fabrics
are not measured here.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Measuring it in the startup scan held every agent half a second or more
before registering, which on a shared CI runner was past the failover
tests' wait. Now it runs once a process, in the background, the first time
it is asked for, and Heartbeat.mem_bandwidth_gbs carries it when known;
the control plane keeps the last figure and a heartbeat without one leaves
it. The agent's tests do not measure.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each agent measures its node's memory read rate when it starts: every CPU
sums a share of a buffer past any cache, a tenth of free memory at most,
the best of four passes, in a fraction of a second; reported in Host
(mem_bandwidth_gbs) and NodeView. A token on a CPU reads every weight it
uses once, so one reply's speed is about that rate over those bytes:
modelplan.CPUTokensPerSecond, 0.68 of the measured rate plus 5 ms a token,
calibrated on the PoC against llama.cpp's predicted_per_second (node01
24.9 GB/s, qwen2.5-1.5b: 14.5 measured, 14.7 estimated; node05 41.9 GB/s,
qwen2.5-14b: 3.3 and 3.2). A model of experts named …-A3B reads its active
part (ActiveShare).
The explorer shows it as CPU tok/s, at Q4_K_M on the fastest node that
holds the model; a repository's page gives it for each GGUF by its size.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Judged by the nodes as they are now, through modelplan.Options, the
planner that serving a model uses: GGUF at Q4_K_M from the parameter count,
safetensors at the size the registry's dtype breakdown gives (packed AWQ,
GPTQ and MXFP4 at four bits), FP8 only where a card has compute capability
8.9, MLX safetensors never. A picture model is judged against one card. A
cluster with cards asks the registry for safetensors too, a hundred at a
time; one without asks for GGUF only. The page says what it judged by: the
largest card, its capability, the most RAM on one node.
On a repository's page a file set no node holds is marked with what it
needs, against what the best node offers, and has no Fetch button.
Tried against the PoC's seven nodes and Hugging Face's trending hundred:
63 shown, the rest too big for one 24 GB Turing card or FP8.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A repository with neither GGUF nor safetensors (by the registry's summary
or its tags) is hidden under "runs here", which is ticked by default, and
counted with the architectures llama.cpp cannot load; unticked it is
listed. An open repository's pickle files are not offered, only counted,
and a repository of pickles only says to look for another version.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Python pickles (.bin, .pt, .pth, .ckpt, .pkl) run whatever they hold when
loaded, as the engine's user, on the node; that is how the malicious models
found on Hugging Face work. The node refuses such a source, or a mirror set
holding one, before fetching anything. A model added with one, or given
one, is refused, while one already in the catalog still loads and is
refused by its node with the same words. vLLM and SGLang downloading a
model by its id get --load-format safetensors. The mirror page marks the
file. allow_pickle: true on a model lets it through on purpose, carried to
the node in EngineProfile.allow_pickle.
BACKLOG M lists the rest: checksums against Hugging Face on every path,
its own scan, trusted publishers, engines without the internet, an eval
before an untrusted model serves.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
llmgw keys cache --similar 0.95 also gives a chat question the answer to
one like it: the last user message embedded by router.cache.embedding_model
(asked on the gateway's account, as a judge is, within 2 s), compared by
cosine with the questions answered in the same conversation — system
prompt, earlier turns, tools and sampling the same. The exact cache is
asked first; an exact hit embeds nothing. Said as x-llmgw-cache: similar
with x-llmgw-cache-similarity, charged as a hit.
The vectors live with their answers in the gateway's memory, counted in
max_bytes and evicted with them, not in a pgvector table as BACKLOG I1
planned: the cache keeps answer text out of the store on purpose. A
question over 2,000 characters is reused only when the same. Migration
0051 holds the setting. BACKLOG I1.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cached_per_1k in a price charges the prompt tokens the answer counts as
cached_tokens at their own rate; zero keeps charging them as prompt tokens.
The count reaches the cost and not a column: CostMicros holds what it
changed. Cached beyond the prompt is clamped, an engine's miscount rather
than a refund. BACKLOG L5.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The router keeps each endpoint's time to first token, smoothed over its last
streamed answers, and the chat profile scores by it beside session, queue,
KV and inflight: the fastest endpoint 1, one twice as slow 0.5, one not yet
measured 1 so a new replica is tried. Load still sees a slow drain from the
engine's own counters; this sees what answers actually took. BACKLOG L4.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docs/kong.md: a plain route or ai-proxy with llmgw as the upstream, the
client's llmgw key passed on so per-team budgets hold, Kong as a provider
behind llmgw, and Kong 3.9's streamed answers counted without completion
tokens. Closes BACKLOG L9.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A site may be required to use a particular guardrail: Azure Content Safety,
Lakera, NeMo, a classifier of its own. A policy's services asks one over
HTTP — a body template given the text, header values from files read each
call, dotted paths to where the answer says unsafe and why — on input,
output or both, to block or flag. On input it reads the latest user turn
before the judge; on output the whole answer once at its end. A service
that does not answer is what the policy's fail says. It is told the text
only, never who sent it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With restore: true on a guardrail policy's input, what redact rewrote as
placeholders comes back in the answer: the model reads "[EMAIL_1]", the
client reads the address it sent. A streamed placeholder split across
events is held until it is whole, and what is held is sent at the end.
The swap runs before the output filter, so the output rules judge the
answer as the client reads it and a placeholder they make of a value the
model invented is never taken for the request's own. The map lives in the
Verdict for the request only; nothing stores, logs or traces it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A control plane that is also a node, serving the starter model and two
judges, needs about 10.5 GB: 3.5 for itself, 2 for the starter, 2 for each
judge, and one more because the scheduler counts whole gigabytes. On 8 GB
the install did every step and then had nowhere to run the starter. The
setup's checks now fail this machine's memory and name what it would serve,
and server init refuses before it installs anything, naming the flags that
leave those models off it. quick-install.md said 8 GB; it says 12.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An inference request and each attempt at an engine carry gen_ai.*: the
operation, the requested and answering model, max_tokens, temperature and
top_p when the client set them, the session as the conversation id, the
input, output and cached tokens, and on an attempt the engine that answered.
A customer's Grafana, Jaeger or Langfuse then shows llmgw's calls as model
calls. No prompt or answer text is ever put in a span, and a test says so.
BACKLOG gains section L: what is worth taking from Kong's AI Gateway, of
which this is the first.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
llmgw's own extension for VS Code, plain JavaScript with no dependencies,
packed into a .vsix by the binary at /setup/vscode/llmgw.vsix with the
serving gateway's address in it, so it needs no build, no Node and no
marketplace. It registers llmgw as a language-model provider: the chat's
model picker, agent mode and any extension that asks VS Code for a model
get every chat model the key may use, sized and with tool and vision
capabilities as the catalog says, streamed, tool calls both ways.
VS Code is a tool on the account page's Connect your editor: a key, the
install command, and an Open in VS Code link that hands the key to the
extension's secret storage. The extension's Sign In opens the page with a
state of its own; a link without it is confirmed, the gateway named, since
it decides where code is sent.
Tested in VS Code 1.138 against the PoC: 27 models listed, an answer from
qwen2.5-0.5b-instruct, and qwen3-coder-30b calling a tool and using its
result (internal/vscodeext/e2e).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
/install takes a reader from three bare Ubuntu machines to a first answer:
what each machine needs and the commands that check it, the release fetched
and its checksum checked, the eight setup screens, the system's state and
the console, and a first request with a key from the account page. Every
screen is from an install of v0.14.9 on three fresh VMs, recorded with tmux
and kept as terminal frames: the page shows them as text, and plays a
session back when asked, fetching it compressed from /install/cast/. The
administrator's password and the API key were masked before the recording
was kept, and a test fails if either shape reaches the page.
/about gains what llmgw now does beyond routing, and its install steps are
today's: the setup screens and node add, not onboard-node.sh and a model
file, with a link into the guide.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The errors other packages give a page (store, catalog, mirror) are
shown in its language; a percentage is written as the language writes
it (%93 in Turkish); the footer's time in its date form; and a stale
form is refused with a sentence rather than "bad request".
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A tool's answer is read by the caller's policies as the prompt it
becomes, so an instruction planted in a ticket is withheld and personal
data redacted; its arguments are read before they leave. ctl mcp add
--rate bounds each caller's calls a minute to a server (migration 0050),
and five failures in a row take a server out for a minute, its tools
off the list.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The same text again was answered from what the engine kept of the last
reading, in a tenth of the time: the check read 4.9 s, then 0.5 and 0.5.
Each judgment now begins with a line of its own.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ctl guard judge check, and POST /api/v1/guard/judges/check, time three
judgments of a 280-token prompt, then a burst at once, against the
policies' wait: ok, tight or slow, about how many requests a second can
be judged, and what to do. server init runs it once the judge is up and
says a machine too slow before the next steps; the wait is not changed
without asking. BACKLOG reordered by what goes wrong where installed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Setting one limit at a scope wrote the whole row, so a tool grant given
alone cleared the scope's rate, budget and models. --replace writes it
whole, as the console's form, which shows every field, now asks for.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
