slurm.containers (ctl slurm containers on) has every Slurm machine's agent
install enroot from NVIDIA's release and build pyxis against its own
Slurm, both pinned and checked against their digests, and on GPU machines
nvidia-container-cli from NVIDIA's repository. enroot keeps each user's
containers under /run/enroot and /var/lib/enroot, needing no login
session, with the shared path in every container. The leader has Slurm
load pyxis only once every machine reports it: plugstack.conf's
"required" fails every job on a machine without the plugin.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tried on the PoC: lending cuda01, the one GPU node, its model had nowhere
else to go and waited out the drain's five-minute grace, while ctl slurm
status said only "1 replica(s) still to move off". It now says the time
that replica is stopped at. Also recorded: Slurm on a GPU node, tried on
cuda01 (gres, ConstrainDevices, accounting of gres/gpu).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
llmgw's Slurm users log in to the controller freely and to another Slurm
machine only while a job of theirs runs there: the agent installs
pam_slurm_adopt and puts it in sshd's account stack for the users it made
alone, so root and the machine's own users are never locked out. Their
session is put in the job's extern step (PrologFlags=Contain), and
pam_systemd, which would move it out, is skipped for them by taking the
session file into sshd's stack with a guard in front of it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Setup asks for webhooks, but one left empty was never mentioned again: a
lost node or a judge down was told nobody. With no notify.webhooks in the
cluster's settings, ctl status and the console's Overview say so, and how
to set one. BACKLOG: the judge's refusal after a placement change, tried
again on idle VMs, did not happen; a cold judge's first answer is 2-3x
slower but inside its budget.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The message said their API keys still worked and should be revoked; the
store has refused a key whose owner is not active for a while. Tried on
the test cluster: refused 26 s after the suspend (auth.cache_ttl, 30 s),
answered again 6 s after reinstating.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
slurm.shared mounts the site's NFS export, has an llmgw node export the path
over NFS to the Slurm machines alone (2049 in the firewall), or looks for a
file system the site mounts. slurm.users "llmgw" makes each active account a
Linux user on every Slurm machine: login from its e-mail, a UID from 200000
kept in slurm_users and never reused, its home on the shared path, its SSH
keys from the portal's Profile (or ctl slurm users keys) read by sshd from
/etc/llmgw/ssh-keys. server init --slurm chooses /shared from the first
machine and llmgw's users; --slurm-shared and --slurm-users change that.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
node add (and server add, mirror peer, patroni init) is usually given an
address, while the ssh config names the machine: `Host gpu01` with
`HostName 10.0.0.21` and the User, IdentityFile or ProxyJump it needs. That
block did not match the address, so the address had to be added to its
Host line by hand. Now the concrete Host names in the config and its
Includes are read, ssh -G says each one's HostName, and the first that is
the address (or resolves to it) is connected through.
Under sudo, the caller's name is now a fallback: a User in the config's
block for the machine comes first.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- server init printed every engine's line again on each look while the
node installed (some 280 lines); each engine's line now once, and again
only when it changes, as node add does.
- A package install that is no engine's (Slurm's, MariaDB's) read
": installing installing mariadb-server, ..."; it is "system
packages: installing: mariadb-server, ..., with apt-get".
- The slow-judge advice says "ctl model resources JUDGE --cores 4",
which was refused for want of --memory: cores alone now keep the
memory as it is.
- server add said both control planes for three.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Since 23.11 Slurm runs badly without accounting storage: the main
scheduler passes every job by as InvalidAccount. slurmdbd takes MySQL or
MariaDB only, so the controller's agent installs MariaDB beside it,
answering that machine alone, makes the database and its user (the
password made from the munge key, never on a command line), runs
slurmdbd and adds the cluster, all before slurmctld. A MariaDB that was
on the machine before keeps its settings. slurm.conf accounts the jobs
there, with what each used from its cgroup and the cards, and the
priority is multifactor again. Port 6819 joins Slurm's in the firewall.
ctl slurm status and the console say whether slurmdbd runs; the install
screen, server init --slurm and the guide say what is put where.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
priority/basic did not end InvalidAccount: 23.11 dropped
accounting_storage/none, and without slurmdbd the main scheduler passes
jobs by whatever the priority plugin. Basic stays, fair share needing
slurmdbd anyway.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Without accounting storage, 23.11's default priority/multifactor has the
main scheduler pass every job by as InvalidAccount; backfill started it a
pass later. slurm.conf sets priority/basic.
Every node was written State=DRAIN, so each start of slurmctld on new
files drained a lent node again until its agent resumed it. A lent node
is written without it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tried on the PoC: five one-second jobs on an 8-CPU node started 10 s
apart. Each asked for no memory, so each was given the node's all, ran
alone, and the next waited for backfill's pass. Not accounting, as
thought. The partition's DefMemPerCPU is now the leanest node's memory
per CPU.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The console's Slurm section shows the controller, and each node that runs
slurmd with whose it is now, its state in Slurm and its jobs. Enter on a
row lends the node (its replicas move off first) or takes it back, by the
same change to slurm.lent as ctl slurm lend and reclaim.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On a cluster upgraded to the kept nodes, the first leader finds none and
wrote slurm.conf without the nodes it had not heard from yet, as before.
A node neither reported nor kept now holds the controller's files back
for slurmsvc.HoldFor (2 minutes) from the leader's first pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tried on the PoC: with a job on a lent node, the leader restarted and its
first slurm.conf had no node in it (none had reported to it yet), so
slurmctld dropped the node and the job ended in NODE_FAIL.
The leader keeps slurm.conf's nodes and the controller's address in the
store's cluster state (slurm-nodes); the next one starts from them and
writes the same files until the nodes report again. With the store
unread, the controller gets no files that leave a node out. A node is
told nothing, rather than that it has no part, until its controller's
address is known: no part stops slurmd, and its jobs with it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
slurm.conf sets SrunPortRange=60001-63000, so a site's firewall can let
srun's step connections in. A node given a part in Slurm adds 6817-6818
and that range to llmgw's nftables table, open to the cluster's networks
and closed to everyone else, as the engines' ports are.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The choices screen has a Slurm part, and --slurm writes slurmctld on this
machine's node and slurmd on every node, those that join later too, into
server.yaml. slurm.all_nodes (ctl slurm setup --all-nodes) takes the nodes
from the leader's inventory. No node is lent at install.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Found on the PoC: InstallPackages reported 'installing munge, slurm-client,
slurmd' under no engine and never cleared it, so the node's page kept
saying so (infiniband-diags and perftest did the same). slurmd started
before the leader has written its node into slurm.conf is refused; the
drop-in now has systemd try it again every 15 s.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The InvalidAccount seen on the PoC was not the reconfigure: 23.11.4 without
accounting storage has its main scheduler mark a batch job so, and backfill
starts it on its next pass (job 4 waited 15 s). bf_interval=10 keeps that
short; the extra restart is taken out again. MailProg=/bin/true quiets the
warning on every start.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Found on the PoC: Slurm 23.11.4 without accounting storage left batch jobs
pending InvalidAccount after scontrol reconfigure until slurmctld was
restarted. And jobacct_gather with nowhere to keep it logs an error per job.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Found on the PoC (Ubuntu 24.04, Slurm 23.11.4): the packaged unit is
Type=notify, so slurmd needs --systemd or systemd times it out; and slurmd
keeps a 'slurmstepd infinity' for its cgroup v2 scope, which was counted as
a job and kept the node from llmgw.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Found on the PoC: scontrol looks for its files even for --version, and a
configless node has none before slurmd has fetched them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
internal/control/slurmsvc tells each node what it is, writes slurm.conf
and gres.conf from the nodes' reports and the inventory, and moves each node
of slurm.nodes between llmgw and Slurm: lent, drained of replicas first, then
resumed in Slurm; taken back, drained in Slurm and cordoned in llmgw until
its last job step ends. Kept every tick from the settings and the agents'
reports, so a new leader carries on. A broken Slurm never keeps a node from
llmgw. GET /api/v1/slurm; ctl slurm status, setup, lend, reclaim, off.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Slurm message says whether the node runs slurmctld on the leader's files,
slurmd configless, both or neither, and whether it is lent. The agent
installs munge and Slurm from the machine's sources (22.05 or newer, for
cgroup v2), writes the munge key, starts slurmd through a drop-in with
--conf-server, resumes a lent node that llmgw drained and drains one that is
not, never touching a drain a Slurm administrator made, and reports its
version, slurmd -C, state and jobs in SlurmStatus. Told it has no part, it
stops what it started. New nodes enter slurm.conf drained.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
slurm.{enabled,cluster_name,controller,nodes,lent,partition} in the cluster's
settings, checked; internal/slurm writes slurm.conf (configless), gres.conf
and cgroup.conf from the nodes' slurmd -C lines and GPUs, derives the munge
key from a cluster secret, and reads scontrol and version output.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The same inputs place a replica on the same node and card, so an on-demand
model that failed to load there was started there again after every
backoff until it was parked. The reconciler remembers where a model's
replicas failed and places the next away from them when anything else
fits (Scheduler.Away); with nowhere else, there again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Registration carries what the agent's startup scan found, and a heartbeat
sends the list only when it changed since last sent; so a file deleted
after the agent started came back on a control plane that restarted. The
fingerprint is now forgotten with the provisioning one on every stream.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The mirror page's form began every GGUF model at 4096 tokens, the
console's at 4096 or the plan's figure. Both now start at what the model
was trained for, cut to the most whose KV cache fits beside the weights
where the form places it; the mirror page's slots are as many of that, up
to four, as the room left holds. Plan carries the trained context.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A set is fetched into a directory named from its address without the
trailing slash; Orphans named it from the catalog's address with it, so
every set in the catalog was offered for deletion: 330 GB of six models on
the PoC. One function, cacheSource, now gives both the name.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
C1. A membership of an organization may be admin as well as member and
manager. An admin sees /org as a manager does and changes the
organization from it: its teams and projects, who of the approved
accounts is in them, its members' roles, and the keys spent in its
projects. Every id a form names is checked against the organization;
limits, accounts, the organization's name and the cluster stay with the
cluster's staff, and an admin cannot take their own role away.
A group from OIDC, SCIM or a directory no longer rewrites a membership
it names as a plain member at every sign-in, which took a manager's role
away (EnsureMembership).
Pages draw a form only for someone who may send it: Nav.May asks the
staff rule staffMay keeps, and a page whose forms a person may not send
gets no form token. A viewer sees the people, limits and mirror pages
without forms; an operator the mirror's.
Forms are compact: labels close to their fields, fields as wide as what
goes in them, smaller buttons, and the short forms on one line.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Four slides of products and cards become one table: llmgw beside API
gateways (LiteLLM, Kong AI Gateway, Agent Router) and serving stacks
(llm-d, NVIDIA Dynamo, vLLM production-stack), six rows. The PDF is
18 slides.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docs/llmgw-intro.pdf is a 21-slide presentation: what llmgw adds on
premise, screens from a running cluster, and how it compares with
LiteLLM, Kong AI Gateway, Agent Router, llm-d, NVIDIA Dynamo and vLLM
production-stack from their own documentation. The /about page and the
published site link to it in the repository, so the binary does not
carry it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A sweep counted every file the nodes were asked to delete as freed as soon
as it asked, and a node refused a set's directory as not empty, so tens of
gigabytes were reported freed and nothing was. The agent removes a set's
directory whole (a link only as a link), and the control plane waits for
each node to list its files again, saying what it no longer lists and what
it kept.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The row of a served model leads to its card, which now changes its context
and longest answer, its cards, its replicas on each profile with a profile
on CPUs to add and its speed beside, stops, starts and restarts it, and
removes it with its files or without. Stopped on the row stops the model
rather than setting its scaling to nothing. The console gains Replicas per
profile, and ctl models shows the counts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A profile's replicas, set on any of a model's profiles, is how many of its
replicas run there; the scaling becomes the sum. The reconciler starts the
ones short of their profile before stopping the ones over, so a change of
split never runs fewer. ctl model replicas sets them, adding a profile on
CPUs to a GGUF model; the router already fills the higher-priority pool
first, which the CPU profile is put below.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each agent measures one core's memory rate beside the whole machine's; the
estimate takes a replica's threads at about one core's rate each, up to the
machine's, calibrated by llama-bench on the PoC. The serve form shows it
beside a card's, warns when slow, and asks for the CPU on purpose when a
reply would take minutes; ctl model plan, the console and serve jobs say
it too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Single files were shared between the shards and replicas asking for them
at the same time; sets were not, and four replicas on one node's four cards
wrote the same .part files over one another until all but one failed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Each router's per-counter request and token counts, this minute's and the
last, ride the standby's two-second exchange with the leader. The leader
answers each standby with everyone else's sum, its own included, and puts
the standbys' sum in force on its own router; each limiter adds the
others' counts to its sliding window.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A standby follows the leader's Ready replicas over the agent port, every
two seconds, and reports back the replicas its router used and the
on-demand models it was asked for. No engine keys travel: every control
plane derives them from the cluster secret. Budgets are raised to the
store's every 30 s. Unheard from the leader for a minute, a standby
passes /v1 on as before. Off by default.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
max_output on POST /api/v1/models/{name}/context, --max-output on ctl model
context, and a field in the console's Context length… form. Alone it changes
only the catalog's limit, which the router holds max_tokens to, and starts no
replica again; with a context it is set with it, never past it.
Strata's context is now changed like the others', through its setup's
--context, which writes the engine's start script at each start. Neither
needs a model removed and added again any more.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FreeToken (FlashML-org/FreeToken) serves mixture-of-experts models several
times a card's size from one card, experts in RAM, from Hugging Face
safetensors checkpoints. Installed like vLLM, from PyPI by uv, on every node
it runs on: compute capability 8.0+, driver 580+, x86-64, and the CUDA 13
toolkit it compiles kernels with. Older cards are passed over: on cuda01's
Quadro RTX 6000 (7.5) 0.2.1 had no kernel image for the first prefill, and
in fp16 hit an illegal memory access on its first request.
ft serve gets the model, address, card and served name; the key goes in
FREETOKEN_API_KEY. Flags that read files or fetch are refused, as are
FREETOKEN_ variables naming paths or the agent's own settings. ctl model
context sets --max-seq-len-override.
Readiness now also waits for an engine whose /health answers 200 while its
body says "loading", as FreeToken's does from the moment its HTTP server is up.
FreeToken is followed release by release: a weekly workflow fails when PyPI
has a version newer than DefaultEngines installs (scripts/engine_releases.py).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The busy figure is nvidia-smi's utilization.gpu: the share of time any
kernel ran, which a kernel on one SM fills. On cuda01 decode read 92% busy
while the SMs had work 69% of the time and issued on 18% of their cycles.
Busy is now said to be that, and beside it: each card's draw against its
power limit (efficiency table, fleet and node charts, the console's GPU
work chart), and where the profiling counters can be read, SMs active, SM
occupancy, tensor cores and DRAM (llmgw_gpu_*_active_percent), from NVML's
GPM on Hopper and later or a dcgm-exporter on the node
(discovery.dcgm_exporter). The GPU map colours by work by default and keeps
busy as a mode. A MIG instance is never published with its card's profile.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
llmgw ctl model gpus MODEL N [--profile P] [--in-place],
POST /api/v1/models/{name}/gpus and the console's GPUs…: how many cards of
one node each replica uses. Tensor parallel for vLLM, SGLang and
TensorRT-LLM, held to the model's KV heads (modelplan.SplitsHeads, now one
rule for the planner and this); the layers split for llama.cpp, its old
--tensor-split dropped. Refused for a count no node has, for cards that
would not hold the weights and the context, and for engines on one card.
The replicas start again beside the old ones; with no room beside them the
command says so, and --in-place stops each first.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ctl model context MODEL TOKENS [--slots N], POST /api/v1/models/{name}/context
and the console's Context length…: the catalog, the engine's arguments and
the KV memory change together, refused past the trained context or what the
cards hold, then the replicas restart one by one.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Found running it through the cluster:
- Strata's server starts its engine in a process group of its own, so the
agent saw a GPU replica on the CPU and its 22 GB as somebody else's
memory. A process the shard started, at any remove, is now the shard's.
- An adopted shard's kept compute was sent at once and never again, so a
wrong one read again on adoption never reached the control plane: what
is sent is sent again when it changes, and a GPU shard kept as the CPU
is read again on adoption.
- /metrics/prometheus is llama.cpp's names without the KV cache; vLLM's,
with it, are /metrics?format=prometheus. The router now sees running 1,
waiting 2 under load.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Strata (github.com/Niko1221/Strata) keeps a mixture-of-experts model's
experts in RAM and the ones used most in VRAM; on cuda01's RTX 6000 it
serves Qwen3.8-Flash-Next 125B at 70-87 tokens a second to one client.
- engines/strata.go: the adapter. The agent starts llmgw strata-serve,
which runs Strata's setup the way its desktop app does
(STRATA_CONTROL_INSTALL: installs nothing, writes only under the
replica's and the node's directories) on the fetched GGUF files, points
the configuration it writes at the installed engine, and execs Strata's
server. The key goes in STRATA_API_KEY, never on a command line.
- A profile's args go to setup: --family, --size (setup's --model, which
is the source everywhere else), --context, --parallel, ... Flags that
move files or pick the engine or GPU are refused.
- provision: the source pinned by digest in a Python environment, and the
release's prebuilt engine unpacked beside it, checked against the digest
GitHub publishes: CUDA 13 (7.5+, driver 580) or CUDA 12.9 (6.0-8.9,
driver 575). Python builds can now carry release assets.
- Metrics are vLLM's names under /metrics/prometheus.
- The catalog wants one card and a family, and a directory, not one .gguf.
Opt-in: not in DefaultEngines.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mirror add-peer, run on the mirror's machine, sets up a mirror on
another over ssh, kept in step with this one: llmgw there if it has
none, mirror init --peer, this mirror told of it and allowed to be read
by it, and mirror.peers in the cluster's settings. The setup screens
offer one on either database machine, as server init --second-mirror.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
mirror.peers names the mirrors kept in step with mirror.api. The
mirror's management calls go to the first that can be reached, and
each mirror is asked every fifteen seconds whether it answers: a shard
is sent its model's and adapters' address on one that does while the
mirror the catalog names is down.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Each mirror reads its peers every thirty seconds and copies the
versions a peer holds whole that it lacks, keeping a copy only if its
checksum is the peer's. A removal is kept with its time (.removed.json)
and listed for peers: a version removed while a peer was down is removed
there when it is back, not copied back from it; one added again after
its removal stands on both. Neither mirror leads.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
From the second review.
- An output service reads the answer with the values restore put back
taken out again (guard.Subject.Restored), as an input service reads the
request redacted.
- With n above 1, the service reads the first choice once it is whole,
not at whichever choice finished first.
- Restored values go into an answer asked for as JSON (response_format)
escaped for it, as into tool-call arguments.
- The embedding failure is logged once a minute through an outage, not
once.
- A card's own lane count and fastest generation come from the kernel
(max_link_width, max_link_speed): the driver's width.max is what card
and slot allow together, so an x16 card in an x8 slot read x8 of x8 and
was never said.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Revoking a server's tools takes them from every level beneath too, and a
level left naming no tool stays closed ([]) while any level above or
beneath names tools: an organization's revoke left its teams the copies
their first grants took, and going back to no list opened the level to
what was above it. A closed level is shown on the page and in the
console with an action to open it (store.OpenTools). A key's owner's
list is among the levels above a key.
- The flag check matches short flags as written again (-m, -hf, -mu were
let through), takes a long flag's beginning for it only for the engines
that parse with argparse (vLLM, SGLang, ComfyUI: --li is --listen), so
llama.cpp's own --chat-template is not refused, and refuses llama.cpp's
short spellings (-hfr, -hff, -mm, -jf, …).
- A stored model with an adapter by Hugging Face id can be changed again:
what it already had is not asked about anew.
- The catalog reads a Hugging Face id as the agent does (engines.IsHubID).
- The page's form cannot move llmgw:admin to another address.
- server init skips the memory check only where the gateway already runs,
not wherever a half-finished first run left server.yaml.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From review.
- /profile asked before signing in keeps its query for ten minutes, and
the sign-in (password or OIDC) returns to it: VS Code's sign-in state
came back lost, and the confirmation came every time. Only /profile is
ever returned to.
- The extension keeps an https gateway at https when a link names it by
plain http (a proxy ending TLS unannounced), so the key is never sent in
the clear. The install command says --force, since dev builds share a
version and VS Code kept the old one.
- Every frame of every install recording is checked for a key, a
password or a private key, not only the page.
- The admin API document again matches the API: f94d902's
MCPServerRequest.clear was not in it.
- CHANGELOG for this round's fixes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From review. An adapter given by its id is downloaded by vLLM itself, and
a TensorRT-LLM model by its id by TensorRT-LLM, past the node's check and
not held to safetensors: a model with either is refused when it is added,
unless it says allow_pickle, with how to mirror its safetensors instead.
vLLM's own model by id stays allowed: the agent gives it --load-format
safetensors. A test holds every preset to its own pickle check.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From review.
- A service two policies name is asked the stricter way of the two, both
directions and block over flag, rather than as the first one says.
- A service answering null where it says unsafe gave no answer, not safe.
- With restore: true, the kept answer had the request's redacted values in
it beside the redacted request; they are taken out again before it is
kept.
- guardrails.md says what fail: open lets through, and that of n answers a
service reads the first.
- The similar cache's embedding failures are logged once a minute, not per
request; the page's test reads the gateway's local MCP handler under its
lock; org/name:Q4_K_M is not taken for a Hugging Face id to read as
safetensors.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From review. A server's description, timeout and rate can be emptied from
the console and the page (MCPServerRequest.Clear), where an empty field
used to leave them as they were. An address with a user and password in
it is refused (adminapi.CheckMCPAddress): the credential belongs in a
header, sealed, and the address is shown and audited as written. The
console accepts llmgw:admin and the page lets it be changed, address
fixed. A grant to a key that is not there is refused rather than said
done.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
At the user's asking: we know what a card's link could be, so a card
linked slower is reported. Each agent reads, apart from the GPU query so
that a driver not knowing a field costs nothing else, the fastest PCIe
generation card and slot allow together, the card's own, and the lanes in
use and the card's most (nvidia-smi; NVML in the nvml build). The current
generation is not compared: it drops while a card idles. A card on fewer
lanes, or a slot of an older generation, is a node warning with the GB/s
it carries against what it could — on the dashboard, the console and in
llmgw doctor, which points at lspci's LnkCap and LnkSta. MIG instances are
said of their card. GPU proto fields 23-26.
The PoC's Quadro RTX 6000 reads Gen3 of Gen3, x16 of x16, idle at Gen1:
nothing to say.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
