Python pickles (.bin, .pt, .pth, .ckpt, .pkl) run whatever they hold when
loaded, as the engine's user, on the node; that is how the malicious models
found on Hugging Face work. The node refuses such a source, or a mirror set
holding one, before fetching anything. A model added with one, or given
one, is refused, while one already in the catalog still loads and is
refused by its node with the same words. vLLM and SGLang downloading a
model by its id get --load-format safetensors. The mirror page marks the
file. allow_pickle: true on a model lets it through on purpose, carried to
the node in EngineProfile.allow_pickle.
BACKLOG M lists the rest: checksums against Hugging Face on every path,
its own scan, trusted publishers, engines without the internet, an eval
before an untrusted model serves.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A control plane that is also a node, serving the starter model and two
judges, needs about 10.5 GB: 3.5 for itself, 2 for the starter, 2 for each
judge, and one more because the scheduler counts whole gigabytes. On 8 GB
the install did every step and then had nowhere to run the starter. The
setup's checks now fail this machine's memory and name what it would serve,
and server init refuses before it installs anything, naming the flags that
leave those models off it. quick-install.md said 8 GB; it says 12.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
llmgw's own extension for VS Code, plain JavaScript with no dependencies,
packed into a .vsix by the binary at /setup/vscode/llmgw.vsix with the
serving gateway's address in it, so it needs no build, no Node and no
marketplace. It registers llmgw as a language-model provider: the chat's
model picker, agent mode and any extension that asks VS Code for a model
get every chat model the key may use, sized and with tool and vision
capabilities as the catalog says, streamed, tool calls both ways.
VS Code is a tool on the account page's Connect your editor: a key, the
install command, and an Open in VS Code link that hands the key to the
extension's secret storage. The extension's Sign In opens the page with a
state of its own; a link without it is confirmed, the gateway named, since
it decides where code is sent.
Tested in VS Code 1.138 against the PoC: 27 models listed, an answer from
qwen2.5-0.5b-instruct, and qwen3-coder-30b calling a tool and using its
result (internal/vscodeext/e2e).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
/install takes a reader from three bare Ubuntu machines to a first answer:
what each machine needs and the commands that check it, the release fetched
and its checksum checked, the eight setup screens, the system's state and
the console, and a first request with a key from the account page. Every
screen is from an install of v0.14.9 on three fresh VMs, recorded with tmux
and kept as terminal frames: the page shows them as text, and plays a
session back when asked, fetching it compressed from /install/cast/. The
administrator's password and the API key were masked before the recording
was kept, and a test fails if either shape reaches the page.
/about gains what llmgw now does beyond routing, and its install steps are
today's: the setup screens and node add, not onboard-node.sh and a model
file, with a link into the guide.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The errors other packages give a page (store, catalog, mirror) are
shown in its language; a percentage is written as the language writes
it (%93 in Turkish); the footer's time in its date form; and a stale
form is refused with a sentence rather than "bad request".
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
auth.ldap checks a username and password against the domain controllers
over LDAPS or StartTLS: found with a service account, checked by binding
as the person. Groups, nested ones included, give staff roles and places
in the tree. Accounts are tied to the objectGUID; the leader reads the
directory every 15 minutes, suspends whom it disabled or removed and
reinstates them after, and refuses a sync that would suspend most.
Entra ID sends no groups for someone in more than 200, only where to
fetch them; that was read as none and took an administrator's role away.
It is now left as it was, and logged. An account made by SCIM or OIDC
can be reinstated again.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Apertus 70B with a 65536-token pool, and then 8B with 131072, on a 24 GB
card loaded their weights and failed to make their context on every start,
the 8B taking the card from the 70B each time only to fail. ctl model serve
now shrinks a pool the card cannot hold beside the weights (KV cache from
the GGUF's shape, 90% of the card, a gigabyte for the engine), saying so,
and refuses weights that do not fit at all; the portal's serve form refuses
such a pool and says the largest that fits.
The mirror page removed the 70B's weights while the catalog still had the
model, whose replicas went on loading the node's copy with nothing said.
Removing weights a catalog model loads is refused, naming the model.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A site whose catalog had no flux-schnell or triposr showed no Images or 3D
page, and nothing said they were one click away. The cluster page now has
a Pictures and 3D models card under the models: each preset, what it
makes, the classes of card here it fits, whether and by whom it was
added, and why the leader has not added it by itself; an Add button, which
says first when it would install ComfyUI on the GPU nodes and restart the
control planes; and the switch for presets.auto. The console's models
screen shows the same under its table: A adds one, asking first in the
same case, and P turns automatic adding on or off.
`ctl model add PRESET` and `ctl model presets` go through the presets API,
so the control plane keeps who added one, and `ctl model presets auto
on|off` flips the setting; against a control plane from before, add sends
the entry as v0.14.4 did.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A game sold in the EU cannot ship what a licence excluding the EU made. On
the profile page a person leaves out licences one by one, or by rule:
every licence forbidding commercial use, every licence granting nothing in
a region. The rules cover models added later. The choice reaches every
principal through the resolved limits, so permits() leaves such models out
of /v1/models, requests, jobs, fallbacks and the portal's pickers, and a
request for one is refused 403 licence_excluded saying which choice did.
Kept in user_excluded_licences (migration 0046), copied with the store and
gone with the person.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
/v1/openapi.json is the API clients use — OpenAI's endpoints, Anthropic's,
and what llmgw adds — in OpenAPI 3.1, generated from internal/clientapi's
route table by the generator the admin API's document comes from, now
told each document's title, server, token and error. A test reads the
router's mux.HandleFunc calls and refuses a route described and not
mounted, or mounted and not described. Asked for before there is a key,
so it needs none.
/docs/api shows it in Swagger UI 5.33.1 (Apache 2.0, served from the
gateway as the page's policy requires): Authorize takes a key, Try it out
sends to this gateway. Staff are offered the admin API's document too,
from /docs/api/admin.json, which nobody else is given.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
/3d beside /images, while some model makes meshes: a picture chosen, the
job asked for through the same path an API client's takes, its progress
shown (waiting for the card, n ahead; making it, the node and percent),
and the mesh in a viewer — model-viewer 4.4.0 (Apache 2.0, with three.js
and lit), served from the gateway since the page's policy loads scripts
from nowhere else — with a download and the model's licence. The mesh is
served to its owner from /3d/jobs/{id}/file; another project's person is
told there is no such job.
A job made from the portal acted for a key that does not exist: the
worker now rebuilds a session's principal from the account that made it,
while it is active and still in the project the job is charged to.
ctl 3d list|get|cancel, over /api/v1/client-jobs (/api/v1/jobs is the
control plane's own work, which ctl jobs shows).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
/images offers the models that make pictures, and the chat no longer
offers them. Like the chat, the browser holds a session and never a key;
the page's request is rebuilt from its own fields and sent down the path
every /v1/images/generations request takes. Size, how many, seed, steps
and a negative prompt; each picture shown with its seed, a download link
and a way to make it again. Nothing is kept but the usage. The menu
offers the page while some model makes pictures.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
/admin/settings edits the document llmgw ctl config edit opens: checked as a
control plane would load it, refused when made from a version since replaced
(the edit kept), its history viewed and rolled back as a new version, every
change audited by size and never by contents, since it can hold webhook
secrets. Administrators only; in English and Turkish.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A Documents picker on /chat offers the collections the person may read in
the chosen project, the default when there is one, or none; the sources an
answer was given are listed under it. The page's choice becomes the
X-Llmgw-Knowledge header; one the page sent itself is dropped. In English
and Turkish.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
/forgot mails a link to set a new password when the cluster's mail settings
say how - SMTP relay, sender, and the public base_url the link points at,
never the request's Host - and says the same whether or not the address has
an account. /reset sets it once, closing every session. Links are kept by
their hash in password_resets, good for an hour, a newer one replacing an
older. Without mail, /forgot says to ask an operator, and llmgw users
reset-link makes the link to hand over. Single sign-on accounts get none.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
guard.enabled (unset is on) makes every policy check nothing while false,
live: llmgw ctl guard off/on set it in the cluster's settings, and ctl
status, ctl guard list and the guardrails page say when they are off.
A refusal by a judge names the hazard it saw - judge: S6,Specialized
Advice - and the verdict is kept with the request in a new usage column,
guard_rule, shown by ctl request, the request log and the guardrails page.
llmgw ctl guard judge ask has a judge read a text, past the cache, and shows
its raw answer and how it is read; the endpoint is kept out of the audit so
no text is recorded.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
llmgw ctl guard judge status lists each judge model the policies ask:
in the catalog, as a judge, replicas ready, its last failure to answer, and
what is wrong in words. enable serves the missing one through model serve -
Llama Guard 3 1B on CPUs, role guard, under the name the policies give it -
with --replace for a model of that name that is not a judge; the console has
it under Do. GET /api/v1/guard/judges, and the summary's judges, put the
problem on ctl status and the console's Overview.
A Llama Guard file served from the mirror page is added as a judge, under
the name the policies ask, rather than as a model clients use the policies
never find. ctl says plainly when the API answers with a page.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Everything in server.yaml that is not one machine's own is now a versioned
document in the store (cluster_settings), the same on every control plane.
The first start against a store imports the file's cluster keys, with the
baseline guard policy when there is none; after that those keys in a file
are ignored, with a warning.
Each control plane polls the document's version and its own server.yaml's
hash. A change is validated and applied: guard live (Gate.Replace), anything
else by a graceful restart that systemd brings back, standbys first and the
leader 30s later. A version that does not load on a machine is not applied;
at start the newest one that loads is used, so a bad edit cannot keep every
control plane down.
llmgw ctl config show/get/set/unset/edit/history/rollback, the admin API
under /api/v1/settings, and the console's Settings screen, which layers the
store's document over the machine's file, write it. Commands beside the
control plane that read cluster keys (init, node add, server add, backup,
upgrade) read the layered configuration; init --bundle writes offline to
the store once it holds the settings.
A standby that becomes the leader re-reads models, engine choices and
facility holds from the store.
Retires appendGuardIfMissing: the import does it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every template and handler message in internal/web goes through a JSON
catalogue under internal/web/locales/<code>/, with English as the fallback.
The language is the person's choice (a cookie set by /_lang/{code}), else
Accept-Language, else landing.language. Tests refuse a key some language
lacks, a translation with other arguments than English, and template text
left in English.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>