Ilker Manap and Claude Opus 5.5
abaf086fd5
A standby's portal is the leader's; the changelog as the release notes read it
...
CI / test (push) Successful in 2m42s
Release / notes (push) Successful in 3s
Release / build (push) Successful in 3m20s
A proxy left pointing at a standby showed an empty cluster: the portal and
the admin pages read the standby's own memory. A standby now passes its
client port to the leader's, all but /health, /ready and /leader, which
must describe the machine a load balancer is choosing; and its admin port
passes the dashboard as well as the API. The leader trusts the control
planes' X-Forwarded-For, so clients keep their own address.
The changelog's candidate section is "Unreleased" again: release notes
for an rc read that section, and v0.11.3-rc.1 got none.
The e2e wait for two endpoints is 30s: under the race detector on a busy
runner, 15s was missed once.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-05 06:34:57 +02:00
Ilker Manap and Claude Opus 5.5
c5b34978f5
Guardrails in the console
...
CI / test (push) Successful in 2m52s
The console never mentioned them. A Guardrails view shows each policy with
its mode, fail behaviour and checks, whether its judge models have replicas
ready, where policies are attached (attach and detach from it), and what
they did in a window. The admin API gains GET /api/v1/guard/events, and GET
/api/v1/guard says what each policy does.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-05 03:36:17 +02:00
Ilker Manap and Claude Opus 5.5
c5e388807a
ctl upgrade in waves, and the other control plane first
...
One node at a time was hours on a fleet of hundreds. Nodes now go one, then
two, four, up to --parallel at once, each wave waited on whole; a wave that
does not come back stops it. The other control plane is upgraded first over
ssh, so a leader restarted last hands over to a standby on the release.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-05 02:43:33 +02:00
Ilker Manap and Claude Opus 5.5
2158a933e0
v0.11.2
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-05 01:30:37 +02:00
Ilker Manap and Claude Opus 5.5
803d1d5fb6
CHANGELOG and BACKLOG for the rest of C24 and the engine fixes; README
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-05 01:30:01 +02:00
Ilker Manap and Claude Opus 5.5
a13a519386
CHANGELOG and BACKLOG for C17-C23 and the first parts of C24; README lists the new docs
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-05 00:17:16 +02:00
Ilker Manap and Claude Opus 5.5
30f7f3474a
Forgetting a person and keeping requests in the cluster: users forget, keys residency --in-cluster
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 23:27:31 +02:00
Ilker Manap and Claude Opus 5.5
35657c1417
Secrets in Vault: env: and vault: references wherever a secret file is named, and the store's own secrets sealed under a master key — Vault transit or a local key
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 23:21:44 +02:00
Ilker Manap and Claude Opus 5.5
bce54f0941
Scaling by the clock: schedule windows with their own min and max, prewarm, control.timezone
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 23:16:56 +02:00
Ilker Manap and Claude Opus 5.5
5b92ae0277
Token-level timing and admission by SLO: TPOT, inter-token and tokens a second by model and tenant; a model past its SLO is saturated for the lower bands
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 23:13:18 +02:00
Ilker Manap and Claude Opus 5.5
10466416df
server restore: an archive checked, dry-run, and put back — configuration with modes and owners, the store in SQLite or PostgreSQL; backup_copy_to copies archives off the machine
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 23:09:59 +02:00
Ilker Manap and Claude Opus 5.5
e502996198
Anthropic's Messages API: /v1/messages translated to chat completions and back, streamed or not, with tools and images; keys in x-api-key
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 22:59:55 +02:00
Ilker Manap and Claude Opus 5.5
9136bc71c2
More of OpenAI's API: Responses structured output, images and previous_response_id; rerank, score and audio passed through; moderations by the judge
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 22:56:25 +02:00
Ilker Manap and Claude Opus 5.5
32ff1dbd6f
Requests kept and replayed: keys requests per project with retention and a share, ctl request --body and --replay
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 22:51:21 +02:00
Ilker Manap and Claude Opus 5.5
841ee21c12
Batches: OpenAI's Files and Batch APIs, served at the lowest priority under a store lease, each line a request of the key that made it
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 21:00:01 +02:00
Ilker Manap and Claude Opus 5.5
889be70b8f
Canary on the console's model menu; docs
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 20:33:59 +02:00
Ilker Manap and Claude Opus 5.5
29a7347df2
Requests can be tagged: metadata, x-llmgw-tags and the user field on the usage record, and counted by the tags router.tag_labels names
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 19:45:37 +02:00
Ilker Manap and Claude Opus 5.5
c553f2041a
The baseline fails closed with two judges, and a judge not answering is a notification
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 19:19:49 +02:00
Ilker Manap and Claude Opus 5.5
ddb0472f9b
A hardened install by default: baseline guardrails with Llama Guard 3 1B judging on a CPU, keys at most 90 days
...
server init (and the setup screens) write guard.default [baseline] —
secrets refused, cards masked, injection recorded, the judge refusing
S1-S4, S7, S9-S12 in prompts and answers, fail open after 5s — fetch the
judge into the mirror and run it on a node's CPU. --no-guard-model leaves
it out. The closing text names what init cannot do: TLS on the client
port and backup recipients.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 17:45:31 +02:00
Ilker Manap and Claude Opus 5.5
1b71c38f89
Notifications: webhooks for a parked model, a lost node, a spent budget and new weights; Alertmanager receiver templates
...
notify.webhooks sends signed JSON or Slack-compatible messages, queued,
retried, de-duplicated hourly; ctl notify test checks each webhook.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 16:58:11 +02:00
Ilker Manap and Claude Opus 5.5
57739d736a
Usage export as CSV: by period and by place, person, key and model, from the rollups
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 16:49:04 +02:00
Ilker Manap and Claude Opus 5.5
8866a1ea0f
Key lifecycle: lifetime limits by scope, rotation with overlap, kinds and expiry in the listing
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 16:45:05 +02:00
Ilker Manap and Claude Opus 5.5
14ea59d83f
Provisioning through SCIM 2.0: accounts and groups kept in step by the identity provider
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 16:40:15 +02:00
Ilker Manap and Claude Opus 5.5
4cf40644c9
Single sign-on through OpenID Connect: accounts, staff roles and memberships from the provider
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 16:36:15 +02:00
Ilker Manap and Claude Opus 5.5
b60cf45adf
Staff roles (viewer, operator, billing) and the console's Audit view
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 16:30:56 +02:00
Ilker Manap and Claude Opus 5.5
d55472c00a
Guardrails in the console: attach and detach on a branch, where each applies from, and a column
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 15:18:42 +02:00
Ilker Manap and Claude Opus 5.5
43f6a585a6
A judge model on the cluster: role guard in the catalog, judge in a guard policy
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 15:09:31 +02:00
Ilker Manap and Claude Opus 5.5
2f07b2ba0b
Document guardrails, correct what data.md said leaves the cluster, and regenerate the reference
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 12:36:37 +02:00
Ilker Manap and Claude Opus 5.5
979aa44804
Hook a guard into the router: judge requests before dispatch and filter answers as they are relayed
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 12:20:24 +02:00
Ilker Manap and Claude Opus 5.5
eb10aee93b
Document TLS between the store's machines, and close section A of the backlog
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 12:12:35 +02:00
Ilker Manap and Claude Opus 5.5
efb4b06fc5
Report a ready engine dying from every connection, adopted ones included
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 11:50:09 +02:00
Ilker Manap and Claude Opus 5.5
7b6276901a
Keep llmgw's internal ports to the cluster's networks with an nftables table of its own
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 11:03:46 +02:00
Ilker Manap and Claude Opus 5.5
7e57863edd
Hide the mirror's bookkeeping, bound GGUF array nesting, and forget old serve jobs and downloads
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:56:16 +02:00
Ilker Manap and Claude Opus 5.5
fdd4dcf841
Serve every page under a hash-based Content-Security-Policy, and update KaTeX to 0.16.47
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:54:04 +02:00
Ilker Manap and Claude Opus 5.5
fd9f665ef8
End a session unused for a day, and purge dead sessions and claims hourly
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:50:56 +02:00
Ilker Manap and Claude Opus 5.5
ba5158d463
List in /v1/models only the models the key may use
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:49:54 +02:00
Ilker Manap and Claude Opus 5.5
f9493aadb9
Guard a GPU-less node's problems, and bound thread ids
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:49:10 +02:00
Ilker Manap and Claude Opus 5.5
a56ad8e4c4
Keep the control plane's fetches off link-local addresses, and see through NAT64, 6to4 and proxies
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:48:01 +02:00
Ilker Manap and Claude Opus 5.5
fbac1aa664
Take only torch from PyTorch's index, and lock its builds with every file's address
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:45:55 +02:00
Ilker Manap and Claude Opus 5.5
12efb7d085
Encrypt backups to backup_recipients with age
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:38:47 +02:00
Ilker Manap and Claude Opus 5.5
3b05ec0538
Have engines listen on the management address, and take SGLang's key off its command line
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:36:42 +02:00
Ilker Manap and Claude Opus 5.5
9b61bbc983
Revoke a removed node's certificate, and require --api-client with a mirror's mutual TLS
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:34:35 +02:00
Ilker Manap and Claude Opus 5.5
fe5b7f88f0
Believe X-Forwarded-* only from router.trusted_proxies
...
A new internal/clientip says who sent a request and how: the connection's
address, scheme and Host, or through a configured proxy what it forwarded.
The base URL in pages and setup scripts, isHTTPS for cookies, and the
per-address limits of auth and the sign-in forms use it. /setup scripts are
no-store.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:29:02 +02:00
Ilker Manap and Claude Opus 5.5
44cd58005a
Count sign-in attempts per address and client, guard the sign-in form with a token, ration both forms per address
...
A wrong password from anywhere counted against the address, so ten locked
its owner out. Attempts are now counted per address and client, with a
looser bound per address alone. The sign-in form carries a token matched
by a cookie (login CSRF). POST /login and /signup are rationed per client
address, a hash waits at most ten seconds for a turn, and server init
writes signup_open: false.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:26:29 +02:00
Ilker Manap and Claude Opus 5.5
f1b49d0251
Bound idle time, header size and request reading on every listener, and connections on the client port
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:21:15 +02:00
Ilker Manap and Claude Opus 5.5
d3fb217b31
Bound the key cache and ration lookups of unknown keys per client address
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:19:38 +02:00
Ilker Manap and Claude Opus 5.5
ebf437a36a
Have the setup's sudo grant remove itself within the hour, and keep a target from reading as an ssh option
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:17:52 +02:00
Ilker Manap and Claude Opus 5.5
1f406247e4
Expire join codes on the setup's side, bind the join port to its address, trust the peer address
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:16:53 +02:00
Ilker Manap and Claude Opus 5.5
27c49d0af9
Refuse the keys of an account that is not active
...
Resolve now checks the owner's state for a personal key; the authenticator
treats an inactive owner as a revoked key. Suspension stops the keys within
the cache TTL, reinstatement brings them back.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:15:32 +02:00
Ilker Manap and Claude Opus 5.5
20ebd21243
Skip capacity fallbacks the key may not use, and charge what the model that answered costs
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:13:59 +02:00