Ilker Manap and Claude Opus 5.5
2158a933e0
v0.11.2
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-05 01:30:37 +02:00
Ilker Manap and Claude Opus 5.5
803d1d5fb6
CHANGELOG and BACKLOG for the rest of C24 and the engine fixes; README
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-05 01:30:01 +02:00
Ilker Manap and Claude Opus 5.5
a13a519386
CHANGELOG and BACKLOG for C17-C23 and the first parts of C24; README lists the new docs
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-05 00:17:16 +02:00
Ilker Manap and Claude Opus 5.5
30f7f3474a
Forgetting a person and keeping requests in the cluster: users forget, keys residency --in-cluster
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 23:27:31 +02:00
Ilker Manap and Claude Opus 5.5
35657c1417
Secrets in Vault: env: and vault: references wherever a secret file is named, and the store's own secrets sealed under a master key — Vault transit or a local key
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 23:21:44 +02:00
Ilker Manap and Claude Opus 5.5
bce54f0941
Scaling by the clock: schedule windows with their own min and max, prewarm, control.timezone
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 23:16:56 +02:00
Ilker Manap and Claude Opus 5.5
5b92ae0277
Token-level timing and admission by SLO: TPOT, inter-token and tokens a second by model and tenant; a model past its SLO is saturated for the lower bands
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 23:13:18 +02:00
Ilker Manap and Claude Opus 5.5
10466416df
server restore: an archive checked, dry-run, and put back — configuration with modes and owners, the store in SQLite or PostgreSQL; backup_copy_to copies archives off the machine
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 23:09:59 +02:00
Ilker Manap and Claude Opus 5.5
e502996198
Anthropic's Messages API: /v1/messages translated to chat completions and back, streamed or not, with tools and images; keys in x-api-key
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 22:59:55 +02:00
Ilker Manap and Claude Opus 5.5
9136bc71c2
More of OpenAI's API: Responses structured output, images and previous_response_id; rerank, score and audio passed through; moderations by the judge
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 22:56:25 +02:00
Ilker Manap and Claude Opus 5.5
32ff1dbd6f
Requests kept and replayed: keys requests per project with retention and a share, ctl request --body and --replay
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 22:51:21 +02:00
Ilker Manap and Claude Opus 5.5
841ee21c12
Batches: OpenAI's Files and Batch APIs, served at the lowest priority under a store lease, each line a request of the key that made it
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 21:00:01 +02:00
Ilker Manap and Claude Opus 5.5
889be70b8f
Canary on the console's model menu; docs
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 20:33:59 +02:00
Ilker Manap and Claude Opus 5.5
29a7347df2
Requests can be tagged: metadata, x-llmgw-tags and the user field on the usage record, and counted by the tags router.tag_labels names
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 19:45:37 +02:00
Ilker Manap and Claude Opus 5.5
c553f2041a
The baseline fails closed with two judges, and a judge not answering is a notification
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 19:19:49 +02:00
Ilker Manap and Claude Opus 5.5
ddb0472f9b
A hardened install by default: baseline guardrails with Llama Guard 3 1B judging on a CPU, keys at most 90 days
...
server init (and the setup screens) write guard.default [baseline] —
secrets refused, cards masked, injection recorded, the judge refusing
S1-S4, S7, S9-S12 in prompts and answers, fail open after 5s — fetch the
judge into the mirror and run it on a node's CPU. --no-guard-model leaves
it out. The closing text names what init cannot do: TLS on the client
port and backup recipients.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 17:45:31 +02:00
Ilker Manap and Claude Opus 5.5
1b71c38f89
Notifications: webhooks for a parked model, a lost node, a spent budget and new weights; Alertmanager receiver templates
...
notify.webhooks sends signed JSON or Slack-compatible messages, queued,
retried, de-duplicated hourly; ctl notify test checks each webhook.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 16:58:11 +02:00
Ilker Manap and Claude Opus 5.5
57739d736a
Usage export as CSV: by period and by place, person, key and model, from the rollups
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 16:49:04 +02:00
Ilker Manap and Claude Opus 5.5
8866a1ea0f
Key lifecycle: lifetime limits by scope, rotation with overlap, kinds and expiry in the listing
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 16:45:05 +02:00
Ilker Manap and Claude Opus 5.5
14ea59d83f
Provisioning through SCIM 2.0: accounts and groups kept in step by the identity provider
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 16:40:15 +02:00
Ilker Manap and Claude Opus 5.5
4cf40644c9
Single sign-on through OpenID Connect: accounts, staff roles and memberships from the provider
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 16:36:15 +02:00
Ilker Manap and Claude Opus 5.5
b60cf45adf
Staff roles (viewer, operator, billing) and the console's Audit view
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 16:30:56 +02:00
Ilker Manap and Claude Opus 5.5
d55472c00a
Guardrails in the console: attach and detach on a branch, where each applies from, and a column
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 15:18:42 +02:00
Ilker Manap and Claude Opus 5.5
43f6a585a6
A judge model on the cluster: role guard in the catalog, judge in a guard policy
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 15:09:31 +02:00
Ilker Manap and Claude Opus 5.5
2f07b2ba0b
Document guardrails, correct what data.md said leaves the cluster, and regenerate the reference
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 12:36:37 +02:00
Ilker Manap and Claude Opus 5.5
979aa44804
Hook a guard into the router: judge requests before dispatch and filter answers as they are relayed
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 12:20:24 +02:00
Ilker Manap and Claude Opus 5.5
eb10aee93b
Document TLS between the store's machines, and close section A of the backlog
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 12:12:35 +02:00
Ilker Manap and Claude Opus 5.5
efb4b06fc5
Report a ready engine dying from every connection, adopted ones included
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 11:50:09 +02:00
Ilker Manap and Claude Opus 5.5
7b6276901a
Keep llmgw's internal ports to the cluster's networks with an nftables table of its own
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 11:03:46 +02:00
Ilker Manap and Claude Opus 5.5
7e57863edd
Hide the mirror's bookkeeping, bound GGUF array nesting, and forget old serve jobs and downloads
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:56:16 +02:00
Ilker Manap and Claude Opus 5.5
fdd4dcf841
Serve every page under a hash-based Content-Security-Policy, and update KaTeX to 0.16.47
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:54:04 +02:00
Ilker Manap and Claude Opus 5.5
fd9f665ef8
End a session unused for a day, and purge dead sessions and claims hourly
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:50:56 +02:00
Ilker Manap and Claude Opus 5.5
ba5158d463
List in /v1/models only the models the key may use
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:49:54 +02:00
Ilker Manap and Claude Opus 5.5
f9493aadb9
Guard a GPU-less node's problems, and bound thread ids
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:49:10 +02:00
Ilker Manap and Claude Opus 5.5
a56ad8e4c4
Keep the control plane's fetches off link-local addresses, and see through NAT64, 6to4 and proxies
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:48:01 +02:00
Ilker Manap and Claude Opus 5.5
fbac1aa664
Take only torch from PyTorch's index, and lock its builds with every file's address
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:45:55 +02:00
Ilker Manap and Claude Opus 5.5
12efb7d085
Encrypt backups to backup_recipients with age
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:38:47 +02:00
Ilker Manap and Claude Opus 5.5
3b05ec0538
Have engines listen on the management address, and take SGLang's key off its command line
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:36:42 +02:00
Ilker Manap and Claude Opus 5.5
9b61bbc983
Revoke a removed node's certificate, and require --api-client with a mirror's mutual TLS
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:34:35 +02:00
Ilker Manap and Claude Opus 5.5
fe5b7f88f0
Believe X-Forwarded-* only from router.trusted_proxies
...
A new internal/clientip says who sent a request and how: the connection's
address, scheme and Host, or through a configured proxy what it forwarded.
The base URL in pages and setup scripts, isHTTPS for cookies, and the
per-address limits of auth and the sign-in forms use it. /setup scripts are
no-store.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:29:02 +02:00
Ilker Manap and Claude Opus 5.5
44cd58005a
Count sign-in attempts per address and client, guard the sign-in form with a token, ration both forms per address
...
A wrong password from anywhere counted against the address, so ten locked
its owner out. Attempts are now counted per address and client, with a
looser bound per address alone. The sign-in form carries a token matched
by a cookie (login CSRF). POST /login and /signup are rationed per client
address, a hash waits at most ten seconds for a turn, and server init
writes signup_open: false.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:26:29 +02:00
Ilker Manap and Claude Opus 5.5
f1b49d0251
Bound idle time, header size and request reading on every listener, and connections on the client port
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:21:15 +02:00
Ilker Manap and Claude Opus 5.5
d3fb217b31
Bound the key cache and ration lookups of unknown keys per client address
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:19:38 +02:00
Ilker Manap and Claude Opus 5.5
ebf437a36a
Have the setup's sudo grant remove itself within the hour, and keep a target from reading as an ssh option
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:17:52 +02:00
Ilker Manap and Claude Opus 5.5
1f406247e4
Expire join codes on the setup's side, bind the join port to its address, trust the peer address
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:16:53 +02:00
Ilker Manap and Claude Opus 5.5
27c49d0af9
Refuse the keys of an account that is not active
...
Resolve now checks the owner's state for a personal key; the authenticator
treats an inactive owner as a revoked key. Suspension stops the keys within
the cache TTL, reinstatement brings them back.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:15:32 +02:00
Ilker Manap and Claude Opus 5.5
20ebd21243
Skip capacity fallbacks the key may not use, and charge what the model that answered costs
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:13:59 +02:00
Ilker Manap and Claude Opus 5.5
2b29016d5b
Ask for a password on Patroni's API and give the store TLS, new clusters and existing ones in place
...
Patroni's REST API accepted changes from anyone on port 8008, and
PostgreSQL ran with sslmode=disable between machines. Setup now keeps a
fourth secret for the API (carried to members of older clusters), issues
each member a certificate from the cluster's authority, writes
restapi.authentication into each member's config and reloads it, and
turns ssl on with hostssl remote rules through an authenticated PATCH of
the shared configuration. The DSN becomes sslmode=verify-full against the
authority's certificate whenever the cluster has TLS, so a control plane
without the authority's key reconnects too. server add authenticates its
PATCH and lets the new control plane in by hostssl. Tried on the e2e
cluster: unauthenticated PATCH 401, plaintext refused, replication
streaming, the control plane on verify-full.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:11:28 +02:00
Ilker Manap and Claude Opus 5.5
9d961c3fc8
Say that the admin API wants its token, and that ctl runs with sudo
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 10:02:56 +02:00
Ilker Manap and Claude Opus 5.5
2d47427cdc
Run engines as llmgw-engine rather than root, and make the user when a node is added
...
The supervisor drops to the run_as user (default llmgw-engine) when the
agent is root: its credential and groups, a 0700 home under the work
directory as the working directory, HOME and XDG_CACHE_HOME there. A node
without the user keeps engines as root and logs why. node add creates the
user in the video and render groups.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 09:57:41 +02:00