Ilker Manap and Claude Opus 5.5
e81f1cba4c
Merge C24 docs portal and client guide
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 23:47:35 +02:00
Ilker Manap and Claude Opus 5.5
bd439aca29
A docs portal at /docs and a client developer's guide
...
The manual the binary carries, rendered on the client port: a sidebar of
the documents in the console's reading order (now one list, llmgw.Manual,
for both), heading anchors, links between documents kept in the portal and
links to files outside the binary sent to the repository, no script.
Public by default; landing.docs_sign_in sends visitors to sign in first.
docs/clients.md shows the official OpenAI and Anthropic SDKs against
llmgw in Python, TypeScript, Go and curl, with llmgw's headers; the
portal's index, the landing page and the account page link to it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 23:46:42 +02:00
Ilker Manap and Claude Opus 5.5
30f7f3474a
Forgetting a person and keeping requests in the cluster: users forget, keys residency --in-cluster
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 23:27:31 +02:00
Ilker Manap and Claude Opus 5.5
d67bd9b2cc
A test against a real Vault, when one is named
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 23:22:36 +02:00
Ilker Manap and Claude Opus 5.5
35657c1417
Secrets in Vault: env: and vault: references wherever a secret file is named, and the store's own secrets sealed under a master key — Vault transit or a local key
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 23:21:44 +02:00
Ilker Manap and Claude Opus 5.5
bce54f0941
Scaling by the clock: schedule windows with their own min and max, prewarm, control.timezone
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 23:16:56 +02:00
Ilker Manap and Claude Opus 5.5
74429f1c00
SLO fields described
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 23:13:30 +02:00
Ilker Manap and Claude Opus 5.5
5b92ae0277
Token-level timing and admission by SLO: TPOT, inter-token and tokens a second by model and tenant; a model past its SLO is saturated for the lower bands
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 23:13:18 +02:00
Ilker Manap and Claude Opus 5.5
10466416df
server restore: an archive checked, dry-run, and put back — configuration with modes and owners, the store in SQLite or PostgreSQL; backup_copy_to copies archives off the machine
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 23:09:59 +02:00
Ilker Manap and Claude Opus 5.5
d29f694aaf
Moderations name a judge's categories once, by name
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 23:03:35 +02:00
Ilker Manap and Claude Opus 5.5
e502996198
Anthropic's Messages API: /v1/messages translated to chat completions and back, streamed or not, with tools and images; keys in x-api-key
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 22:59:55 +02:00
Ilker Manap and Claude Opus 5.5
9136bc71c2
More of OpenAI's API: Responses structured output, images and previous_response_id; rerank, score and audio passed through; moderations by the judge
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 22:56:25 +02:00
Ilker Manap and Claude Opus 5.5
32ff1dbd6f
Requests kept and replayed: keys requests per project with retention and a share, ctl request --body and --replay
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 22:51:21 +02:00
Ilker Manap and Claude Opus 5.5
841ee21c12
Batches: OpenAI's Files and Batch APIs, served at the lowest priority under a store lease, each line a request of the key that made it
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 21:00:01 +02:00
Ilker Manap and Claude Opus 5.5
889be70b8f
Canary on the console's model menu; docs
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 20:33:59 +02:00
Ilker Manap and Claude Opus 5.5
a1dfd4df1d
Canary of a model's newer weights: ctl model canary, promote and canaries, rolled back by itself when it serves worse
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 20:31:26 +02:00
Ilker Manap and Claude Opus 5.5
0a4076928a
Canary routing: a model's canary takes its share of requests, a session stays on one side, x-llmgw-variant pins one, and the two are compared
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 19:56:16 +02:00
Ilker Manap and Claude Opus 5.5
29a7347df2
Requests can be tagged: metadata, x-llmgw-tags and the user field on the usage record, and counted by the tags router.tag_labels names
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 19:45:37 +02:00
Ilker Manap and Claude Opus 5.5
e59ff52932
Merge features-c: section C of the backlog, items 1-5, and a hardened install
...
Audit log and staff roles, OIDC and SCIM, key lifecycle, usage export,
notifications; server init writes baseline guardrails with two Llama Guard
judges failing closed, and keys of at most 90 days.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 19:20:41 +02:00
Ilker Manap and Claude Opus 5.5
c553f2041a
The baseline fails closed with two judges, and a judge not answering is a notification
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 19:19:49 +02:00
Ilker Manap and Claude Opus 5.5
07af03aa93
server init: the firewall and the judge before the starter model, and a model that fits nowhere said within two minutes
...
A starter model with no room failed init after ten minutes and left the
firewall and the judge uninstalled.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 19:11:12 +02:00
Ilker Manap and Claude Opus 5.5
ddb0472f9b
A hardened install by default: baseline guardrails with Llama Guard 3 1B judging on a CPU, keys at most 90 days
...
server init (and the setup screens) write guard.default [baseline] —
secrets refused, cards masked, injection recorded, the judge refusing
S1-S4, S7, S9-S12 in prompts and answers, fail open after 5s — fetch the
judge into the mirror and run it on a node's CPU. --no-guard-model leaves
it out. The closing text names what init cannot do: TLS on the client
port and backup recipients.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 17:45:31 +02:00
Ilker Manap and Claude Opus 5.5
6abd510628
Docs: one way in — quick-install starts at the setup screens, quickstart becomes development, install.md no longer says engines are yours to install
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 17:32:54 +02:00
Ilker Manap and Claude Opus 5.5
4979e08d64
Console: rotate a key, cap key lifetimes, test the webhooks; README, onboarding and console docs caught up
...
The README's status stopped at v0.9 and listed gaps since closed; onboarding
still said there was no SSO and no roles.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 17:28:24 +02:00
Ilker Manap and Claude Opus 5.5
1b71c38f89
Notifications: webhooks for a parked model, a lost node, a spent budget and new weights; Alertmanager receiver templates
...
notify.webhooks sends signed JSON or Slack-compatible messages, queued,
retried, de-duplicated hourly; ctl notify test checks each webhook.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 16:58:11 +02:00
Ilker Manap and Claude Opus 5.5
646ff996c7
gofmt the usage export
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 16:49:14 +02:00
Ilker Manap and Claude Opus 5.5
57739d736a
Usage export as CSV: by period and by place, person, key and model, from the rollups
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 16:49:04 +02:00
Ilker Manap and Claude Opus 5.5
15a7237a02
Match a mapped project however its path is spaced: the store writes "Acme / ml / train"
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 16:45:47 +02:00
Ilker Manap and Claude Opus 5.5
8866a1ea0f
Key lifecycle: lifetime limits by scope, rotation with overlap, kinds and expiry in the listing
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 16:45:05 +02:00
Ilker Manap and Claude Opus 5.5
178964cf6a
gofmt
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 16:40:27 +02:00
Ilker Manap and Claude Opus 5.5
14ea59d83f
Provisioning through SCIM 2.0: accounts and groups kept in step by the identity provider
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 16:40:15 +02:00
Ilker Manap and Claude Opus 5.5
5cff456b9b
Let /login/oidc through without an API key: /login alone was exempt
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 16:37:24 +02:00
Ilker Manap and Claude Opus 5.5
dfe4093aea
gofmt the OIDC pages
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 16:36:20 +02:00
Ilker Manap and Claude Opus 5.5
4cf40644c9
Single sign-on through OpenID Connect: accounts, staff roles and memberships from the provider
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 16:36:15 +02:00
Ilker Manap and Claude Opus 5.5
b60cf45adf
Staff roles (viewer, operator, billing) and the console's Audit view
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 16:30:56 +02:00
Ilker Manap and Claude Opus 5.5
f8fcecd640
Audit log: every change through the admin API and the administration pages, with ctl audit
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 16:25:56 +02:00
Ilker Manap and Claude Opus 5.5
bcd2413535
Merge guardrails-b: section B of the backlog (guardrails)
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 16:09:48 +02:00
Ilker Manap and Claude Opus 5.5
4df30fc175
Record section B's state after trying it on the PoC
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 15:40:18 +02:00
Ilker Manap and Claude Opus 5.5
bd4736125b
Do not name a judge model in the error a client gets for a model nobody serves
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 15:37:52 +02:00
Ilker Manap and Claude Opus 5.5
dbcecd1e85
Judge an answer with Llama Guard 3's own prompt, as a completion, not a conversation ending on the answer
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 15:35:57 +02:00
Ilker Manap and Claude Opus 5.5
742d6633e2
Read a judge's verdict after the answer llama.cpp goes on from, and log a verdict that cannot be read
...
Found on the PoC: llama.cpp takes a conversation ending on the assistant's
turn as a reply to continue, so Llama Guard answered "Rome.safe" and every
answer judged was served unjudged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 15:33:23 +02:00
Ilker Manap and Claude Opus 5.5
c69e6f151c
Serve a judge model in one command: ctl model serve --guard-format
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 15:27:35 +02:00
Ilker Manap and Claude Opus 5.5
62038484d6
Never lose a word held back at one release when the next has nothing to rewrite
...
Found on the PoC: a streamed list came out as "1. 2. 3." — each item cut at
one release and dropped at the next, which sent the engine's own events.
A property test now cuts an answer every way and checks it arrives whole.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 15:24:22 +02:00
Ilker Manap and Claude Opus 5.5
2cc52fd1a7
Read an unwritten guard mode as enforce when merging, not as the weakest
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 15:22:07 +02:00
Ilker Manap and Claude Opus 5.5
83e1444981
Regenerate the configuration reference with the judge and the guard role
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 15:18:51 +02:00
Ilker Manap and Claude Opus 5.5
d55472c00a
Guardrails in the console: attach and detach on a branch, where each applies from, and a column
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 15:18:42 +02:00
Ilker Manap and Claude Opus 5.5
02aac84ca5
The /admin/guard page, a Guardrails link, and the policies in force on the profile page
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 15:15:38 +02:00
Ilker Manap and Claude Opus 5.5
7e18b8ab53
Show guardrail refusals and stopped answers on the chat page, and the verdict in ctl request
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 15:13:15 +02:00
Ilker Manap and Claude Opus 5.5
47e847423d
Guardrail metrics and alerts: checks, latency, judge failures, fail-open, redactions, stopped streams, tenant blocks
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 15:11:50 +02:00
Ilker Manap and Claude Opus 5.5
43f6a585a6
A judge model on the cluster: role guard in the catalog, judge in a guard policy
...
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-04 15:09:31 +02:00