Commit Graph
472 Commits
Author SHA1 Message Date
Ilker ManapandClaude Opus 5.5 e81f1cba4c Merge C24 docs portal and client guide
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 23:47:35 +02:00
Ilker ManapandClaude Opus 5.5 bd439aca29 A docs portal at /docs and a client developer's guide
The manual the binary carries, rendered on the client port: a sidebar of
the documents in the console's reading order (now one list, llmgw.Manual,
for both), heading anchors, links between documents kept in the portal and
links to files outside the binary sent to the repository, no script.
Public by default; landing.docs_sign_in sends visitors to sign in first.

docs/clients.md shows the official OpenAI and Anthropic SDKs against
llmgw in Python, TypeScript, Go and curl, with llmgw's headers; the
portal's index, the landing page and the account page link to it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 23:46:42 +02:00
Ilker ManapandClaude Opus 5.5 30f7f3474a Forgetting a person and keeping requests in the cluster: users forget, keys residency --in-cluster
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 23:27:31 +02:00
Ilker ManapandClaude Opus 5.5 d67bd9b2cc A test against a real Vault, when one is named
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 23:22:36 +02:00
Ilker ManapandClaude Opus 5.5 35657c1417 Secrets in Vault: env: and vault: references wherever a secret file is named, and the store's own secrets sealed under a master key — Vault transit or a local key
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 23:21:44 +02:00
Ilker ManapandClaude Opus 5.5 bce54f0941 Scaling by the clock: schedule windows with their own min and max, prewarm, control.timezone
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 23:16:56 +02:00
Ilker ManapandClaude Opus 5.5 74429f1c00 SLO fields described
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 23:13:30 +02:00
Ilker ManapandClaude Opus 5.5 5b92ae0277 Token-level timing and admission by SLO: TPOT, inter-token and tokens a second by model and tenant; a model past its SLO is saturated for the lower bands
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 23:13:18 +02:00
Ilker ManapandClaude Opus 5.5 10466416df server restore: an archive checked, dry-run, and put back — configuration with modes and owners, the store in SQLite or PostgreSQL; backup_copy_to copies archives off the machine
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 23:09:59 +02:00
Ilker ManapandClaude Opus 5.5 d29f694aaf Moderations name a judge's categories once, by name
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 23:03:35 +02:00
Ilker ManapandClaude Opus 5.5 e502996198 Anthropic's Messages API: /v1/messages translated to chat completions and back, streamed or not, with tools and images; keys in x-api-key
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 22:59:55 +02:00
Ilker ManapandClaude Opus 5.5 9136bc71c2 More of OpenAI's API: Responses structured output, images and previous_response_id; rerank, score and audio passed through; moderations by the judge
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 22:56:25 +02:00
Ilker ManapandClaude Opus 5.5 32ff1dbd6f Requests kept and replayed: keys requests per project with retention and a share, ctl request --body and --replay
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 22:51:21 +02:00
Ilker ManapandClaude Opus 5.5 841ee21c12 Batches: OpenAI's Files and Batch APIs, served at the lowest priority under a store lease, each line a request of the key that made it
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 21:00:01 +02:00
Ilker ManapandClaude Opus 5.5 889be70b8f Canary on the console's model menu; docs
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 20:33:59 +02:00
Ilker ManapandClaude Opus 5.5 a1dfd4df1d Canary of a model's newer weights: ctl model canary, promote and canaries, rolled back by itself when it serves worse
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 20:31:26 +02:00
Ilker ManapandClaude Opus 5.5 0a4076928a Canary routing: a model's canary takes its share of requests, a session stays on one side, x-llmgw-variant pins one, and the two are compared
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 19:56:16 +02:00
Ilker ManapandClaude Opus 5.5 29a7347df2 Requests can be tagged: metadata, x-llmgw-tags and the user field on the usage record, and counted by the tags router.tag_labels names
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 19:45:37 +02:00
Ilker ManapandClaude Opus 5.5 e59ff52932 Merge features-c: section C of the backlog, items 1-5, and a hardened install
CI / test (push) Successful in 4m13s
Audit log and staff roles, OIDC and SCIM, key lifecycle, usage export,
notifications; server init writes baseline guardrails with two Llama Guard
judges failing closed, and keys of at most 90 days.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 19:20:41 +02:00
Ilker ManapandClaude Opus 5.5 c553f2041a The baseline fails closed with two judges, and a judge not answering is a notification
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 19:19:49 +02:00
Ilker ManapandClaude Opus 5.5 07af03aa93 server init: the firewall and the judge before the starter model, and a model that fits nowhere said within two minutes
A starter model with no room failed init after ten minutes and left the
firewall and the judge uninstalled.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 19:11:12 +02:00
Ilker ManapandClaude Opus 5.5 ddb0472f9b A hardened install by default: baseline guardrails with Llama Guard 3 1B judging on a CPU, keys at most 90 days
server init (and the setup screens) write guard.default [baseline] —
secrets refused, cards masked, injection recorded, the judge refusing
S1-S4, S7, S9-S12 in prompts and answers, fail open after 5s — fetch the
judge into the mirror and run it on a node's CPU. --no-guard-model leaves
it out. The closing text names what init cannot do: TLS on the client
port and backup recipients.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 17:45:31 +02:00
Ilker ManapandClaude Opus 5.5 6abd510628 Docs: one way in — quick-install starts at the setup screens, quickstart becomes development, install.md no longer says engines are yours to install
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 17:32:54 +02:00
Ilker ManapandClaude Opus 5.5 4979e08d64 Console: rotate a key, cap key lifetimes, test the webhooks; README, onboarding and console docs caught up
The README's status stopped at v0.9 and listed gaps since closed; onboarding
still said there was no SSO and no roles.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 17:28:24 +02:00
Ilker ManapandClaude Opus 5.5 1b71c38f89 Notifications: webhooks for a parked model, a lost node, a spent budget and new weights; Alertmanager receiver templates
notify.webhooks sends signed JSON or Slack-compatible messages, queued,
retried, de-duplicated hourly; ctl notify test checks each webhook.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 16:58:11 +02:00
Ilker ManapandClaude Opus 5.5 646ff996c7 gofmt the usage export
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 16:49:14 +02:00
Ilker ManapandClaude Opus 5.5 57739d736a Usage export as CSV: by period and by place, person, key and model, from the rollups
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 16:49:04 +02:00
Ilker ManapandClaude Opus 5.5 15a7237a02 Match a mapped project however its path is spaced: the store writes "Acme / ml / train"
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 16:45:47 +02:00
Ilker ManapandClaude Opus 5.5 8866a1ea0f Key lifecycle: lifetime limits by scope, rotation with overlap, kinds and expiry in the listing
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 16:45:05 +02:00
Ilker ManapandClaude Opus 5.5 178964cf6a gofmt
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 16:40:27 +02:00
Ilker ManapandClaude Opus 5.5 14ea59d83f Provisioning through SCIM 2.0: accounts and groups kept in step by the identity provider
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 16:40:15 +02:00
Ilker ManapandClaude Opus 5.5 5cff456b9b Let /login/oidc through without an API key: /login alone was exempt
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 16:37:24 +02:00
Ilker ManapandClaude Opus 5.5 dfe4093aea gofmt the OIDC pages
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 16:36:20 +02:00
Ilker ManapandClaude Opus 5.5 4cf40644c9 Single sign-on through OpenID Connect: accounts, staff roles and memberships from the provider
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 16:36:15 +02:00
Ilker ManapandClaude Opus 5.5 b60cf45adf Staff roles (viewer, operator, billing) and the console's Audit view
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 16:30:56 +02:00
Ilker ManapandClaude Opus 5.5 f8fcecd640 Audit log: every change through the admin API and the administration pages, with ctl audit
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 16:25:56 +02:00
Ilker ManapandClaude Opus 5.5 bcd2413535 Merge guardrails-b: section B of the backlog (guardrails)
CI / test (push) Successful in 2m37s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 16:09:48 +02:00
Ilker ManapandClaude Opus 5.5 4df30fc175 Record section B's state after trying it on the PoC
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 15:40:18 +02:00
Ilker ManapandClaude Opus 5.5 bd4736125b Do not name a judge model in the error a client gets for a model nobody serves
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 15:37:52 +02:00
Ilker ManapandClaude Opus 5.5 dbcecd1e85 Judge an answer with Llama Guard 3's own prompt, as a completion, not a conversation ending on the answer
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 15:35:57 +02:00
Ilker ManapandClaude Opus 5.5 742d6633e2 Read a judge's verdict after the answer llama.cpp goes on from, and log a verdict that cannot be read
Found on the PoC: llama.cpp takes a conversation ending on the assistant's
turn as a reply to continue, so Llama Guard answered "Rome.safe" and every
answer judged was served unjudged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 15:33:23 +02:00
Ilker ManapandClaude Opus 5.5 c69e6f151c Serve a judge model in one command: ctl model serve --guard-format
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 15:27:35 +02:00
Ilker ManapandClaude Opus 5.5 62038484d6 Never lose a word held back at one release when the next has nothing to rewrite
Found on the PoC: a streamed list came out as "1. 2. 3." — each item cut at
one release and dropped at the next, which sent the engine's own events.
A property test now cuts an answer every way and checks it arrives whole.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 15:24:22 +02:00
Ilker ManapandClaude Opus 5.5 2cc52fd1a7 Read an unwritten guard mode as enforce when merging, not as the weakest
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 15:22:07 +02:00
Ilker ManapandClaude Opus 5.5 83e1444981 Regenerate the configuration reference with the judge and the guard role
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 15:18:51 +02:00
Ilker ManapandClaude Opus 5.5 d55472c00a Guardrails in the console: attach and detach on a branch, where each applies from, and a column
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 15:18:42 +02:00
Ilker ManapandClaude Opus 5.5 02aac84ca5 The /admin/guard page, a Guardrails link, and the policies in force on the profile page
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 15:15:38 +02:00
Ilker ManapandClaude Opus 5.5 7e18b8ab53 Show guardrail refusals and stopped answers on the chat page, and the verdict in ctl request
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 15:13:15 +02:00
Ilker ManapandClaude Opus 5.5 47e847423d Guardrail metrics and alerts: checks, latency, judge failures, fail-open, redactions, stopped streams, tenant blocks
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 15:11:50 +02:00
Ilker ManapandClaude Opus 5.5 43f6a585a6 A judge model on the cluster: role guard in the catalog, judge in a guard policy
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 15:09:31 +02:00