deniz/llmgw
1
0 Fork 0
forked from ilker/llmgw
Code Pull Requests Activity
457 Commits 2 Branches 5 Tags
a1dfd4df1dcc4eee1253f2f66ff3b6be3af5f97f
Commit Graph
385 Commits
Author SHA1 Message Date
Ilker ManapandClaude Opus 5.5 a1dfd4df1d Canary of a model's newer weights: ctl model canary, promote and canaries, rolled back by itself when it serves worse 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 20:31:26 +02:00
Ilker ManapandClaude Opus 5.5 0a4076928a Canary routing: a model's canary takes its share of requests, a session stays on one side, x-llmgw-variant pins one, and the two are compared 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 19:56:16 +02:00
Ilker ManapandClaude Opus 5.5 29a7347df2 Requests can be tagged: metadata, x-llmgw-tags and the user field on the usage record, and counted by the tags router.tag_labels names 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 19:45:37 +02:00
Ilker ManapandClaude Opus 5.5 c553f2041a The baseline fails closed with two judges, and a judge not answering is a notification 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 19:19:49 +02:00
Ilker ManapandClaude Opus 5.5 07af03aa93 server init: the firewall and the judge before the starter model, and a model that fits nowhere said within two minutes 
A starter model with no room failed init after ten minutes and left the
firewall and the judge uninstalled.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 19:11:12 +02:00
Ilker ManapandClaude Opus 5.5 ddb0472f9b A hardened install by default: baseline guardrails with Llama Guard 3 1B judging on a CPU, keys at most 90 days 
server init (and the setup screens) write guard.default [baseline] —
secrets refused, cards masked, injection recorded, the judge refusing
S1-S4, S7, S9-S12 in prompts and answers, fail open after 5s — fetch the
judge into the mirror and run it on a node's CPU. --no-guard-model leaves
it out. The closing text names what init cannot do: TLS on the client
port and backup recipients.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 17:45:31 +02:00
Ilker ManapandClaude Opus 5.5 6abd510628 Docs: one way in — quick-install starts at the setup screens, quickstart becomes development, install.md no longer says engines are yours to install 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 17:32:54 +02:00
Ilker ManapandClaude Opus 5.5 4979e08d64 Console: rotate a key, cap key lifetimes, test the webhooks; README, onboarding and console docs caught up 
The README's status stopped at v0.9 and listed gaps since closed; onboarding
still said there was no SSO and no roles.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 17:28:24 +02:00
Ilker ManapandClaude Opus 5.5 1b71c38f89 Notifications: webhooks for a parked model, a lost node, a spent budget and new weights; Alertmanager receiver templates 
notify.webhooks sends signed JSON or Slack-compatible messages, queued,
retried, de-duplicated hourly; ctl notify test checks each webhook.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 16:58:11 +02:00
Ilker ManapandClaude Opus 5.5 646ff996c7 gofmt the usage export 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 16:49:14 +02:00
Ilker ManapandClaude Opus 5.5 57739d736a Usage export as CSV: by period and by place, person, key and model, from the rollups 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 16:49:04 +02:00
Ilker ManapandClaude Opus 5.5 15a7237a02 Match a mapped project however its path is spaced: the store writes "Acme / ml / train" 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 16:45:47 +02:00
Ilker ManapandClaude Opus 5.5 8866a1ea0f Key lifecycle: lifetime limits by scope, rotation with overlap, kinds and expiry in the listing 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 16:45:05 +02:00
Ilker ManapandClaude Opus 5.5 178964cf6a gofmt 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 16:40:27 +02:00
Ilker ManapandClaude Opus 5.5 14ea59d83f Provisioning through SCIM 2.0: accounts and groups kept in step by the identity provider 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 16:40:15 +02:00
Ilker ManapandClaude Opus 5.5 5cff456b9b Let /login/oidc through without an API key: /login alone was exempt 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 16:37:24 +02:00
Ilker ManapandClaude Opus 5.5 dfe4093aea gofmt the OIDC pages 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 16:36:20 +02:00
Ilker ManapandClaude Opus 5.5 4cf40644c9 Single sign-on through OpenID Connect: accounts, staff roles and memberships from the provider 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 16:36:15 +02:00
Ilker ManapandClaude Opus 5.5 b60cf45adf Staff roles (viewer, operator, billing) and the console's Audit view 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 16:30:56 +02:00
Ilker ManapandClaude Opus 5.5 f8fcecd640 Audit log: every change through the admin API and the administration pages, with ctl audit 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 16:25:56 +02:00
Ilker ManapandClaude Opus 5.5 bd4736125b Do not name a judge model in the error a client gets for a model nobody serves 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 15:37:52 +02:00
Ilker ManapandClaude Opus 5.5 dbcecd1e85 Judge an answer with Llama Guard 3's own prompt, as a completion, not a conversation ending on the answer 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 15:35:57 +02:00
Ilker ManapandClaude Opus 5.5 742d6633e2 Read a judge's verdict after the answer llama.cpp goes on from, and log a verdict that cannot be read 
Found on the PoC: llama.cpp takes a conversation ending on the assistant's
turn as a reply to continue, so Llama Guard answered "Rome.safe" and every
answer judged was served unjudged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 15:33:23 +02:00
Ilker ManapandClaude Opus 5.5 c69e6f151c Serve a judge model in one command: ctl model serve --guard-format 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 15:27:35 +02:00
Ilker ManapandClaude Opus 5.5 62038484d6 Never lose a word held back at one release when the next has nothing to rewrite 
Found on the PoC: a streamed list came out as "1. 2. 3." — each item cut at
one release and dropped at the next, which sent the engine's own events.
A property test now cuts an answer every way and checks it arrives whole.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 15:24:22 +02:00
Ilker ManapandClaude Opus 5.5 2cc52fd1a7 Read an unwritten guard mode as enforce when merging, not as the weakest 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 15:22:07 +02:00
Ilker ManapandClaude Opus 5.5 83e1444981 Regenerate the configuration reference with the judge and the guard role 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 15:18:51 +02:00
Ilker ManapandClaude Opus 5.5 d55472c00a Guardrails in the console: attach and detach on a branch, where each applies from, and a column 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 15:18:42 +02:00
Ilker ManapandClaude Opus 5.5 02aac84ca5 The /admin/guard page, a Guardrails link, and the policies in force on the profile page 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 15:15:38 +02:00
Ilker ManapandClaude Opus 5.5 7e18b8ab53 Show guardrail refusals and stopped answers on the chat page, and the verdict in ctl request 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 15:13:15 +02:00
Ilker ManapandClaude Opus 5.5 47e847423d Guardrail metrics and alerts: checks, latency, judge failures, fail-open, redactions, stopped streams, tenant blocks 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 15:11:50 +02:00
Ilker ManapandClaude Opus 5.5 43f6a585a6 A judge model on the cluster: role guard in the catalog, judge in a guard policy 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 15:09:31 +02:00
Ilker ManapandClaude Opus 5.5 0b4c4f0698 Put guard policies in the configuration reference 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:40:26 +02:00
Ilker ManapandClaude Opus 5.5 dc788e5cda Opt-in capability checks and structured-answer validation in guard policies 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:39:40 +02:00
Ilker ManapandClaude Opus 5.5 2f07b2ba0b Document guardrails, correct what data.md said leaves the cluster, and regenerate the reference 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:36:37 +02:00
Ilker ManapandClaude Opus 5.5 ec737cffb5 no_store: refuse conversations, notes and models outside the cluster under a policy that keeps nothing 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:36:02 +02:00
Ilker ManapandClaude Opus 5.5 13e6f19301 Attach guard policies to scopes: store, admin API and ctl guard; run the gate when server.yaml defines policies 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:34:47 +02:00
Ilker ManapandClaude Opus 5.5 b4bdd250af The gate: judge requests and answers by the policies in force, buffered or a sentence at a time 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:31:55 +02:00
Ilker ManapandClaude Opus 5.5 ccdc07fe0c Guard policies: a library in server.yaml, merged so nothing below loosens what is above 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:27:23 +02:00
Ilker ManapandClaude Opus 5.5 e3ef5b18c1 Test national phone numbers and bare digit runs 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:25:32 +02:00
Ilker ManapandClaude Opus 5.5 ee33524599 Find personal data and secrets by validated pattern, and prompt injection through disguises 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:25:20 +02:00
Ilker ManapandClaude Opus 5.5 9b9349183b gofmt the guard hooks 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:23:02 +02:00
Ilker Manap 7716d9a631 Merge main 2026-10-04 12:22:52 +02:00
Ilker ManapandClaude Opus 5.5 24d717282b gofmt what the security work left unformatted 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:22:51 +02:00
Ilker ManapandClaude Opus 5.5 979aa44804 Hook a guard into the router: judge requests before dispatch and filter answers as they are relayed 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:20:24 +02:00
Ilker ManapandClaude Opus 5.5 e701f92806 Add the guard interface, and filter hooks in the proxy for buffered and streamed answers 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:16:00 +02:00
Ilker ManapandClaude Opus 5.5 43d3c12302 Make a member's own access rules hostssl when the cluster keeps them there 
A cluster set up before the rules moved into the shared configuration kept
them in each member's config.yml, which the TLS step never looked at — and
then said its remote rules were hostssl. Found on the PoC: plaintext
connections were still accepted.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:11:05 +02:00
Ilker ManapandClaude Opus 5.5 191c65e3db Put etcd and Patroni's API on TLS, and move a running cluster there in maintenance mode 
Members' etcd speaks TLS to peers and clients with a certificate from the
cluster's authority and asks clients for one; Patroni reaches it with the
member's certificate and serves its API over TLS. A running plain-HTTP
cluster is moved with Patroni paused, so PostgreSQL serves throughout:
Patroni's lease-free keys are carried over to the new etcd.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:07:17 +02:00
Ilker ManapandClaude Opus 5.5 fed8c75c6c Do not warn of a missing firewall during the node add that writes it 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 11:58:18 +02:00
Ilker ManapandClaude Opus 5.5 efb4b06fc5 Report a ready engine dying from every connection, adopted ones included 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 11:50:09 +02:00