ilker/llmgw
2
0 Fork 1
Code Issues Pull Requests Actions Packages Projects Releases 95 Wiki Activity
433 Commits 9 Branches 95 Tags
dbcecd1e85aaa6be7eac4b704429a7a0a9440bd2
Commit Graph
433 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Ilker ManapandClaude Opus 5.5 dbcecd1e85 Judge an answer with Llama Guard 3's own prompt, as a completion, not a conversation ending on the answer 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 15:35:57 +02:00
Ilker ManapandClaude Opus 5.5 742d6633e2 Read a judge's verdict after the answer llama.cpp goes on from, and log a verdict that cannot be read 
Found on the PoC: llama.cpp takes a conversation ending on the assistant's
turn as a reply to continue, so Llama Guard answered "Rome.safe" and every
answer judged was served unjudged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 15:33:23 +02:00
Ilker ManapandClaude Opus 5.5 c69e6f151c Serve a judge model in one command: ctl model serve --guard-format 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 15:27:35 +02:00
Ilker ManapandClaude Opus 5.5 62038484d6 Never lose a word held back at one release when the next has nothing to rewrite 
Found on the PoC: a streamed list came out as "1. 2. 3." — each item cut at
one release and dropped at the next, which sent the engine's own events.
A property test now cuts an answer every way and checks it arrives whole.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 15:24:22 +02:00
Ilker ManapandClaude Opus 5.5 2cc52fd1a7 Read an unwritten guard mode as enforce when merging, not as the weakest 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 15:22:07 +02:00
Ilker ManapandClaude Opus 5.5 83e1444981 Regenerate the configuration reference with the judge and the guard role 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 15:18:51 +02:00
Ilker ManapandClaude Opus 5.5 d55472c00a Guardrails in the console: attach and detach on a branch, where each applies from, and a column 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 15:18:42 +02:00
Ilker ManapandClaude Opus 5.5 02aac84ca5 The /admin/guard page, a Guardrails link, and the policies in force on the profile page 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 15:15:38 +02:00
Ilker ManapandClaude Opus 5.5 7e18b8ab53 Show guardrail refusals and stopped answers on the chat page, and the verdict in ctl request 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 15:13:15 +02:00
Ilker ManapandClaude Opus 5.5 47e847423d Guardrail metrics and alerts: checks, latency, judge failures, fail-open, redactions, stopped streams, tenant blocks 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 15:11:50 +02:00
Ilker ManapandClaude Opus 5.5 43f6a585a6 A judge model on the cluster: role guard in the catalog, judge in a guard policy 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 15:09:31 +02:00
Ilker ManapandClaude Opus 5.5 0b4c4f0698 Put guard policies in the configuration reference 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:40:26 +02:00
Ilker ManapandClaude Opus 5.5 dc788e5cda Opt-in capability checks and structured-answer validation in guard policies 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:39:40 +02:00
Ilker ManapandClaude Opus 5.5 2f07b2ba0b Document guardrails, correct what data.md said leaves the cluster, and regenerate the reference 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:36:37 +02:00
Ilker ManapandClaude Opus 5.5 ec737cffb5 no_store: refuse conversations, notes and models outside the cluster under a policy that keeps nothing 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:36:02 +02:00
Ilker ManapandClaude Opus 5.5 13e6f19301 Attach guard policies to scopes: store, admin API and ctl guard; run the gate when server.yaml defines policies 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:34:47 +02:00
Ilker ManapandClaude Opus 5.5 b4bdd250af The gate: judge requests and answers by the policies in force, buffered or a sentence at a time 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:31:55 +02:00
Ilker ManapandClaude Opus 5.5 ccdc07fe0c Guard policies: a library in server.yaml, merged so nothing below loosens what is above 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:27:23 +02:00
Ilker ManapandClaude Opus 5.5 e3ef5b18c1 Test national phone numbers and bare digit runs 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:25:32 +02:00
Ilker ManapandClaude Opus 5.5 ee33524599 Find personal data and secrets by validated pattern, and prompt injection through disguises 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:25:20 +02:00
Ilker ManapandClaude Opus 5.5 9b9349183b gofmt the guard hooks 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:23:02 +02:00
Ilker Manap 7716d9a631 Merge main 2026-10-04 12:22:52 +02:00
Ilker ManapandClaude Opus 5.5 24d717282b gofmt what the security work left unformatted
CI / test (push) Successful in 4m7s
Details 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:22:51 +02:00
Ilker ManapandClaude Opus 5.5 979aa44804 Hook a guard into the router: judge requests before dispatch and filter answers as they are relayed 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:20:24 +02:00
Ilker ManapandClaude Opus 5.5 e701f92806 Add the guard interface, and filter hooks in the proxy for buffered and streamed answers 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:16:00 +02:00
Ilker ManapandClaude Opus 5.5 379074b6a5 Merge security-a: section A of the backlog (security)
CI / test (push) Failing after 17s
Details 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:12:54 +02:00
Ilker ManapandClaude Opus 5.5 eb10aee93b Document TLS between the store's machines, and close section A of the backlog 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:12:35 +02:00
Ilker ManapandClaude Opus 5.5 43d3c12302 Make a member's own access rules hostssl when the cluster keeps them there 
A cluster set up before the rules moved into the shared configuration kept
them in each member's config.yml, which the TLS step never looked at — and
then said its remote rules were hostssl. Found on the PoC: plaintext
connections were still accepted.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:11:05 +02:00
Ilker ManapandClaude Opus 5.5 191c65e3db Put etcd and Patroni's API on TLS, and move a running cluster there in maintenance mode 
Members' etcd speaks TLS to peers and clients with a certificate from the
cluster's authority and asks clients for one; Patroni reaches it with the
member's certificate and serves its API over TLS. A running plain-HTTP
cluster is moved with Patroni paused, so PostgreSQL serves throughout:
Patroni's lease-free keys are carried over to the new etcd.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:07:17 +02:00
Ilker ManapandClaude Opus 5.5 fed8c75c6c Do not warn of a missing firewall during the node add that writes it 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 11:58:18 +02:00
Ilker ManapandClaude Opus 5.5 855a34f0f7 Record the firewall as live on the PoC 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 11:52:28 +02:00
Ilker ManapandClaude Opus 5.5 efb4b06fc5 Report a ready engine dying from every connection, adopted ones included 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 11:50:09 +02:00
Ilker ManapandClaude Opus 5.5 bd155c18ac Give the secrets directory its mode when init hands it to root, so the service can still read the store's address 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 11:39:49 +02:00
Ilker ManapandClaude Opus 5.5 dae7afbc17 Record what of section A is done and what is left 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 11:05:10 +02:00
Ilker ManapandClaude Opus 5.5 7b6276901a Keep llmgw's internal ports to the cluster's networks with an nftables table of its own 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 11:03:46 +02:00
Ilker ManapandClaude Opus 5.5 7e57863edd Hide the mirror's bookkeeping, bound GGUF array nesting, and forget old serve jobs and downloads 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:56:16 +02:00
Ilker ManapandClaude Opus 5.5 fdd4dcf841 Serve every page under a hash-based Content-Security-Policy, and update KaTeX to 0.16.47 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:54:04 +02:00
Ilker ManapandClaude Opus 5.5 fd9f665ef8 End a session unused for a day, and purge dead sessions and claims hourly 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:50:56 +02:00
Ilker ManapandClaude Opus 5.5 ba5158d463 List in /v1/models only the models the key may use 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:49:54 +02:00
Ilker ManapandClaude Opus 5.5 f9493aadb9 Guard a GPU-less node's problems, and bound thread ids 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:49:10 +02:00
Ilker ManapandClaude Opus 5.5 a56ad8e4c4 Keep the control plane's fetches off link-local addresses, and see through NAT64, 6to4 and proxies 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:48:01 +02:00
Ilker ManapandClaude Opus 5.5 fbac1aa664 Take only torch from PyTorch's index, and lock its builds with every file's address 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:45:55 +02:00
Ilker ManapandClaude Opus 5.5 12efb7d085 Encrypt backups to backup_recipients with age 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:38:47 +02:00
Ilker ManapandClaude Opus 5.5 3b05ec0538 Have engines listen on the management address, and take SGLang's key off its command line 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:36:42 +02:00
Ilker ManapandClaude Opus 5.5 9b61bbc983 Revoke a removed node's certificate, and require --api-client with a mirror's mutual TLS 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:34:35 +02:00
Ilker ManapandClaude Opus 5.5 fe5b7f88f0 Believe X-Forwarded-* only from router.trusted_proxies 
A new internal/clientip says who sent a request and how: the connection's
address, scheme and Host, or through a configured proxy what it forwarded.
The base URL in pages and setup scripts, isHTTPS for cookies, and the
per-address limits of auth and the sign-in forms use it. /setup scripts are
no-store.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:29:02 +02:00
Ilker ManapandClaude Opus 5.5 44cd58005a Count sign-in attempts per address and client, guard the sign-in form with a token, ration both forms per address 
A wrong password from anywhere counted against the address, so ten locked
its owner out. Attempts are now counted per address and client, with a
looser bound per address alone. The sign-in form carries a token matched
by a cookie (login CSRF). POST /login and /signup are rationed per client
address, a hash waits at most ten seconds for a turn, and server init
writes signup_open: false.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:26:29 +02:00
Ilker ManapandClaude Opus 5.5 f1b49d0251 Bound idle time, header size and request reading on every listener, and connections on the client port 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:21:15 +02:00
Ilker ManapandClaude Opus 5.5 d3fb217b31 Bound the key cache and ration lookups of unknown keys per client address 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:19:38 +02:00
Ilker ManapandClaude Opus 5.5 ebf437a36a Have the setup's sudo grant remove itself within the hour, and keep a target from reading as an ssh option 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:17:52 +02:00