ilker/llmgw
2
0 Fork 1
Code Issues Pull Requests Actions Packages Projects Releases 95 Wiki Activity
427 Commits 9 Branches 95 Tags
d55472c00a636f725d1ade38c0809042fd2936f4
Commit Graph
427 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Ilker ManapandClaude Opus 5.5 d55472c00a Guardrails in the console: attach and detach on a branch, where each applies from, and a column 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 15:18:42 +02:00
Ilker ManapandClaude Opus 5.5 02aac84ca5 The /admin/guard page, a Guardrails link, and the policies in force on the profile page 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 15:15:38 +02:00
Ilker ManapandClaude Opus 5.5 7e18b8ab53 Show guardrail refusals and stopped answers on the chat page, and the verdict in ctl request 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 15:13:15 +02:00
Ilker ManapandClaude Opus 5.5 47e847423d Guardrail metrics and alerts: checks, latency, judge failures, fail-open, redactions, stopped streams, tenant blocks 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 15:11:50 +02:00
Ilker ManapandClaude Opus 5.5 43f6a585a6 A judge model on the cluster: role guard in the catalog, judge in a guard policy 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 15:09:31 +02:00
Ilker ManapandClaude Opus 5.5 0b4c4f0698 Put guard policies in the configuration reference 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:40:26 +02:00
Ilker ManapandClaude Opus 5.5 dc788e5cda Opt-in capability checks and structured-answer validation in guard policies 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:39:40 +02:00
Ilker ManapandClaude Opus 5.5 2f07b2ba0b Document guardrails, correct what data.md said leaves the cluster, and regenerate the reference 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:36:37 +02:00
Ilker ManapandClaude Opus 5.5 ec737cffb5 no_store: refuse conversations, notes and models outside the cluster under a policy that keeps nothing 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:36:02 +02:00
Ilker ManapandClaude Opus 5.5 13e6f19301 Attach guard policies to scopes: store, admin API and ctl guard; run the gate when server.yaml defines policies 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:34:47 +02:00
Ilker ManapandClaude Opus 5.5 b4bdd250af The gate: judge requests and answers by the policies in force, buffered or a sentence at a time 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:31:55 +02:00
Ilker ManapandClaude Opus 5.5 ccdc07fe0c Guard policies: a library in server.yaml, merged so nothing below loosens what is above 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:27:23 +02:00
Ilker ManapandClaude Opus 5.5 e3ef5b18c1 Test national phone numbers and bare digit runs 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:25:32 +02:00
Ilker ManapandClaude Opus 5.5 ee33524599 Find personal data and secrets by validated pattern, and prompt injection through disguises 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:25:20 +02:00
Ilker ManapandClaude Opus 5.5 9b9349183b gofmt the guard hooks 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:23:02 +02:00
Ilker Manap 7716d9a631 Merge main 2026-10-04 12:22:52 +02:00
Ilker ManapandClaude Opus 5.5 24d717282b gofmt what the security work left unformatted
CI / test (push) Successful in 4m7s
Details 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:22:51 +02:00
Ilker ManapandClaude Opus 5.5 979aa44804 Hook a guard into the router: judge requests before dispatch and filter answers as they are relayed 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:20:24 +02:00
Ilker ManapandClaude Opus 5.5 e701f92806 Add the guard interface, and filter hooks in the proxy for buffered and streamed answers 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:16:00 +02:00
Ilker ManapandClaude Opus 5.5 379074b6a5 Merge security-a: section A of the backlog (security)
CI / test (push) Failing after 17s
Details 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:12:54 +02:00
Ilker ManapandClaude Opus 5.5 eb10aee93b Document TLS between the store's machines, and close section A of the backlog 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:12:35 +02:00
Ilker ManapandClaude Opus 5.5 43d3c12302 Make a member's own access rules hostssl when the cluster keeps them there 
A cluster set up before the rules moved into the shared configuration kept
them in each member's config.yml, which the TLS step never looked at — and
then said its remote rules were hostssl. Found on the PoC: plaintext
connections were still accepted.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:11:05 +02:00
Ilker ManapandClaude Opus 5.5 191c65e3db Put etcd and Patroni's API on TLS, and move a running cluster there in maintenance mode 
Members' etcd speaks TLS to peers and clients with a certificate from the
cluster's authority and asks clients for one; Patroni reaches it with the
member's certificate and serves its API over TLS. A running plain-HTTP
cluster is moved with Patroni paused, so PostgreSQL serves throughout:
Patroni's lease-free keys are carried over to the new etcd.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 12:07:17 +02:00
Ilker ManapandClaude Opus 5.5 fed8c75c6c Do not warn of a missing firewall during the node add that writes it 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 11:58:18 +02:00
Ilker ManapandClaude Opus 5.5 855a34f0f7 Record the firewall as live on the PoC 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 11:52:28 +02:00
Ilker ManapandClaude Opus 5.5 efb4b06fc5 Report a ready engine dying from every connection, adopted ones included 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 11:50:09 +02:00
Ilker ManapandClaude Opus 5.5 bd155c18ac Give the secrets directory its mode when init hands it to root, so the service can still read the store's address 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 11:39:49 +02:00
Ilker ManapandClaude Opus 5.5 dae7afbc17 Record what of section A is done and what is left 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 11:05:10 +02:00
Ilker ManapandClaude Opus 5.5 7b6276901a Keep llmgw's internal ports to the cluster's networks with an nftables table of its own 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 11:03:46 +02:00
Ilker ManapandClaude Opus 5.5 7e57863edd Hide the mirror's bookkeeping, bound GGUF array nesting, and forget old serve jobs and downloads 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:56:16 +02:00
Ilker ManapandClaude Opus 5.5 fdd4dcf841 Serve every page under a hash-based Content-Security-Policy, and update KaTeX to 0.16.47 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:54:04 +02:00
Ilker ManapandClaude Opus 5.5 fd9f665ef8 End a session unused for a day, and purge dead sessions and claims hourly 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:50:56 +02:00
Ilker ManapandClaude Opus 5.5 ba5158d463 List in /v1/models only the models the key may use 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:49:54 +02:00
Ilker ManapandClaude Opus 5.5 f9493aadb9 Guard a GPU-less node's problems, and bound thread ids 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:49:10 +02:00
Ilker ManapandClaude Opus 5.5 a56ad8e4c4 Keep the control plane's fetches off link-local addresses, and see through NAT64, 6to4 and proxies 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:48:01 +02:00
Ilker ManapandClaude Opus 5.5 fbac1aa664 Take only torch from PyTorch's index, and lock its builds with every file's address 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:45:55 +02:00
Ilker ManapandClaude Opus 5.5 12efb7d085 Encrypt backups to backup_recipients with age 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:38:47 +02:00
Ilker ManapandClaude Opus 5.5 3b05ec0538 Have engines listen on the management address, and take SGLang's key off its command line 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:36:42 +02:00
Ilker ManapandClaude Opus 5.5 9b61bbc983 Revoke a removed node's certificate, and require --api-client with a mirror's mutual TLS 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:34:35 +02:00
Ilker ManapandClaude Opus 5.5 fe5b7f88f0 Believe X-Forwarded-* only from router.trusted_proxies 
A new internal/clientip says who sent a request and how: the connection's
address, scheme and Host, or through a configured proxy what it forwarded.
The base URL in pages and setup scripts, isHTTPS for cookies, and the
per-address limits of auth and the sign-in forms use it. /setup scripts are
no-store.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:29:02 +02:00
Ilker ManapandClaude Opus 5.5 44cd58005a Count sign-in attempts per address and client, guard the sign-in form with a token, ration both forms per address 
A wrong password from anywhere counted against the address, so ten locked
its owner out. Attempts are now counted per address and client, with a
looser bound per address alone. The sign-in form carries a token matched
by a cookie (login CSRF). POST /login and /signup are rationed per client
address, a hash waits at most ten seconds for a turn, and server init
writes signup_open: false.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:26:29 +02:00
Ilker ManapandClaude Opus 5.5 f1b49d0251 Bound idle time, header size and request reading on every listener, and connections on the client port 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:21:15 +02:00
Ilker ManapandClaude Opus 5.5 d3fb217b31 Bound the key cache and ration lookups of unknown keys per client address 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:19:38 +02:00
Ilker ManapandClaude Opus 5.5 ebf437a36a Have the setup's sudo grant remove itself within the hour, and keep a target from reading as an ssh option 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:17:52 +02:00
Ilker ManapandClaude Opus 5.5 1f406247e4 Expire join codes on the setup's side, bind the join port to its address, trust the peer address 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:16:53 +02:00
Ilker ManapandClaude Opus 5.5 27c49d0af9 Refuse the keys of an account that is not active 
Resolve now checks the owner's state for a personal key; the authenticator
treats an inactive owner as a revoked key. Suspension stops the keys within
the cache TTL, reinstatement brings them back.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:15:32 +02:00
Ilker ManapandClaude Opus 5.5 20ebd21243 Skip capacity fallbacks the key may not use, and charge what the model that answered costs 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:13:59 +02:00
Ilker ManapandClaude Opus 5.5 b201275fd9 Take agent_tls.insecure as consent to an agent port without TLS too 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:11:40 +02:00
Ilker ManapandClaude Opus 5.5 2b29016d5b Ask for a password on Patroni's API and give the store TLS, new clusters and existing ones in place 
Patroni's REST API accepted changes from anyone on port 8008, and
PostgreSQL ran with sslmode=disable between machines. Setup now keeps a
fourth secret for the API (carried to members of older clusters), issues
each member a certificate from the cluster's authority, writes
restapi.authentication into each member's config and reloads it, and
turns ssl on with hostssl remote rules through an authenticated PATCH of
the shared configuration. The DSN becomes sslmode=verify-full against the
authority's certificate whenever the cluster has TLS, so a control plane
without the authority's key reconnects too. server add authenticates its
PATCH and lets the new control plane in by hostssl. Tried on the e2e
cluster: unauthenticated PATCH 401, plaintext refused, replication
streaming, the control plane on verify-full.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:11:28 +02:00
Ilker ManapandClaude Opus 5.5 9d961c3fc8 Say that the admin API wants its token, and that ctl runs with sudo 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 10:02:56 +02:00