bd4736125b
Do not name a judge model in the error a client gets for a model nobody serves
Ilker ManapandClaude Opus 5.52026-10-04 15:37:52 +02:00
dbcecd1e85
Judge an answer with Llama Guard 3's own prompt, as a completion, not a conversation ending on the answer
Ilker ManapandClaude Opus 5.52026-10-04 15:35:57 +02:00
742d6633e2
Read a judge's verdict after the answer llama.cpp goes on from, and log a verdict that cannot be read
Ilker ManapandClaude Opus 5.52026-10-04 15:33:23 +02:00
d55472c00a
Guardrails in the console: attach and detach on a branch, where each applies from, and a column
Ilker ManapandClaude Opus 5.52026-10-04 15:18:42 +02:00
02aac84ca5
The /admin/guard page, a Guardrails link, and the policies in force on the profile page
Ilker ManapandClaude Opus 5.52026-10-04 15:15:38 +02:00
7e18b8ab53
Show guardrail refusals and stopped answers on the chat page, and the verdict in ctl request
Ilker ManapandClaude Opus 5.52026-10-04 15:13:15 +02:00
2f07b2ba0b
Document guardrails, correct what data.md said leaves the cluster, and regenerate the reference
Ilker ManapandClaude Opus 5.52026-10-04 12:36:37 +02:00
ec737cffb5
no_store: refuse conversations, notes and models outside the cluster under a policy that keeps nothing
Ilker ManapandClaude Opus 5.52026-10-04 12:36:02 +02:00
13e6f19301
Attach guard policies to scopes: store, admin API and ctl guard; run the gate when server.yaml defines policies
Ilker ManapandClaude Opus 5.52026-10-04 12:34:47 +02:00
b4bdd250af
The gate: judge requests and answers by the policies in force, buffered or a sentence at a time
Ilker ManapandClaude Opus 5.52026-10-04 12:31:55 +02:00
ccdc07fe0c
Guard policies: a library in server.yaml, merged so nothing below loosens what is above
Ilker ManapandClaude Opus 5.52026-10-04 12:27:23 +02:00
ee33524599
Find personal data and secrets by validated pattern, and prompt injection through disguises
Ilker ManapandClaude Opus 5.52026-10-04 12:25:20 +02:00
979aa44804
Hook a guard into the router: judge requests before dispatch and filter answers as they are relayed
Ilker ManapandClaude Opus 5.52026-10-04 12:20:24 +02:00
e701f92806
Add the guard interface, and filter hooks in the proxy for buffered and streamed answers
Ilker ManapandClaude Opus 5.52026-10-04 12:16:00 +02:00
191c65e3db
Put etcd and Patroni's API on TLS, and move a running cluster there in maintenance mode
Ilker ManapandClaude Opus 5.52026-10-04 12:07:17 +02:00
bd155c18ac
Give the secrets directory its mode when init hands it to root, so the service can still read the store's address
Ilker ManapandClaude Opus 5.52026-10-04 11:39:49 +02:00
7b6276901a
Keep llmgw's internal ports to the cluster's networks with an nftables table of its own
Ilker ManapandClaude Opus 5.52026-10-04 11:03:46 +02:00
7e57863edd
Hide the mirror's bookkeeping, bound GGUF array nesting, and forget old serve jobs and downloads
Ilker ManapandClaude Opus 5.52026-10-04 10:56:16 +02:00
fdd4dcf841
Serve every page under a hash-based Content-Security-Policy, and update KaTeX to 0.16.47
Ilker ManapandClaude Opus 5.52026-10-04 10:54:04 +02:00
a56ad8e4c4
Keep the control plane's fetches off link-local addresses, and see through NAT64, 6to4 and proxies
Ilker ManapandClaude Opus 5.52026-10-04 10:48:01 +02:00
fbac1aa664
Take only torch from PyTorch's index, and lock its builds with every file's address
Ilker ManapandClaude Opus 5.52026-10-04 10:45:55 +02:00
3b05ec0538
Have engines listen on the management address, and take SGLang's key off its command line
Ilker ManapandClaude Opus 5.52026-10-04 10:36:42 +02:00
9b61bbc983
Revoke a removed node's certificate, and require --api-client with a mirror's mutual TLS
Ilker ManapandClaude Opus 5.52026-10-04 10:34:35 +02:00
44cd58005a
Count sign-in attempts per address and client, guard the sign-in form with a token, ration both forms per address
Ilker ManapandClaude Opus 5.52026-10-04 10:26:29 +02:00
f1b49d0251
Bound idle time, header size and request reading on every listener, and connections on the client port
Ilker ManapandClaude Opus 5.52026-10-04 10:21:15 +02:00
ebf437a36a
Have the setup's sudo grant remove itself within the hour, and keep a target from reading as an ssh option
Ilker ManapandClaude Opus 5.52026-10-04 10:17:52 +02:00
1f406247e4
Expire join codes on the setup's side, bind the join port to its address, trust the peer address
Ilker ManapandClaude Opus 5.52026-10-04 10:16:53 +02:00
20ebd21243
Skip capacity fallbacks the key may not use, and charge what the model that answered costs
Ilker ManapandClaude Opus 5.52026-10-04 10:13:59 +02:00
2b29016d5b
Ask for a password on Patroni's API and give the store TLS, new clusters and existing ones in place
Ilker ManapandClaude Opus 5.52026-10-04 10:11:28 +02:00
2d47427cdc
Run engines as llmgw-engine rather than root, and make the user when a node is added
Ilker ManapandClaude Opus 5.52026-10-04 09:57:41 +02:00
0bd5305e9a
Install only releases signed with the key built in; fetch a node's release on the control plane
Ilker ManapandClaude Opus 5.52026-10-04 09:48:03 +02:00
e01107d525
Write down what the review of 2026-10-04 found to do: security first, then guardrails, then features
Ilker ManapandClaude Fable 5.12026-10-04 09:38:01 +02:00
a1b1663e9d
A btop-like monitor in the console, from Prometheus; node exporters discovered and installed
Ilker ManapandClaude Opus 5.52026-10-04 01:51:05 +02:00
e12996f7dd
Serve and update models in the background from the console, and follow them under Jobs
Ilker ManapandClaude Opus 5.52026-10-04 01:27:02 +02:00
7cb9d4fa01
Busy is not unfit: rank files by the hardware the cluster has, and let a model wait for its cards
Ilker ManapandClaude Opus 5.52026-10-04 01:17:32 +02:00
c2be429554
Tab or down from a form's Apply goes round to the first field instead of crashing the console
Ilker ManapandClaude Opus 5.52026-10-04 01:12:19 +02:00
3b22ccce72
Merge fit-planner: plan a repository's files against the fleet, and serve the one that suits it
Ilker ManapandClaude Opus 5.52026-10-04 01:09:40 +02:00
0ea3008f64
On a CPU prefer a middling quantisation, on a card eight bits; past them a bigger file ranks lower
Ilker ManapandClaude Opus 5.52026-10-04 01:05:55 +02:00
54db7aecb7
Plan a repository's files against the fleet, and serve the one that suits it the way it fits
Ilker ManapandClaude Opus 5.52026-10-04 01:03:48 +02:00
a16097cfe8
Serve vision models with their projector and original weights on vLLM; group the mirror by version
Ilker ManapandClaude Opus 5.52026-10-04 00:42:59 +02:00
44425193bd
Track the registry's new weights for served models, and move a model to them or back
Ilker ManapandClaude Opus 5.52026-10-04 00:12:20 +02:00
a96b1a54b2
Hugging Face in the console: models that chat when browsing, ages read at a glance, long menus scroll
Ilker ManapandClaude Opus 5.52026-10-03 23:47:26 +02:00
6a7b4bb6a0
Merge tui-actions: everything settable on a running cluster, from the console
Ilker ManapandClaude Opus 5.52026-10-03 23:35:14 +02:00
eb7dac8a67
The console manages people, keys, organizations and the mirror, and scrolls a tall sidebar
Ilker ManapandClaude Opus 5.52026-10-03 23:30:24 +02:00
c7d5ee16b5
Act on the console's rows: scale, class, owner, resources, adapters, restart, remove; cordon, drain, engines, upgrade; quotas; cluster tasks
Ilker ManapandClaude Opus 5.52026-10-03 23:23:12 +02:00
36cdbce125
Merge ha: replicated store, second control plane, join codes, setup and console screens
Ilker ManapandClaude Opus 5.52026-10-03 22:59:49 +02:00