4721b8e353
Llama Guard 3's verdicts carry their hazards' names without a categories list
Ilker ManapandClaude Opus 5.52026-10-06 12:44:53 +02:00
249a91aeec
Guardrails turn off and on; a judge's refusal says which hazard; ctl guard judge ask
Ilker ManapandClaude Opus 5.52026-10-06 12:41:08 +02:00
f1a3199e55
KV routing tests wait for both replicas to be scraped: a replica not yet heard from is passed over, and the test was racing that
Ilker ManapandClaude Opus 5.52026-10-05 01:27:14 +02:00
117f417ef5
Precise KV-index routing: vLLM KV cache events through the agent, routed by leading blocks held
Ilker ManapandClaude Opus 5.52026-10-05 01:21:11 +02:00
0416b827fd
ctl bench reads a guarded engine's metrics with the engine's key, or says why it cannot
Ilker ManapandClaude Opus 5.52026-10-05 00:20:25 +02:00
8cc8d7139b
SGLang read in its own metric names, started serving them, and asked for its own help
Ilker ManapandClaude Opus 5.52026-10-05 00:20:25 +02:00
9f5a4bf11f
The about page's fonts carry Turkish letters: latin-ext subsets beside the latin ones
Ilker ManapandClaude Opus 5.52026-10-05 00:18:08 +02:00
a13a519386
CHANGELOG and BACKLOG for C17-C23 and the first parts of C24; README lists the new docs
Ilker ManapandClaude Opus 5.52026-10-05 00:17:16 +02:00
bae16426a2
Offline sites and Kubernetes: llmgw bundle create, import and trust, offline.mirror, server init --bundle; a Helm chart for the control plane
Ilker ManapandClaude Opus 5.52026-10-05 00:13:39 +02:00
11231d8c2b
The portal in English and Turkish: a catalogue per language, a switcher at the foot of every page, landing.language, Turkish numbers and dates
Ilker ManapandClaude Opus 5.52026-10-05 00:12:53 +02:00
f0477f140d
Quality suites, and new weights held to one: ctl eval, model update/promote --eval, model gate
Ilker ManapandClaude Opus 5.52026-10-05 00:04:43 +02:00
11b5d44702
A Terraform provider for llmgw: models, quotas and guard attachments through the admin API
Ilker ManapandClaude Opus 5.52026-10-05 00:04:35 +02:00
e725adc2f0
Tracing beyond the router: reconcile passes, placement and a replica's start traced down to the node
Ilker ManapandClaude Opus 5.52026-10-05 00:03:17 +02:00
0fd55a862b
MIG: a card in MIG mode is placed as its instances, each a GPU of class <card>-mig-<profile>
Ilker ManapandClaude Opus 5.52026-10-05 00:01:14 +02:00
44f3a9e31d
Reusing answers: a response cache per project, exact match, opt-in with a TTL
Ilker ManapandClaude Opus 5.52026-10-04 23:50:53 +02:00
a95c519bc5
The admin API described in OpenAPI 3.1: generated from a route table by docgen, served at /api/v1/openapi.json
Ilker ManapandClaude Opus 5.52026-10-04 23:47:59 +02:00
30f7f3474a
Forgetting a person and keeping requests in the cluster: users forget, keys residency --in-cluster
Ilker ManapandClaude Opus 5.52026-10-04 23:27:31 +02:00
35657c1417
Secrets in Vault: env: and vault: references wherever a secret file is named, and the store's own secrets sealed under a master key — Vault transit or a local key
Ilker ManapandClaude Opus 5.52026-10-04 23:21:44 +02:00
bce54f0941
Scaling by the clock: schedule windows with their own min and max, prewarm, control.timezone
Ilker ManapandClaude Opus 5.52026-10-04 23:16:56 +02:00
5b92ae0277
Token-level timing and admission by SLO: TPOT, inter-token and tokens a second by model and tenant; a model past its SLO is saturated for the lower bands
Ilker ManapandClaude Opus 5.52026-10-04 23:13:18 +02:00
10466416df
server restore: an archive checked, dry-run, and put back — configuration with modes and owners, the store in SQLite or PostgreSQL; backup_copy_to copies archives off the machine
Ilker ManapandClaude Opus 5.52026-10-04 23:09:59 +02:00
e502996198
Anthropic's Messages API: /v1/messages translated to chat completions and back, streamed or not, with tools and images; keys in x-api-key
Ilker ManapandClaude Opus 5.52026-10-04 22:59:55 +02:00
9136bc71c2
More of OpenAI's API: Responses structured output, images and previous_response_id; rerank, score and audio passed through; moderations by the judge
Ilker ManapandClaude Opus 5.52026-10-04 22:56:25 +02:00
32ff1dbd6f
Requests kept and replayed: keys requests per project with retention and a share, ctl request --body and --replay
Ilker ManapandClaude Opus 5.52026-10-04 22:51:21 +02:00
841ee21c12
Batches: OpenAI's Files and Batch APIs, served at the lowest priority under a store lease, each line a request of the key that made it
Ilker ManapandClaude Opus 5.52026-10-04 21:00:01 +02:00
a1dfd4df1d
Canary of a model's newer weights: ctl model canary, promote and canaries, rolled back by itself when it serves worse
Ilker ManapandClaude Opus 5.52026-10-04 20:31:26 +02:00
0a4076928a
Canary routing: a model's canary takes its share of requests, a session stays on one side, x-llmgw-variant pins one, and the two are compared
Ilker ManapandClaude Opus 5.52026-10-04 19:56:16 +02:00
29a7347df2
Requests can be tagged: metadata, x-llmgw-tags and the user field on the usage record, and counted by the tags router.tag_labels names
Ilker ManapandClaude Opus 5.52026-10-04 19:45:37 +02:00
e59ff52932
Merge features-c: section C of the backlog, items 1-5, and a hardened install
Ilker ManapandClaude Opus 5.52026-10-04 19:20:41 +02:00
c553f2041a
The baseline fails closed with two judges, and a judge not answering is a notification
Ilker ManapandClaude Opus 5.52026-10-04 19:19:49 +02:00
07af03aa93
server init: the firewall and the judge before the starter model, and a model that fits nowhere said within two minutes
Ilker ManapandClaude Opus 5.52026-10-04 19:11:12 +02:00
ddb0472f9b
A hardened install by default: baseline guardrails with Llama Guard 3 1B judging on a CPU, keys at most 90 days
Ilker ManapandClaude Opus 5.52026-10-04 17:45:31 +02:00
6abd510628
Docs: one way in — quick-install starts at the setup screens, quickstart becomes development, install.md no longer says engines are yours to install
Ilker ManapandClaude Opus 5.52026-10-04 17:32:54 +02:00
4979e08d64
Console: rotate a key, cap key lifetimes, test the webhooks; README, onboarding and console docs caught up
Ilker ManapandClaude Opus 5.52026-10-04 17:28:24 +02:00
1b71c38f89
Notifications: webhooks for a parked model, a lost node, a spent budget and new weights; Alertmanager receiver templates
Ilker ManapandClaude Opus 5.52026-10-04 16:58:11 +02:00
57739d736a
Usage export as CSV: by period and by place, person, key and model, from the rollups
Ilker ManapandClaude Opus 5.52026-10-04 16:49:04 +02:00
15a7237a02
Match a mapped project however its path is spaced: the store writes "Acme / ml / train"
Ilker ManapandClaude Opus 5.52026-10-04 16:45:47 +02:00
8866a1ea0f
Key lifecycle: lifetime limits by scope, rotation with overlap, kinds and expiry in the listing
Ilker ManapandClaude Opus 5.52026-10-04 16:45:05 +02:00
14ea59d83f
Provisioning through SCIM 2.0: accounts and groups kept in step by the identity provider
Ilker ManapandClaude Opus 5.52026-10-04 16:40:15 +02:00
4cf40644c9
Single sign-on through OpenID Connect: accounts, staff roles and memberships from the provider
Ilker ManapandClaude Opus 5.52026-10-04 16:36:15 +02:00
f8fcecd640
Audit log: every change through the admin API and the administration pages, with ctl audit
Ilker ManapandClaude Opus 5.52026-10-04 16:25:56 +02:00
