Ilker ManapandClaude Opus 5.5 26630274f4 Passages of one organization never share a vLLM prefix cache with another's ...

A request given passages carries a cache_salt derived from its organization
when the replica chosen is vLLM, so how fast an answer starts cannot say what
another tenant's documents held. Other engines are not sent the field.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

2026-10-08 16:02:19 +02:00

..

account

A forgotten password is set through a one-time link

2026-10-07 13:20:11 +02:00

accounting

Notifications: webhooks for a parked model, a lost node, a spent budget and new weights; Alertmanager receiver templates

2026-10-04 16:58:11 +02:00

agent

An adopted engine that turns ready after registration is reported

2026-10-07 19:43:13 +02:00

audit

Guardrails turn off and on; a judge's refusal says which hazard; ctl guard judge ask

2026-10-06 12:41:08 +02:00

auth

Anthropic's Messages API: /v1/messages translated to chat completions and back, streamed or not, with tools and images; keys in x-api-key

2026-10-04 22:59:55 +02:00

bench

ctl bench reads a guarded engine's metrics with the engine's key, or says why it cannot

2026-10-05 00:20:25 +02:00

bundle

Offline sites and Kubernetes: llmgw bundle create, import and trust, offline.mirror, server init --bundle; a Helm chart for the control plane

2026-10-05 00:13:39 +02:00

canary

Canary routing: a model's canary takes its share of requests, a session stays on one side, x-llmgw-variant pins one, and the two are compared

2026-10-04 19:56:16 +02:00

cli

Collections a part of the tree answers from when a request names none

2026-10-08 15:52:43 +02:00

clientip

Believe X-Forwarded-* only from router.trusted_proxies

2026-10-04 10:29:02 +02:00

config

llmgw stop, start and restart act on the whole system

2026-10-07 18:01:45 +02:00

control

Collections a part of the tree answers from when a request names none

2026-10-08 15:52:43 +02:00

docgen

Qwen3Guard is the judge, and judges on a GPU and on CPUs are asked in turn

2026-10-08 10:22:47 +02:00

doctor

A node's own readiness and its multi-node readiness are kept apart

2026-10-07 13:11:16 +02:00

firewall

The firewall turns off and on with the cluster's settings; llmgw wipe

2026-10-06 08:00:06 +02:00

gguf

Hide the mirror's bookkeeping, bound GGUF array nesting, and forget old serve jobs and downloads

2026-10-04 10:56:16 +02:00

guard

Qwen3Guard is the judge, and judges on a GPU and on CPUs are asked in turn

2026-10-08 10:22:47 +02:00

hf

Plan a repository's files against the fleet, and serve the one that suits it the way it fits

2026-10-04 01:03:48 +02:00

join

Bound idle time, header size and request reading on every listener, and connections on the client port

2026-10-04 10:21:15 +02:00

kb

llmgw-kb collections names each collection's owner as people know it

2026-10-08 15:42:40 +02:00

kvevents

Precise KV-index routing: vLLM KV cache events through the agent, routed by leading blocks held

2026-10-05 01:21:11 +02:00

mail

A forgotten password is set through a one-time link

2026-10-07 13:20:11 +02:00

mirror

Merge C18 offline bundles and the Helm chart

2026-10-05 00:15:47 +02:00

modelplan

MIG: a card in MIG mode is placed as its instances, each a GPU of class <card>-mig-<profile>

2026-10-05 00:01:14 +02:00

notify

Secrets in Vault: env: and vault: references wherever a secret file is named, and the store's own secrets sealed under a master key — Vault transit or a local key

2026-10-04 23:21:44 +02:00

obs

Tracing beyond the router: reconcile passes, placement and a replica's start traced down to the node

2026-10-05 00:03:17 +02:00

onboard

A node's own readiness and its multi-node readiness are kept apart

2026-10-07 13:11:16 +02:00

pgha

Knowledge bases in the store: collections, grants and chunks kept apart per tenant

2026-10-08 12:47:57 +02:00

preflight

Open setup screens or a console when llmgw runs bare in a terminal

2026-10-03 21:33:59 +02:00

promq

Show usage over time, per organization, team, project and person

2026-09-28 08:11:31 +02:00

release

Offline sites and Kubernetes: llmgw bundle create, import and trust, offline.mirror, server init --bundle; a Helm chart for the control plane

2026-10-05 00:13:39 +02:00

router

Passages of one organization never share a vLLM prefix cache with another's

2026-10-08 16:02:19 +02:00

safedial

gofmt what the security work left unformatted

2026-10-04 12:22:51 +02:00

scim

Match a mapped project however its path is spaced: the store writes "Acme / ml / train"

2026-10-04 16:45:47 +02:00

secrets

A test against a real Vault, when one is named

2026-10-04 23:22:36 +02:00

settings

The cluster's settings live in the store, and every change takes effect

2026-10-05 15:25:23 +02:00

sim

Onboard a node with a script, and find two examples that never worked

2026-09-16 12:28:36 +02:00

sourcecheck

Keep the control plane's fetches off link-local addresses, and see through NAT64, 6to4 and proxies

2026-10-04 10:48:01 +02:00

store

Collections a part of the tree answers from when a request names none

2026-10-08 15:52:43 +02:00

tlsconf

Put etcd and Patroni's API on TLS, and move a running cluster there in maintenance mode

2026-10-04 12:07:17 +02:00

version

Switch the project language to English

2026-09-16 01:15:21 +02:00

web

The portal's chat answers from the documents a person picks

2026-10-08 15:59:23 +02:00