Ilker ManapandClaude Opus 5.5 fa6e68c8e8 A model put first is not evicted, idle, for a lower class ...

The VLM is to come first on the one card. Its class makes it take the card
from flux, triposr or sf3d when asked for, but idle eviction ignored class:
a 3D job two minutes into a lull would have taken the card from it. Idle
eviction offers only replicas of no higher class than the model wanting
room, and the wait such a model is told runs to the holder's idle timeout.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

2026-10-13 10:15:14 +02:00

..

account

A forgotten password is set through a one-time link

2026-10-07 13:20:11 +02:00

accounting

A ComfyUI model may make meshes: from a picture, with its licence inside

2026-10-10 18:30:52 +02:00

agent

The agent hands the gate the repository a node loads its weights by

2026-10-12 23:28:22 +02:00

audit

Guardrails turn off and on; a judge's refusal says which hazard; ctl guard judge ask

2026-10-06 12:41:08 +02:00

auth

Anthropic's Messages API: /v1/messages translated to chat completions and back, streamed or not, with tools and images; keys in x-api-key

2026-10-04 22:59:55 +02:00

bench

ctl bench reads a guarded engine's metrics with the engine's key, or says why it cannot

2026-10-05 00:20:25 +02:00

bundle

Offline sites and Kubernetes: llmgw bundle create, import and trust, offline.mirror, server init --bundle; a Helm chart for the control plane

2026-10-05 00:13:39 +02:00

canary

Canary routing: a model's canary takes its share of requests, a session stays on one side, x-llmgw-variant pins one, and the two are compared

2026-10-04 19:56:16 +02:00

cli

G4: a page that makes 3D models, and ctl 3d

2026-10-10 19:16:20 +02:00

clientapi

Any ComfyUI workflow runs as a job, from the inputs its catalog entry declares

2026-10-11 14:28:15 +02:00

clientip

Believe X-Forwarded-* only from router.trusted_proxies

2026-10-04 10:29:02 +02:00

comfy

A custom node's compiled packages are built for the card that runs them

2026-10-11 17:23:05 +02:00

config

A ComfyUI model may make meshes: from a picture, with its licence inside

2026-10-10 18:30:52 +02:00

control

A model put first is not evicted, idle, for a lower class

2026-10-13 10:15:14 +02:00

docgen

A custom node's compiled packages are built for the card that runs them

2026-10-11 17:23:05 +02:00

doctor

A node's own readiness and its multi-node readiness are kept apart

2026-10-07 13:11:16 +02:00

firewall

The firewall setting reaches the mirror machine and the store's members

2026-10-09 08:53:17 +02:00

gguf

Hide the mirror's bookkeeping, bound GGUF array nesting, and forget old serve jobs and downloads

2026-10-04 10:56:16 +02:00

guard

Qwen3Guard is the judge, and judges on a GPU and on CPUs are asked in turn

2026-10-08 10:22:47 +02:00

hf

Plan a repository's files against the fleet, and serve the one that suits it the way it fits

2026-10-04 01:03:48 +02:00

join

Bound idle time, header size and request reading on every listener, and connections on the client port

2026-10-04 10:21:15 +02:00

kb

llmgw-kb collections names each collection's owner as people know it

2026-10-08 15:42:40 +02:00

kvevents

Precise KV-index routing: vLLM KV cache events through the agent, routed by leading blocks held

2026-10-05 01:21:11 +02:00

mail

A forgotten password is set through a one-time link

2026-10-07 13:20:11 +02:00

mirror

The firewall setting reaches the mirror machine and the store's members

2026-10-09 08:53:17 +02:00

modelplan

MIG: a card in MIG mode is placed as its instances, each a GPU of class <card>-mig-<profile>

2026-10-05 00:01:14 +02:00

notify

Two control planes send a notification once

2026-10-10 13:36:08 +02:00

obs

Tracing beyond the router: reconcile passes, placement and a replica's start traced down to the node

2026-10-05 00:03:17 +02:00

onboard

A node's own readiness and its multi-node readiness are kept apart

2026-10-07 13:11:16 +02:00

pgha

The store's PostgreSQL sized to its machine's memory

2026-10-08 21:48:53 +02:00

preflight

Open setup screens or a console when llmgw runs bare in a terminal

2026-10-03 21:33:59 +02:00

promq

Show usage over time, per organization, team, project and person

2026-09-28 08:11:31 +02:00

release

Offline sites and Kubernetes: llmgw bundle create, import and trust, offline.mirror, server init --bundle; a Helm chart for the control plane

2026-10-05 00:13:39 +02:00

router

A picture in a prompt counts as what a model makes of it, not its base64

2026-10-11 20:45:06 +02:00

safedial

gofmt what the security work left unformatted

2026-10-04 12:22:51 +02:00

scim

Match a mapped project however its path is spaced: the store writes "Acme / ml / train"

2026-10-04 16:45:47 +02:00

secrets

A test against a real Vault, when one is named

2026-10-04 23:22:36 +02:00

settings

The cluster's settings live in the store, and every change takes effect

2026-10-05 15:25:23 +02:00

sim

Onboard a node with a script, and find two examples that never worked

2026-09-16 12:28:36 +02:00

sourcecheck

Keep the control plane's fetches off link-local addresses, and see through NAT64, 6to4 and proxies

2026-10-04 10:48:01 +02:00

store

A person leaves licences out, and no key of theirs reaches a model under one

2026-10-11 16:10:22 +02:00

tlsconf

Put etcd and Patroni's API on TLS, and move a running cluster there in maintenance mode

2026-10-04 12:07:17 +02:00

version

Switch the project language to English

2026-09-16 01:15:21 +02:00

web

A person leaves licences out, and no key of theirs reaches a model under one

2026-10-11 16:10:22 +02:00