Ilker ManapandClaude Opus 5.5 78e41460f9 Scale and place by what a replica was measured to serve ...

Stored ctl bench results reach the reconciler: requests per replica past
the measured SLO concurrency scale a model up, a replica is taken back only
if the rest stay under 70% of theirs, and among profiles of equal priority
the most measured tokens per GPU or core is tried first. ctl model target
sizes a model for a throughput against the room there is.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

2026-09-30 18:21:08 +02:00

..

agent.go

Add llmgw doctor: what a node's GPUs, NVLink and fabric need, checked

2026-09-29 03:51:09 +02:00

backup_test.go

Close what a second security review found still open

2026-09-30 02:25:07 +02:00

backup.go

Close what a second security review found still open

2026-09-30 02:25:07 +02:00

bench.go

Give every engine a key only the router holds

2026-09-30 00:38:29 +02:00

capacity.go

Keep what a benchmark found out, instead of leaving it in a terminal

2026-09-27 11:51:58 +02:00

cert.go

Add a private CA and mutual TLS for the agent stream

2026-09-16 07:28:56 +02:00

ctl_fabrictest.go

Measure two nodes' fabric rail by rail: llmgw ctl fabric-test

2026-09-29 21:29:23 +02:00

ctl_logs.go

Show a node's logs on the admin page

2026-09-28 12:27:47 +02:00

ctl_request.go

Follow one request from its id to the replica that served it

2026-09-30 02:36:45 +02:00

ctl_restart.go

Park a model that fails every time it starts, and say so everywhere

2026-09-30 01:21:57 +02:00

ctl_status_test.go

Read a queued change as a success in ctl

2026-09-30 01:14:12 +02:00

ctl_upgrade_test.go

Upgrade the cluster with one command, over the agents' own streams

2026-09-30 02:54:36 +02:00

ctl_upgrade.go

Upgrade the cluster with one command, over the agents' own streams

2026-09-30 02:54:36 +02:00

ctl.go

Scale and place by what a replica was measured to serve

2026-09-30 18:21:08 +02:00

ctlweights.go

Carry a download's progress to where somebody is looking at it

2026-09-27 07:49:08 +02:00

dispatcherref_test.go

Measure two nodes' fabric rail by rail: llmgw ctl fabric-test

2026-09-29 21:29:23 +02:00

doctor.go

Install vLLM on every GPU node that can run it, as llama.cpp is

2026-09-29 13:15:54 +02:00

enginecheck_test.go

Ship binaries with releases, and ask engines whether our flags still exist

2026-09-16 16:44:29 +02:00

enginecheck.go

Check llmgw's engine flags against real vLLM and llama.cpp

2026-09-16 16:52:11 +02:00

filter_test.go

Let ctl ask a question instead of printing the whole fleet

2026-09-27 07:30:51 +02:00

filter.go

Let ctl ask a question instead of printing the whole fleet

2026-09-27 07:30:51 +02:00

keys.go

Let a person belong to many places, or to none

2026-09-28 19:03:46 +02:00

memory.go

Manage people and the tenancy tree, from the web and the CLI

2026-09-28 16:26:39 +02:00

mirror_gate_test.go

Release v0.10.4: CPU threads, tracing, and the rest of the review

2026-09-30 04:00:35 +02:00

mirror_test.go

Say when a replica is running an edited profile, and check a source from the web too

2026-09-27 22:42:26 +02:00

mirror.go

Release v0.10.4: CPU threads, tracing, and the rest of the review

2026-09-30 04:00:35 +02:00

mirrorinit.go

Put the mirror on its own machine in two commands

2026-09-29 22:17:24 +02:00

model_serve.go

Reserve CPU cores and require memory for CPU models

2026-09-30 17:43:13 +02:00

model.go

Scale and place by what a replica was measured to serve

2026-09-30 18:21:08 +02:00

modelplan.go

Make the planned entry use an engine that can serve the precision

2026-09-17 07:48:20 +02:00

movestore.go

Move a store to another one without reissuing every key

2026-09-25 16:28:38 +02:00

node.go

Let server init ask for what it needs, and node add take several nodes

2026-09-29 21:54:18 +02:00

org_test.go

Let a person belong to many places, or to none

2026-09-28 19:03:46 +02:00

org.go

Let a person belong to many places, or to none

2026-09-28 19:03:46 +02:00

quota.go

Scale and place by what a replica was measured to serve

2026-09-30 18:21:08 +02:00

resolve.go

Manage people and the tenancy tree, from the web and the CLI

2026-09-28 16:26:39 +02:00

root.go

Add a node with one command: llmgw node add, with a terminal screen

2026-09-29 19:33:47 +02:00

servable_test.go

Refuse weights the nodes' llama.cpp cannot load, before fetching them

2026-09-28 20:44:36 +02:00

server.go

Scale and place by what a replica was measured to serve

2026-09-30 18:21:08 +02:00

serverinit_cluster_test.go

Put the mirror on its own machine in two commands

2026-09-29 22:17:24 +02:00

serverinit_cluster.go

Put the mirror on its own machine in two commands

2026-09-29 22:17:24 +02:00

serverinit_postgres_test.go

Find the PostgreSQL a machine has, rather than assume its names

2026-09-28 14:47:55 +02:00

serverinit_postgres.go

Find the PostgreSQL a machine has, rather than assume its names

2026-09-28 14:47:55 +02:00

serverinit_test.go

Let a person belong to many places, or to none

2026-09-28 19:03:46 +02:00

serverinit.go

Put the mirror on its own machine in two commands

2026-09-29 22:17:24 +02:00

simengine.go

Onboard a node with a script, and find two examples that never worked

2026-09-16 12:28:36 +02:00

sourcenote_test.go

Say when a replica is running an edited profile, and check a source from the web too

2026-09-27 22:42:26 +02:00

sourcenote.go

Say when a replica is running an edited profile, and check a source from the web too

2026-09-27 22:42:26 +02:00

status_test.go

Make the planned entry use an engine that can serve the precision

2026-09-17 07:48:20 +02:00

status.go

Park a model that fails every time it starts, and say so everywhere

2026-09-30 01:21:57 +02:00

storeflag_test.go

Stop calling the PostgreSQL store SQLite, and stop guessing where it is

2026-09-27 12:20:40 +02:00

storeflag.go

Stop calling the PostgreSQL store SQLite, and stop guessing where it is

2026-09-27 12:20:40 +02:00

threads.go

Manage people and the tenancy tree, from the web and the CLI

2026-09-28 16:26:39 +02:00

toolcheck_test.go

Stop tool-check blaming the engine for what the model did

2026-09-18 08:31:09 +02:00

toolcheck.go

Stop tool-check blaming the engine for what the model did

2026-09-18 08:31:09 +02:00

tui_init_test.go

Put the mirror on its own machine in two commands

2026-09-29 22:17:24 +02:00

tui_init.go

Release v0.10.4: CPU threads, tracing, and the rest of the review

2026-09-30 04:00:35 +02:00

tui_node_test.go

Let server init ask for what it needs, and node add take several nodes

2026-09-29 21:54:18 +02:00

tui_node.go

Print only what must outlive init's screen, and drop bubbles

2026-09-29 22:37:28 +02:00

tui_widgets.go

Print only what must outlive init's screen, and drop bubbles

2026-09-29 22:37:28 +02:00

usage.go

Show what a group spent, and fix the reports that never ran on PostgreSQL

2026-09-27 12:26:14 +02:00

users.go

Give an organization's managers a page of their own

2026-09-28 19:23:12 +02:00