Ilker ManapandClaude Opus 5.5 8547528fb7 Prices from what the cluster costs: ctl prices suggest ...

auth.gpu_hour_cost (and core_hour_cost) is what an hour of a GPU (a core)
costs the operator. The leader keeps, by the hour, the GPU- and core-hours
each model's replicas hold (held_1h); llmgw ctl prices suggest spreads a
model's over the tokens it served, an output token counted as four input
ones and a cached prompt token as a tenth of one, and --apply writes the
prices into auth.pricing. Idle hours are in the cost, so a model held and
little used comes out dear; one held and billed nothing is shown with what
it held.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

2026-11-15 06:54:55 +01:00

..

admintoken_test.go

Guard the admin API and pprof with a token only root and the server can read

2026-10-04 10:02:44 +02:00

admintoken.go

Guard the admin API and pprof with a token only root and the server can read

2026-10-04 10:02:44 +02:00

agent.go

The nodes' agent settings kept with the cluster's, agent.yaml down to what an agent needs to start

2026-10-08 18:06:36 +02:00

answercache_test.go

Reusing answers: a response cache per project, exact match, opt-in with a TTL

2026-10-04 23:50:53 +02:00

answercache.go

Answers to similar questions, not only the same one

2026-10-25 17:47:11 +01:00

audit.go

Audit log: every change through the admin API and the administration pages, with ctl audit

2026-10-04 16:25:56 +02:00

backup_test.go

Backups reach a replicated store: pg_dump is given the cluster's authority with verify-full

2026-10-08 21:04:46 +02:00

backup.go

Backups reach a replicated store: pg_dump is given the cluster's authority with verify-full

2026-10-08 21:04:46 +02:00

backupcopy.go

server restore: an archive checked, dry-run, and put back — configuration with modes and owners, the store in SQLite or PostgreSQL; backup_copy_to copies archives off the machine

2026-10-04 23:09:59 +02:00

bench_overhead_test.go

What llmgw adds to a request, measured: ctl bench overhead

2026-10-17 00:23:26 +02:00

bench_overhead.go

What llmgw adds to a request, measured: ctl bench overhead

2026-10-17 00:23:26 +02:00

bench_test.go

ctl bench reads a guarded engine's metrics with the engine's key, or says why it cannot

2026-10-05 00:20:25 +02:00

bench.go

What llmgw adds to a request, measured: ctl bench overhead

2026-10-17 00:23:26 +02:00

bundle.go

The cluster's settings live in the store, and every change takes effect

2026-10-05 15:25:23 +02:00

capacity.go

Keep what a benchmark found out, instead of leaving it in a terminal

2026-09-27 11:51:58 +02:00

cert.go

Add a private CA and mutual TLS for the agent stream

2026-09-16 07:28:56 +02:00

clustersettings_test.go

The cluster's settings live in the store, and every change takes effect

2026-10-05 15:25:23 +02:00

clustersettings.go

Slurm beside llmgw, P3 and P4: the leader hands nodes over; ctl slurm

2026-11-11 12:33:14 +01:00

comfyuigate.go

ComfyUI as an engine of the cluster's own

2026-10-10 00:39:10 +02:00

controlplanes_test.go

The probe reads the floating address from its file each time

2026-10-05 02:56:23 +02:00

controlplanes.go

The probe reads the floating address from its file each time

2026-10-05 02:56:23 +02:00

ctl_3d.go

G4: a page that makes 3D models, and ctl 3d

2026-10-10 19:16:20 +02:00

ctl_config_test.go

The guardrails' judge has its own command, and its trouble is shown

2026-10-06 10:47:37 +02:00

ctl_config.go

The cluster's settings live in the store, and every change takes effect

2026-10-05 15:25:23 +02:00

ctl_eval_test.go

Quality suites, and new weights held to one: ctl eval, model update/promote --eval, model gate

2026-10-05 00:04:43 +02:00

ctl_eval.go

Quality suites, and new weights held to one: ctl eval, model update/promote --eval, model gate

2026-10-05 00:04:43 +02:00

ctl_fabrictest.go

Measure two nodes' fabric rail by rail: llmgw ctl fabric-test

2026-09-29 21:29:23 +02:00

ctl_logs.go

Show a node's logs on the admin page

2026-09-28 12:27:47 +02:00

ctl_mcp.go

The company's MCP tools under the guardrails, a rate and a breaker

2026-10-20 23:00:22 +02:00

ctl_prices.go

Prices from what the cluster costs: ctl prices suggest

2026-11-15 06:54:55 +01:00

ctl_request.go

A ComfyUI model may make meshes: from a picture, with its licence inside

2026-10-10 18:30:52 +02:00

ctl_restart.go

ctl restart --max-surge and --max-unavailable

2026-10-07 13:09:10 +02:00

ctl_slurm_test.go

Slurm beside llmgw, P3 and P4: the leader hands nodes over; ctl slurm

2026-11-11 12:33:14 +01:00

ctl_slurm.go

Slurm's jobs in containers: pyxis and enroot, installed by the agents

2026-11-14 21:31:20 +01:00

ctl_status_test.go

Status says when the cluster's alerts go nowhere

2026-11-12 22:49:37 +01:00

ctl_upgrade_test.go

Cluster upgrades: nodes told together with progress said, done when back on the version, the download kept past a broken stream

2026-10-30 11:14:02 +01:00

ctl_upgrade.go

Cluster upgrades: nodes told together with progress said, done when back on the version, the download kept past a broken stream

2026-10-30 11:14:02 +01:00

ctl.go

Prices from what the cluster costs: ctl prices suggest

2026-11-15 06:54:55 +01:00

ctlweights.go

Deleting unused model files says what went, and sets go whole

2026-11-03 10:29:16 +01:00

dispatcherref_test.go

Measure two nodes' fabric rail by rail: llmgw ctl fabric-test

2026-09-29 21:29:23 +02:00

doctor.go

A node's own readiness and its multi-node readiness are kept apart

2026-10-07 13:11:16 +02:00

enginebuild.go

Choose each engine's build from the node's hardware, for every GPU generation

2026-10-01 14:35:58 +02:00

enginecheck_gpu_test.go

An adopted engine's state is written without its command; engine-check finds the installed build

2026-10-07 14:26:28 +02:00

enginecheck_test.go

SGLang read in its own metric names, started serving them, and asked for its own help

2026-10-05 00:20:25 +02:00

enginecheck.go

FreeToken, an engine for models larger than the card

2026-10-31 22:10:05 +01:00

facility_test.go

Facility-aware placement: racks, PDUs, holds and power caps

2026-10-05 01:17:49 +02:00

facility.go

Facility-aware placement: racks, PDUs, holds and power caps

2026-10-05 01:17:49 +02:00

federation.go

Federation: another llmgw cluster as a provider, its models as PEER/MODEL

2026-10-05 01:21:30 +02:00

filter_test.go

Let ctl ask a question instead of printing the whole fleet

2026-09-27 07:30:51 +02:00

filter.go

Facility-aware placement: racks, PDUs, holds and power caps

2026-10-05 01:17:49 +02:00

firewallcmd_test.go

The firewall setting reaches the mirror machine and the store's members

2026-10-09 08:53:17 +02:00

firewallcmd.go

The firewall setting reaches the mirror machine and the store's members

2026-10-09 08:53:17 +02:00

firewallpub_test.go

The firewall setting reaches the mirror machine and the store's members

2026-10-09 08:53:17 +02:00

firewallpub.go

The firewall setting reaches the mirror machine and the store's members

2026-10-09 08:53:17 +02:00

guard_judge.go

How fast the judge is, measured, and said at install

2026-10-20 22:35:19 +02:00

guard.go

Guardrails turn off and on; a judge's refusal says which hazard; ctl guard judge ask

2026-10-06 12:41:08 +02:00

install_lines_test.go

Found on a fresh three-machine install: what the screens said wrong

2026-11-12 10:38:18 +01:00

join.go

Offline sites and Kubernetes: llmgw bundle create, import and trust, offline.mirror, server init --bundle; a Helm chart for the control plane

2026-10-05 00:13:39 +02:00

judgekeeper_test.go

The judge init installs is no longer replaced by the leader's own

2026-10-19 20:37:10 +02:00

judgekeeper.go

The judge init installs is no longer replaced by the leader's own

2026-10-19 20:37:10 +02:00

judges_known.go

A judge the leader serves by itself is sized as init's is

2026-10-19 16:32:04 +02:00

kb.go

Training data from a collection: llmgw ctl kb dataset

2026-10-08 20:04:37 +02:00

keys_lifecycle_test.go

Key lifecycle: lifetime limits by scope, rotation with overlap, kinds and expiry in the listing

2026-10-04 16:45:05 +02:00

keys.go

keys limits changes only the limits it is given

2026-10-20 17:32:43 +02:00

leaderforward_test.go

Standbys that answer the client API themselves, behind router.active_active

2026-11-01 13:47:47 +01:00

leaderforward.go

Standbys that answer the client API themselves, behind router.active_active

2026-11-01 13:47:47 +01:00

memory.go

Manage people and the tenancy tree, from the web and the CLI

2026-09-28 16:26:39 +02:00

mig_test.go

MIG: a card in MIG mode is placed as its instances, each a GPU of class <card>-mig-<profile>

2026-10-05 00:01:14 +02:00

mirror_gate_test.go

Release v0.10.4: CPU threads, tracing, and the rest of the review

2026-09-30 04:00:35 +02:00

mirror_import_test.go

TensorRT-LLM as an engine, and models from Ollama into the mirror

2026-10-05 00:02:01 +02:00

mirror_import.go

TensorRT-LLM as an engine, and models from Ollama into the mirror

2026-10-05 00:02:01 +02:00

mirror_peer_test.go

mirror add-peer on a mirror machine of its own says what to run on a control plane

2026-10-29 18:30:30 +01:00

mirror_peer.go

mirror add-peer on a mirror machine of its own says what to run on a control plane

2026-10-29 18:30:30 +01:00

mirror_test.go

Hide the mirror's bookkeeping, bound GGUF array nesting, and forget old serve jobs and downloads

2026-10-04 10:56:16 +02:00

mirror.go

Weights from Hugging Face pinned to a commit and checked against the sha256 it published

2026-10-30 07:33:15 +01:00

mirrorinit.go

A second mirror: llmgw mirror add-peer, and on a store machine at setup

2026-10-29 13:43:27 +01:00

model_canary.go

Quality suites, and new weights held to one: ctl eval, model update/promote --eval, model gate

2026-10-05 00:04:43 +02:00

model_presets_test.go

The cluster page and the console say which picture and 3D models are here, and add them

2026-10-13 20:48:40 +02:00

model_presets.go

The cluster page and the console say which picture and 3D models are here, and add them

2026-10-13 20:48:40 +02:00

model_serve.go

CHANGELOG: one list

2026-10-30 10:39:30 +01:00

model_update.go

Quality suites, and new weights held to one: ctl eval, model update/promote --eval, model gate

2026-10-05 00:04:43 +02:00

model.go

Found on a fresh three-machine install: what the screens said wrong

2026-11-12 10:38:18 +01:00

modelplan.go

How fast a model answers on the CPU, before it is served

2026-11-03 08:52:15 +01:00

movestore.go

Move a store to another one without reissuing every key

2026-09-25 16:28:38 +02:00

node.go

A machine given by address is reached through its ssh config Host

2026-11-12 10:42:23 +01:00

notify.go

Notifications: webhooks for a parked model, a lost node, a spent budget and new weights; Alertmanager receiver templates

2026-10-04 16:58:11 +02:00

org_test.go

keys limits changes only the limits it is given

2026-10-20 17:32:43 +02:00

org.go

Let a person belong to many places, or to none

2026-09-28 19:03:46 +02:00

quota.go

Set capacity quotas, class, owner and reservation from the admin pages

2026-09-30 22:40:45 +02:00

release.go

Sign releases, and install only signed ones once a key is trusted

2026-09-30 23:03:48 +02:00

requestlog.go

Forgetting a person and keeping requests in the cluster: users forget, keys residency --in-cluster

2026-10-04 23:27:31 +02:00

resolve.go

Manage people and the tenancy tree, from the web and the CLI

2026-09-28 16:26:39 +02:00

restore_test.go

server restore: an archive checked, dry-run, and put back — configuration with modes and owners, the store in SQLite or PostgreSQL; backup_copy_to copies archives off the machine

2026-10-04 23:09:59 +02:00

restore.go

server restore: an archive checked, dry-run, and put back — configuration with modes and owners, the store in SQLite or PostgreSQL; backup_copy_to copies archives off the machine

2026-10-04 23:09:59 +02:00

root.go

Strata runs as an engine: a 125B MoE on one card

2026-10-29 15:09:13 +01:00

selfupgrade.go

An upgrade that leaves nodes behind says which

2026-10-14 09:01:09 +02:00

servable_test.go

Refuse weights the nodes' llama.cpp cannot load, before fetching them

2026-09-28 20:44:36 +02:00

server_agent_test.go

Take agent_tls.insecure as consent to an agent port without TLS too

2026-10-04 10:11:40 +02:00

server_args_test.go

A broken directory setting, or a second server, no longer takes the cluster down

2026-10-19 11:10:43 +02:00

server.go

Prices from what the cluster costs: ctl prices suggest

2026-11-15 06:54:55 +01:00

serveradd.go

Found on a fresh three-machine install: what the screens said wrong

2026-11-12 10:38:18 +01:00

serverinit_bundle_test.go

Offline sites and Kubernetes: llmgw bundle create, import and trust, offline.mirror, server init --bundle; a Helm chart for the control plane

2026-10-05 00:13:39 +02:00

serverinit_bundle.go

The cluster's settings live in the store, and every change takes effect

2026-10-05 15:25:23 +02:00

serverinit_choices_test.go

Slurm's shared files and users: one path everywhere, accounts as Linux users

2026-11-12 21:34:34 +01:00

serverinit_choices.go

Slurm's shared files and users: one path everywhere, accounts as Linux users

2026-11-12 21:34:34 +01:00

serverinit_cluster_test.go

A three-machine install can give the judges machines of their own

2026-10-19 17:45:00 +02:00

serverinit_cluster.go

Found on a fresh three-machine install: what the screens said wrong

2026-11-12 10:38:18 +01:00

serverinit_firewall.go

The firewall setting reaches the mirror machine and the store's members

2026-10-09 08:53:17 +02:00

serverinit_judges_test.go

A three-machine install can give the judges machines of their own

2026-10-19 17:45:00 +02:00

serverinit_judges.go

How fast the judge is, measured, and said at install

2026-10-20 22:35:19 +02:00

serverinit_memory_test.go

Fixes from the second review: grants, flags, updates, setup

2026-10-29 12:31:07 +01:00

serverinit_notify_test.go

Setup asks where alerts go, and sends each a test

2026-10-14 13:55:48 +02:00

serverinit_notify.go

Setup asks where alerts go, and sends each a test

2026-10-14 13:55:48 +02:00

serverinit_patroni.go

Setup lets every part be chosen, each with its settings; three machines each run the gateway

2026-10-15 11:22:36 +02:00

serverinit_postgres_test.go

Find the PostgreSQL a machine has, rather than assume its names

2026-09-28 14:47:55 +02:00

serverinit_postgres.go

The store's PostgreSQL sized to its machine's memory

2026-10-08 21:48:53 +02:00

serverinit_standby.go

Setup lets every part be chosen, each with its settings; three machines each run the gateway

2026-10-15 11:22:36 +02:00

serverinit_test.go

The mirror's address is one the nodes reach, and node add checks it

2026-10-06 09:36:09 +02:00

serverinit.go

Slurm's shared files and users: one path everywhere, accounts as Linux users

2026-11-12 21:34:34 +01:00

service_test.go

llmgw start --wait waits for the models, not for the setting

2026-10-07 21:43:45 +02:00

service.go

llmgw stop, start and restart act on the whole system

2026-10-07 18:01:45 +02:00

simengine.go

Onboard a node with a script, and find two examples that never worked

2026-09-16 12:28:36 +02:00

sourcenote_test.go

Say when a replica is running an edited profile, and check a source from the web too

2026-09-27 22:42:26 +02:00

sourcenote.go

Say when a replica is running an edited profile, and check a source from the web too

2026-09-27 22:42:26 +02:00

standbyroutes_test.go

Rate limits counted across the control planes with router.active_active

2026-11-01 17:30:08 +01:00

standbyroutes.go

Rate limits counted across the control planes with router.active_active

2026-11-01 17:30:08 +01:00

status_test.go

An on-demand model that is up reads as ready, not as not placed

2026-10-10 01:01:55 +02:00

status.go

Status says when the cluster's alerts go nowhere

2026-11-12 22:49:37 +01:00

storeflag_test.go

Stop calling the PostgreSQL store SQLite, and stop guessing where it is

2026-09-27 12:20:40 +02:00

storeflag.go

Stop calling the PostgreSQL store SQLite, and stop guessing where it is

2026-09-27 12:20:40 +02:00

strataserve.go

Strata runs as an engine: a 125B MoE on one card

2026-10-29 15:09:13 +01:00

system.go

llmgw start --wait waits for the models, not for the setting

2026-10-07 21:43:45 +02:00

threads.go

Manage people and the tenancy tree, from the web and the CLI

2026-09-28 16:26:39 +02:00

toolcheck_test.go

llmgw ctl tool-check asks with the engine's key

2026-10-08 11:24:37 +02:00

toolcheck.go

llmgw ctl tool-check asks with the engine's key

2026-10-08 11:24:37 +02:00

train.go

Trained adapters published, private to their owners like the knowledge bases they come from

2026-10-08 21:30:50 +02:00

tui_actions_test.go

Canary on the console's model menu; docs

2026-10-04 20:33:59 +02:00

tui_actions.go

Deleting unused model files says what went, and sets go whole

2026-11-03 10:29:16 +01:00

tui_audit.go

Usage export as CSV: by period and by place, person, key and model, from the rollups

2026-10-04 16:49:04 +02:00

tui_chart.go

Place chart points by their time, so a window's samples fill the chart's width

2026-10-04 08:55:19 +02:00

tui_choices.go

Slurm's shared files and users: one path everywhere, accounts as Linux users

2026-11-12 21:34:34 +01:00

tui_console.go

Slurm in the console: each node, and lend or take back from its row

2026-11-11 22:53:26 +01:00

tui_docs.go

A docs portal at /docs and a client developer's guide

2026-10-04 23:46:42 +02:00

tui_federation_test.go

Federation: another llmgw cluster as a provider, its models as PEER/MODEL

2026-10-05 01:21:30 +02:00

tui_federation.go

Federation: another llmgw cluster as a provider, its models as PEER/MODEL

2026-10-05 01:21:30 +02:00

tui_form.go

Setup lets every part be chosen, each with its settings; three machines each run the gateway

2026-10-15 11:22:36 +02:00

tui_frame.go

Show the version top right on every screen

2026-10-04 01:18:45 +02:00

tui_guard.go

Guardrails in the console

2026-10-05 03:36:17 +02:00

tui_hf.go

The serve forms start at the longest context that fits

2026-11-08 14:38:41 +01:00

tui_home.go

Watch the cluster and edit every setting from the console

2026-10-03 22:54:09 +02:00

tui_init_test.go

Setup lets every part be chosen, each with its settings; three machines each run the gateway

2026-10-15 11:22:36 +02:00

tui_init.go

Setup lets every part be chosen, each with its settings; three machines each run the gateway

2026-10-15 11:22:36 +02:00

tui_jobs.go

Serve and update models in the background from the console, and follow them under Jobs

2026-10-04 01:27:02 +02:00

tui_kb_test.go

The console's Knowledge screen, and the manual's page on knowledge bases

2026-10-08 13:58:12 +02:00

tui_kb.go

Collections a part of the tree answers from when a request names none

2026-10-08 15:52:43 +02:00

tui_mcp_test.go

The company's MCP servers in the console

2026-10-28 13:23:25 +01:00

tui_mcp.go

Fixes from the second review: grants, flags, updates, setup

2026-10-29 12:31:07 +01:00

tui_mirror.go

The control plane uses whichever mirror answers

2026-10-29 13:43:27 +01:00

tui_monitor.go

How much of each GPU works, beside whether anything does

2026-10-31 20:11:26 +01:00

tui_node_overview_test.go

A node's console opens on what the machine is doing

2026-10-19 12:55:27 +02:00

tui_node_overview.go

A node's console opens on what the machine is doing

2026-10-19 12:55:27 +02:00

tui_node_test.go

Let server init ask for what it needs, and node add take several nodes

2026-09-29 21:54:18 +02:00

tui_node.go

Add a standby control plane with server add, and draw the screens as windows

2026-10-03 22:29:12 +02:00

tui_people.go

An organization's admin, and forms only for those who may send them

2026-11-08 11:38:06 +01:00

tui_presets_test.go

The cluster page and the console say which picture and 3D models are here, and add them

2026-10-13 20:48:40 +02:00

tui_presets.go

The cluster page and the console say which picture and 3D models are here, and add them

2026-10-13 20:48:40 +02:00

tui_quality_test.go

Quality suites, and new weights held to one: ctl eval, model update/promote --eval, model gate

2026-10-05 00:04:43 +02:00

tui_quality.go

Quality suites, and new weights held to one: ctl eval, model update/promote --eval, model gate

2026-10-05 00:04:43 +02:00

tui_setup_test.go

Setup lets every part be chosen, each with its settings; three machines each run the gateway

2026-10-15 11:22:36 +02:00

tui_setup.go

A machine too small for the models it would serve is said so first

2026-10-25 16:19:00 +01:00

tui_signin_test.go

Sign-in is set up from the console, and a directory tried before it is kept

2026-10-19 12:55:27 +02:00

tui_signin.go

Sign-in is set up from the console, and a directory tried before it is kept

2026-10-19 12:55:27 +02:00

tui_slurm_test.go

Slurm's shared files and users: one path everywhere, accounts as Linux users

2026-11-12 21:34:34 +01:00

tui_slurm.go

Slurm's jobs in containers: pyxis and enroot, installed by the agents

2026-11-14 21:31:20 +01:00

tui_widgets.go

Open setup screens or a console when llmgw runs bare in a terminal

2026-10-03 21:33:59 +02:00

usage.go

Usage export as CSV: by period and by place, person, key and model, from the rollups

2026-10-04 16:49:04 +02:00

users.go

Suspending a person says their keys are refused, which they are

2026-11-12 22:38:54 +01:00

wipe_test.go

The firewall turns off and on with the cluster's settings; llmgw wipe

2026-10-06 08:00:06 +02:00

wipe.go

The firewall turns off and on with the cluster's settings; llmgw wipe

2026-10-06 08:00:06 +02:00