llmgw/docs/console.md

The console

sudo llmgw, with nothing after it, on a machine llmgw runs on. On a machine with nothing installed it opens the setup screens; on a control plane, the console described here; on a node, the node's own console, which edits agent.yaml. Everything on these screens can also be done by command — each screen says which — so a script never needs them and a person never needs to remember the commands.

The keyboard drives it; there is no mouse. ? opens this page from anywhere, and the Manual entry holds every other page, as llmgw docs does without the console.

The layout

A bar across the top names the machine. Down the left, the entries, in four groups: Watch, People, Help, Settings and Do. The panel on the right shows the entry chosen. The bottom line always says which keys do what at that moment, and a question or a result opens as a dialog over the panel.

Key	Does
↑ ↓	Choose an entry, or a row in the panel
→ or Enter	Go into the panel
Enter on a row	What can be done to it: its details first, then the rest
← or Esc	Back out: from a page to its list, from the panel to the entries
A	Add one, on a view that holds things: a quota, an account, a key
PgUp PgDn	Scroll a long panel
?	This page
Q	Leave; the cluster keeps running
Ctrl-C	Leave at once

What is watched is read again every three seconds.

Forms and questions

A change that needs values opens a form, filled in with what is set now. Tab or ↓ moves to the next field, ← → changes a choice, and Enter on Apply sends it; a form of one field is sent by Enter on it. A mistake the form can see — a maximum below the minimum, a word where a number goes — is said in the form and nothing is sent. One the control plane refuses comes back as a dialog with its reason, word for word.

Anything that cannot be undone — removing, draining, deleting, revoking — asks first, and only Y goes ahead.

Tasks that take a while and print as they go — serving a model, upgrading the cluster, a backup, adding a node — close the screen and run in the terminal, so their output is there to read; Enter brings the console back.

Watch

Overview is llmgw ctl status: nodes, GPUs by class, fabrics, replicas, models, the store's members, and anything parked, unplaced or idle.

Nodes lists every node with its state, load, GPUs, replicas, disk and agent version. On a row:

• Details — its address, system, load, engines and GPUs, and any warning.
• Cordon places nothing new on it; Uncordon undoes it.
• Drain moves its replicas off one at a time, each replacement up before the original stops.
• vllm build / sglang build (nodes with GPUs) — which build of the engine the node installs: auto follows its hardware, none installs none.
• Upgrade its agent to a release; its engines keep serving.
• Delete model files nothing uses frees the node's disk.
• Remove from the cluster — the control plane forgets it; the machine is not touched.

GPUs is every card: how busy, its memory, temperature, power, and the model holding it.

Models lists the catalog: ready against wanted, status, class, profiles, and UPDATE when the registry has new weights for it. On a row:

• Details — description, replicas, class, owner, profiles, source, where its weights came from, what it reserves against what it uses, and its replicas.
• Replicas (scale) — the minimum kept running, the most it scales to, and for a model loaded on demand how long it may sit unused. A maximum of 0 stops it.
• Class — critical, standard or opportunistic: whose capacity gives way when there is not enough for everyone.
• Owner — the organization, team or project whose capacity quota it counts against.
• Resources — the memory and cores one replica reserves on its node.
• LoRA adapters — name=source[@scale], separated by commas.
• Restart its replicas, one by one or in place.
• Update to … — when the repository it came from has published different weights for its files: they are fetched into the mirror beside the ones in use, checked, swapped in, and the replicas replaced one at a time. Go back to … returns to the weights the update replaced. Ask the registry for new weights now checks without waiting for the next scheduled look (mirror.update_check, six hours by default).
• Why is it placed where it is — llmgw ctl explain-placement.
• Try again, for a model parked after failing every start.
• Remove — its replicas stop and it leaves the catalog. A model from server.yaml is changed there instead; the console says so.

A on Models adds one from a definition file on this machine: the YAML a model takes in server.yaml.

Replicas lists every replica; on one, Restart it, replaced or in place.

Usage is the last 24 hours by team, project or organization (→ changes which): requests, tokens, errors and spend.

Quotas are capacity quotas: the GPUs, cores and memory each organization, team or project is guaranteed and may at most hold, and what it holds now. A adds one; on a row, change or remove it.

Mirror is what the model mirror holds, one row for each version of each model — a file, or the set of files a model of several is: shards, a vision projector, a config and tokenizer — with how far its download has got and which models use it. A downloads a URL into it; on a row, its files with their checksums and where each was fetched from, the source a model definition names for it, or remove it — with a warning when a model still uses it.

Logs shows a node's agent log, newest at the bottom, errors and warnings coloured.

People

The accounts, keys and organizations live in the store; these views read it and change it with the same commands as llmgw users, llmgw keys and llmgw org, so their checks and messages are the ones shown.

Accounts — who has signed up, their state, whether they see the cluster dashboard, and what they belong to. A creates an account, already approved. On a row: approve one waiting, suspend or reinstate, give or take the cluster dashboard, add to or take out of an organization, team or project, their limits, a new password (their sessions close), delete.

API keys — every key, its project, owner, state and last use. A issues one; the key is shown once, in the result, and never again. On a row: its limits, or revoke it.

Organizations — the tenancy tree: organizations, their teams, the teams' projects, with members, keys and limits. A adds an organization. On a row: add a team or project inside it, rename, move, limits for the level and for each of its members, its capacity quota, and for a project how long conversations and agent notes are kept. Remove takes away only something empty.

Limits are requests and tokens a minute, a budget per day or month, and the models allowed. Each level only narrows what is above it.

Help

Manual lists every page there is: these, the installation and operation guides, the configuration reference, a page for every command, the README and the changes in each release. On a page:

Key	Does
↑ ↓, PgUp PgDn, Space	Scroll
g, G	The top, the end
T	Contents: the page's headings, Enter goes to one
/	Find; the lines holding it are marked
n, N	The next match, the one before
[ ]	The topic before, the one after
←	Back to the list

llmgw docs opens the same manual outside the console, on any machine, installed or not; llmgw docs install opens one page; piped, it prints the page's text.

Settings

server.yaml (on a node, agent.yaml) is every key of the file, as a tree of menus built from the configuration reference: the value set, or the one that applies when nothing is, its type, and what it is for. Enter changes a value or opens a block; on a list or map of blocks, Enter opens an entry and D removes one. U puts a key back to its default. S saves: the file is checked the way the service reads it before anything is written, and its comments are kept. The service reads it when it starts, so the console offers to restart it. Another control plane keeps its own copy.

Do

• Add a node — installs llmgw on a machine over ssh, checks it, gives it a certificate and starts its agent (sudo llmgw node add).
• Replicate the store — moves the store onto two more machines under Patroni (offered only while it is not replicated).
• Serve a model — Hugging Face's models: trending, most downloaded, most liked or newest (S changes which), or / to search. F switches between GGUF files, which llama.cpp runs on the CPU or a GPU, and original safetensors weights, which vLLM runs on GPUs. Enter on a repository plans it against the cluster as it is, before anything is fetched: the model's shape is read from one file's header, or its config.json, and every file is listed with the best way it runs here — on one card, on several of a node's cards, across several nodes' cards over a fabric (original weights only), or on a CPU node — what each card or node would hold, and whether that hardware is free now. The file that suits the cluster best is first, marked ★; one that fits nowhere says so. Busy is not unfit: which file is best follows the hardware the cluster has, not what it is running, so a file for a card that is in use is still offered for it — served, it waits for the card, or with a fallback runs elsewhere until then. Choosing a file offers every way it fits, best first, and what to do when that one is busy: the next best way, written as a lower-priority profile so placement falls back to it, or wait. A vision model offers its projector too, fetched beside it and loaded with it so the model reads images. What is chosen is fetched into the mirror, checked, added to the catalog and started, and a replica is asked to say hello (llmgw ctl model serve; the plan alone is llmgw ctl model plan). mirror.registry points it at another registry, for a site without the internet.
• Upgrade the cluster to a release: this machine, then every node in turn (sudo llmgw ctl upgrade).
• Restart all replicas, one at a time.
• Back up now — the store and /etc/llmgw into one archive (sudo llmgw server backup).
• Leave.

The node's console

On a node, sudo llmgw opens agent.yaml in the same settings view, and the manual. The cluster is watched and changed from a control plane.