llmgw/docs/console.md

# The console

`sudo llmgw`, with nothing after it, on a machine llmgw runs on. On a machine
with nothing installed it opens the setup screens; on a control plane, the
console described here; on a node, the node's own console, which edits
`agent.yaml`. Everything on these screens can also be done by command — each
screen says which — so a script never needs them and a person never needs
to remember the commands.

The keyboard drives it; there is no mouse. **?** opens this page from
anywhere, and the **Manual** entry holds every other page, as `llmgw docs`
does without the console.

## The layout

A bar across the top names the machine. Down the left, the entries, in four
groups: **Watch**, **People**, **Help**, **Settings** and **Do**. The panel on
the right shows the entry chosen. The bottom line always says which keys do
what at that moment, and a question or a result opens as a dialog over the
panel.

| Key | Does |
|---|---|
| ↑ ↓ | Choose an entry, or a row in the panel |
| → or Enter | Go into the panel |
| Enter on a row | What can be done to it: its details first, then the rest |
| ← or Esc | Back out: from a page to its list, from the panel to the entries |
| A | Add one, on a view that holds things: a quota, an account, a key |
| PgUp PgDn | Scroll a long panel |
| ? | This page |
| Q | Leave; the cluster keeps running |
| Ctrl-C | Leave at once |

What is watched is read again every three seconds.

## Forms and questions

A change that needs values opens a form, filled in with what is set now. Tab
or ↓ moves to the next field, ← → changes a choice, and Enter on **Apply**
sends it; a form of one field is sent by Enter on it. A mistake the form can
see — a maximum below the minimum, a word where a number goes — is said in the
form and nothing is sent. One the control plane refuses comes back as a dialog
with its reason, word for word.

Anything that cannot be undone — removing, draining, deleting, revoking —
asks first, and only **Y** goes ahead.

Tasks that take a while and print as they go — serving a model, upgrading
the cluster, a backup, adding a node — close the screen and run in the
terminal, so their output is there to read; Enter brings the console back.

## Watch

**Overview** is `llmgw ctl status`: nodes, GPUs by class, fabrics, replicas,
models, the store's members, and anything parked, unplaced or idle.

**Nodes** lists every node with its state, load, GPUs, replicas, disk and
agent version. On a row:

- **Details** — its address, system, load, engines and GPUs, and any warning.
- **Cordon** places nothing new on it; **Uncordon** undoes it.
- **Drain** moves its replicas off one at a time, each replacement up before
  the original stops.
- **vllm build / sglang build** (nodes with GPUs) — which build of the engine
  the node installs: `auto` follows its hardware, `none` installs none.
- **Upgrade its agent** to a release; its engines keep serving.
- **Delete model files nothing uses** frees the node's disk.
- **Remove from the cluster** — the control plane forgets it; the machine is
  not touched.

**GPUs** is every card: how busy, its memory, temperature, power, and the
model holding it.

**Models** lists the catalog: ready against wanted, status, class, profiles,
and **UPDATE** when the registry has new weights for it. On a row:

- **Details** — description, replicas, class, owner, profiles, source, where
  its weights came from, what it reserves against what it uses, and its
  replicas.
- **Replicas (scale)** — the minimum kept running, the most it scales to,
  and for a model loaded on demand how long it may sit unused. A maximum of
  0 stops it.
- **Class** — critical, standard or opportunistic: whose capacity gives way
  when there is not enough for everyone.
- **Owner** — the organization, team or project whose capacity quota it
  counts against.
- **Resources** — the memory and cores one replica reserves on its node.
- **LoRA adapters** — `name=source[@scale]`, separated by commas.
- **Restart its replicas**, one by one or in place.
- **Update to …** — when the repository it came from has published different
  weights for its files: they are fetched into the mirror beside the ones in
  use, checked, swapped in, and the replicas replaced one at a time.
  **Go back to …** returns to the weights the update replaced.
  **Ask the registry for new weights now** checks without waiting for the
  next scheduled look (`mirror.update_check`, six hours by default).
- **Why is it placed where it is** — `llmgw ctl explain-placement`.
- **Try again**, for a model parked after failing every start.
- **Remove** — its replicas stop and it leaves the catalog. A model from
  `server.yaml` is changed there instead; the console says so.

**A** on Models adds one from a definition file on this machine: the YAML a
model takes in `server.yaml`.

**Replicas** lists every replica; on one, **Restart it**, replaced or in
place.

**Usage** is the last 24 hours by team, project or organization (→ changes
which): requests, tokens, errors and spend.

**Quotas** are capacity quotas: the GPUs, cores and memory each organization,
team or project is guaranteed and may at most hold, and what it holds now.
**A** adds one; on a row, change or remove it.

**Mirror** is what the model mirror holds, one row for each version of each
model — a file, or the set of files a model of several is: shards, a vision
projector, a config and tokenizer — with how far its download has got and
which models use it. **A** downloads a URL into it; on a row, its files with
their checksums and where each was fetched from, the source a model
definition names for it, or remove it — with a warning when a model still
uses it.

**Logs** shows a node's agent log, newest at the bottom, errors and warnings
coloured.

## People

The accounts, keys and organizations live in the store; these views read it
and change it with the same commands as `llmgw users`, `llmgw keys` and
`llmgw org`, so their checks and messages are the ones shown.

**Accounts** — who has signed up, their state, whether they see the cluster
dashboard, and what they belong to. **A** creates an account, already
approved. On a row: approve one waiting, suspend or reinstate, give or take
the cluster dashboard, add to or take out of an organization, team or
project, their limits, a new password (their sessions close), delete.

**API keys** — every key, its project, owner, state and last use. **A**
issues one; the key is shown once, in the result, and never again. On a row:
its limits, or revoke it.

**Organizations** — the tenancy tree: organizations, their teams, the teams'
projects, with members, keys and limits. **A** adds an organization. On a
row: add a team or project inside it, rename, move, limits for the level and
for each of its members, its capacity quota, and for a project how long
conversations and agent notes are kept. **Remove** takes away only something
empty.

Limits are requests and tokens a minute, a budget per day or month, and the
models allowed. Each level only narrows what is above it.

## Help

**Manual** lists every page there is: these, the installation and operation
guides, the configuration reference, a page for every command, the README
and the changes in each release. On a page:

| Key | Does |
|---|---|
| ↑ ↓, PgUp PgDn, Space | Scroll |
| g, G | The top, the end |
| T | Contents: the page's headings, Enter goes to one |
| / | Find; the lines holding it are marked |
| n, N | The next match, the one before |
| [ ] | The topic before, the one after |
| ← | Back to the list |

`llmgw docs` opens the same manual outside the console, on any machine,
installed or not; `llmgw docs install` opens one page; piped, it prints the
page's text.

## Settings

**server.yaml** (on a node, **agent.yaml**) is every key of the file, as a
tree of menus built from the configuration reference: the value set, or the
one that applies when nothing is, its type, and what it is for. Enter changes
a value or opens a block; on a list or map of blocks, Enter opens an entry and
**D** removes one. **U** puts a key back to its default. **S** saves: the file
is checked the way the service reads it before anything is written, and its
comments are kept. The service reads it when it starts, so the console offers
to restart it. Another control plane keeps its own copy.

## Do

- **Add a node** — installs llmgw on a machine over ssh, checks it, gives it a
  certificate and starts its agent (`sudo llmgw node add`).
- **Replicate the store** — moves the store onto two more machines under
  Patroni (offered only while it is not replicated).
- **Serve a model** — Hugging Face's models: trending, most downloaded,
  most liked or newest (**S** changes which), or **/** to search. **F**
  switches between GGUF files, which llama.cpp runs on the CPU or a GPU, and
  original safetensors weights, which vLLM runs on GPUs. Enter on a
  repository plans it against the cluster as it is, before anything is
  fetched: the model's shape is read from one file's header, or its
  config.json, and every file is listed with the best way it runs here — on
  one card, on several of a node's cards, across several nodes' cards over a
  fabric (original weights only), or on a CPU node — what each card or node
  would hold, and whether that hardware is free now. The file that suits the
  cluster best is first, marked ★; one that fits nowhere says so. Choosing a
  file offers every way it fits, best first, and what to do when that one is
  busy: the next best way, written as a lower-priority profile so placement
  falls back to it, or wait. A vision model offers its projector too, fetched
  beside it and loaded with it so the model reads images. What is chosen is
  fetched into the mirror, checked, added to the catalog and started, and a
  replica is asked to say hello (`llmgw ctl model serve`; the plan alone is
  `llmgw ctl model plan`).
  `mirror.registry` points it at another registry, for a site without the
  internet.
- **Upgrade the cluster** to a release: this machine, then every node in turn
  (`sudo llmgw ctl upgrade`).
- **Restart all replicas**, one at a time.
- **Back up now** — the store and `/etc/llmgw` into one archive
  (`sudo llmgw server backup`).
- **Leave**.

## The node's console

On a node, `sudo llmgw` opens `agent.yaml` in the same settings view, and the
manual. The cluster is watched and changed from a control plane.