A btop-like monitor in the console, from Prometheus; node exporters discovered and installed

Monitor draws CPU, memory, GPUs, requests, tokens, latency, network and disk
as Braille charts over 5m/1h/24h/7d, then per node, card and model
sparklines; node and model pages have their own. History is read from
monitoring.prometheus through POST /api/v1/metrics/range, or without it
from what the console saw. The control plane serves the cluster's machines
for Prometheus http_sd at /prometheus/targets on its metrics port, and node
add installs prometheus-node-exporter.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-04 01:51:05 +02:00
1 parent 2e55487eca
commit a1b1663e9d
16 files changed
+1217 -4

No files matched your search

+13
View File
@@ -57,6 +57,19 @@ read, and Enter brings the console back.
**Overview** is `llmgw ctl status`: nodes, GPUs by class, fabrics, replicas,
models, the store's members, and anything parked, unplaced or idle.
**Monitor** draws the cluster the way btop draws a machine: CPU, memory,
GPU busy and GPU memory, requests and tokens a second, time to first token,
requests in flight and queued, network and disk — Braille charts, filled
and coloured green to red by height — over 5 minutes, an hour, a day or a
week (**1**–**4**, or **R** for the next). Below them each node's CPU,
memory and network now and as a sparkline, each card's busy, memory, heat
and power, and each model's requests a second. The history comes from the
Prometheus named by `monitoring.prometheus` in server.yaml; without one the
charts draw what this console has seen since it opened, and say so. Network
and disk come from each machine's node exporter, which `llmgw node add`
installs. A node's and a model's own pages have their charts too, over the
same range.
**Nodes** lists every node with its state, load, GPUs, replicas, disk and
agent version. On a row: