A btop-like monitor in the console, from Prometheus; node exporters discovered and installed
Monitor draws CPU, memory, GPUs, requests, tokens, latency, network and disk as Braille charts over 5m/1h/24h/7d, then per node, card and model sparklines; node and model pages have their own. History is read from monitoring.prometheus through POST /api/v1/metrics/range, or without it from what the console saw. The control plane serves the cluster's machines for Prometheus http_sd at /prometheus/targets on its metrics port, and node add installs prometheus-node-exporter. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
2e55487eca
commit
a1b1663e9d
16 files changed
+1217
-4
No files matched your search
@@ -76,6 +76,26 @@ is not an update.
|
||||
A model from `server.yaml` is reported but not changed: its source is edited
|
||||
there.
|
||||
|
||||
### The monitor: the cluster drawn like btop
|
||||
|
||||
The console has a **Monitor** under Watch: CPU, memory, GPU busy and memory,
|
||||
requests, tokens, time to first token, in flight and queued, network and
|
||||
disk, as Braille charts filled and coloured by height, over 5 minutes, an
|
||||
hour, a day or a week (1–4 or R); then each node's CPU, memory and network,
|
||||
each card's busy, memory, heat and power, and each model's requests, as
|
||||
sparklines. A node's and a model's own pages carry their charts too.
|
||||
|
||||
The history is read from the Prometheus in `monitoring.prometheus`, through
|
||||
the admin API (`POST /api/v1/metrics/range`); without one, the charts draw
|
||||
what the console has seen since it opened.
|
||||
|
||||
The control plane now lists the cluster's machines for Prometheus' service
|
||||
discovery, at `/prometheus/targets` on its metrics port — nodes, itself and
|
||||
the store's members, each labelled with its llmgw id — and `llmgw node add`
|
||||
installs `prometheus-node-exporter` on the machine (`--no-node-exporter`
|
||||
leaves it off). `deploy/monitoring/prometheus.yml` scrapes them as the
|
||||
`nodes` job.
|
||||
|
||||
### Serving and updating run in the background
|
||||
|
||||
Serving a model or moving one to new weights from the console starts a job
|
||||
|
||||
Reference in new issue
Block a user