{{define "node"}} llmgw — {{.Node.ID}}

{{.Node.ID}}

{{.Node.State}} {{with .Node.OS}} · {{.}}{{end}}{{with .Node.MgmtAddr}} · {{.}}{{end}} · last seen {{or .Node.LastSeenAgo "never"}} ago · logs · cluster

{{if .OnAdminListener}}{{else}}{{template "nav" .Nav}}{{end}}
{{/* Refreshed in place, like the cluster page. */}}
{{with .Node}}
{{with .Load}}
{{printf "%.0f" .CPUPercent}}%
CPU{{if .CPUs}} · {{.CPUs}} cores{{end}}
{{template "minibar" pctbar .CPUPercent}}
{{bytes .MemUsed}}
RAM in use · of {{bytes .MemTotal}}
{{template "minibar" pctbar .MemPercent}}
{{printf "%.2f" .Load1}}
load · {{printf "%.2f" .Load5}} 5m · {{printf "%.2f" .Load15}} 15m
{{else}}
—
CPU and RAM not measured lately
{{end}} {{if .DiskTotalGB}}
{{.DiskFreeGB}}G
disk free · of {{.DiskTotalGB}}G
{{end}} {{if .GPUTotal}}
{{.GPUFree}}/{{.GPUTotal}}
GPUs free
{{end}}
{{if .Warnings}}{{range .Warnings}}

{{.}}

{{end}}{{end}} {{if .Unhealthy}}

{{join .Unhealthy}}

{{end}} {{end}}

What runs here

{{if .Engines}} {{range .Engines}}{{end}}
ModelReplicaStateCPUMemory
{{.Model}} {{.Replica}} {{.State}} {{template "minibar" pctbar .CPUShare}}{{printf "%.1f" .CPUCores}} cores {{if .MemoryBytes}}{{bytes .MemoryBytes}}{{else}}—{{end}}
{{else}}

No engines are placed on this node.

{{end}}

Engines: {{if .Node.Engines}}{{join .Node.Engines}}{{else}}none installed{{end}}{{if .Node.Models}} · {{len .Node.Models}} model file(s) on disk{{end}}.

{{if .HasHistory}}{{with .Charts}}

The last {{.Window}}

{{if .Err}}

No history: {{.Err}}

{{else}}
{{template "tschart" dict "t" .CPU "title" "CPU" "note" "All cores together, from the node's heartbeat."}} {{template "tschart" dict "t" .Memory "title" "RAM in use" "note" "Total less what the kernel could hand to a new process."}}
{{template "tschart" dict "t" .EngineCPU "title" "Cores busy, by model" "note" "Each engine's process group, the way top counts it."}} {{template "tschart" dict "t" .EngineMemory "title" "Engine memory, by model" "note" "Resident memory: the weights and the KV cache, mostly."}}
{{template "tschart" dict "t" .Load "title" "Load average" "note" "Above the core count means work is waiting for a CPU."}} {{template "tschart" dict "t" .Queue "title" "Engine queue depth, by replica" "note" "Requests accepted and not yet started, on the replicas here now."}}
{{end}} {{end}}{{else}}

No metrics store is configured, so there is no history here; monitoring.prometheus in the server configuration adds it.

{{end}}
{{end}}