{{define "node"}} llmgw — {{.Node.ID}}

{{.Node.ID}}

{{.Node.State}} {{with .Node.OS}} · {{.}}{{end}}{{with .Node.MgmtAddr}} · {{.}}{{end}} · last seen {{or .Node.LastSeenAgo "never"}} ago · logs

{{if .OnAdminListener}}{{else}}{{template "nav" .Nav}}{{end}}
{{/* Refreshed in place, like the cluster page. */}}
{{with .Node}}
{{with .Load}}
{{printf "%.0f" .CPUPercent}}%
CPU{{if .CPUs}} · {{.CPUs}} cores{{end}}
{{template "minibar" pctbar .CPUPercent}}
{{bytes .MemUsed}}
RAM in use · of {{bytes .MemTotal}}
{{template "minibar" pctbar .MemPercent}}
{{printf "%.2f" .Load1}}
load · {{printf "%.2f" .Load5}} 5m · {{printf "%.2f" .Load15}} 15m
{{else}}
—
CPU and RAM not measured lately
{{end}} {{if .DiskTotalGB}}
{{.DiskFreeGB}}G
disk free · of {{.DiskTotalGB}}G
{{end}} {{if .GPUTotal}}
{{.GPUFree}}/{{.GPUTotal}}
GPUs free
{{end}}
{{if .Warnings}}{{range .Warnings}}

{{.}}

{{end}}{{end}} {{if .Unhealthy}}

{{join .Unhealthy}}

{{end}} {{end}} {{if or .Node.IBPorts .Node.Fabrics}}

Fabric

{{if .Node.Fabrics}}On {{join .Node.Fabrics}}.{{else}}On no fabric: no port a declared fabric names can carry traffic, so no multi-node replica is placed here.{{end}}

{{if .Node.IBPorts}} {{range .Node.IBPorts}}{{end}}
AdapterStateRateFabricErrors since bootSentReceived
{{.HCA}}/{{.Port}}{{if eq .LinkLayer "Ethernet"}} RoCE{{end}} {{.State}}{{with .PhysState}} {{.}}{{end}} {{if .RateGbps}}{{.RateGbps}} Gb/s{{else}}—{{end}} {{or .Fabric "—"}} {{if .Errors}}{{range $k, $v := .Errors}}{{$k}} {{$v}} {{end}}{{else}}none{{end}} {{if .TxBytes}}{{bytes (u64 .TxBytes)}}{{else}}—{{end}} {{if .RxBytes}}{{bytes (u64 .RxBytes)}}{{else}}—{{end}}

Re-read on every heartbeat. A port that goes down takes its fabric off this node within seconds; one that comes back returns it. Errors are cumulative since the adapter came up: a number that grows is the one to look at.

{{end}} {{end}}

What runs here

{{if .Engines}} {{range .Engines}}{{end}}
ModelReplicaStateCPUMemory
{{.Model}} {{.Replica}} {{.State}} {{template "minibar" pctbar .CPUShare}}{{printf "%.1f" .CPUCores}} cores {{if .MemoryBytes}}{{bytes .MemoryBytes}}{{else}}—{{end}}
{{else}}

No engines are placed on this node.

{{end}}

Engines: {{if .Node.Engines}}{{join .Node.Engines}}{{else}}none installed{{end}}{{if .Node.Models}} · {{len .Node.Models}} model file(s) on disk{{end}}.

{{if .HasHistory}}{{with .Charts}}

The last {{.Window}}

{{if .Err}}

No history: {{.Err}}

{{else}}
{{template "tschart" dict "t" .CPU "title" "CPU" "note" "All cores together, from the node's heartbeat."}} {{template "tschart" dict "t" .Memory "title" "RAM in use" "note" "Total less what the kernel could hand to a new process."}}
{{template "tschart" dict "t" .EngineCPU "title" "Cores busy, by model" "note" "Each engine's process group, the way top counts it."}} {{template "tschart" dict "t" .EngineMemory "title" "Engine memory, by model" "note" "Resident memory, including the weights mapped from disk. The kernel can take those pages back, so this can exceed RAM in use."}}
{{template "tschart" dict "t" .Load "title" "Load average" "note" "Above the core count means work is waiting for a CPU."}} {{template "tschart" dict "t" .Queue "title" "Engine queue depth, by replica" "note" "Requests accepted and not yet started, on the replicas here now."}}
{{end}} {{end}}{{else}}

No metrics store is configured, so there is no history here; monitoring.prometheus in the server configuration adds it.

{{end}}
{{end}}