{{define "dashboard"}}
{{.NodeCount}} nodes · {{.ReadyReplicas}} of {{.TotalReplicas}} replicas ready · {{.ModelCount}} models
| Node | State | GPUs | Classes | Fabrics | Replicas | Last seen | Agent |
|---|---|---|---|---|---|---|---|
{{.ID}}{{if .MgmtAddr}} {{.MgmtAddr}}{{end}} |
{{.State}} | {{.GPUFree}}/{{.GPUTotal}} | {{join .Classes}} | {{join .Fabrics}} | {{.Replicas}} | {{or .LastSeenAgo "—"}} | {{or .AgentVersion "—"}} |
| {{join .Unhealthy}} | |||||||
No agents have registered.
{{end}}| Model | Aliases | Ready | Wanted | Profiles | Fallbacks |
|---|---|---|---|---|---|
{{.Name}} |
{{join .Aliases}} | {{.Ready}} | {{.Min}}–{{.Max}} | {{join .Profiles}} | {{join .Fallbacks}} |
The catalog is empty.
{{end}} {{if .Unplaced}}{{.Model}}
{{range .Rejections}}{{.Profile}} @ {{or .Pool "-"}}{{if .Node}}/{{.Node}}{{end}}: {{.Reason}}
{{end}}
{{end}}
{{end}}
| Replica | Model | State | Pool | Endpoint | Age | Nodes |
|---|---|---|---|---|---|---|
{{.ID}} |
{{.Model}} | {{.State}} | {{or .Pool "—"}} | {{if .Endpoint}}{{.Endpoint}}{{else}}—{{end}} |
{{duration .AgeSec}} | {{range .Shards}}{{.Node}} {{end}} |
| {{.Err}} | ||||||
No replicas.
{{end}}| Key | Project | Requests | Errors | Prompt | Completion | Spend | Last |
|---|---|---|---|---|---|---|---|
{{if .KeyName}}{{.KeyName}} {{.KeyPrefix}}{{else}}{{.KeyID}}{{end}} |
{{.ProjectID}} | {{.Requests}} | {{if .Errors}}{{.Errors}}{{else}}0{{end}} | {{.Prompt}} | {{.Completion}} | {{units .CostMicros}} | {{ago .LastAt}} |
Spend is in units, whatever a unit was configured to mean. Usage is written in batches, so the last few seconds may be missing.
{{else}}No requests recorded in the last 24 hours.
{{end}} {{else}}Authentication is off, so there is no identity to account against.
{{end}}