Ilker ManapandClaude Opus 5
c6df0659af
CI / test (push) Successful in 1m50s
The dashboard charted what the cluster had been billed for — requests, tokens, spend, all from llmgw's own store — and nothing about whether it was healthy. Saturation, queue depth, cache occupancy and replica counts were numbers in a table at the instant the page loaded, with no way to see that one of them had been climbing for an hour. Four line charts now, read back from wherever the metrics were scraped to. Read back rather than kept twice. llmgw publishes these already; what a gauge cannot do is remember, and a ring buffer in this process would be a rival copy of the same numbers with its own retention, lost on every restart and disagreeing with the one an operator has on screen elsewhere. monitoring.prometheus is empty by default and the panels are then absent rather than empty. One axis per chart, always. Two measures of different scale belong in two charts: a second y-axis lets any two lines be made to cross wherever the author likes and the reader cannot tell it happened. Four colours, handed out in fixed order, never cycled. A fifth series is summed into "other" and drawn in grey — dropping it would make a chart that does not add up, and inventing a hue for it would put two lines on the page that nobody can tell apart. The fold is by timestamp, not by position: the tail's series do not share sample times, and adding them by index slides points sideways onto each other's clock. A legend for every chart with more than one line, because colour alone is not an encoding a reader can rely on. A flat zero is drawn along the bottom rather than left blank: nothing happening is a real answer, and an empty frame reads as a broken chart. A single sample is not a line and says so instead of putting a dot in the corner. One failed query fails all four panels. Three drawn and one blank invites reading the blank one as "nothing is happening there". NaN is dropped rather than plotted. Prometheus sends it as a string for a sample it has no value for, Go's ParseFloat accepts it happily, and one NaN coordinate is an SVG path that does not draw at all — taking the rest of the line with it. Axis maxima are rounded to a number a person would pick. An axis topping out at 0.8347 makes every gridline unreadable, and the shape is what a chart is for. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
93 lines
2.6 KiB
Go
93 lines
2.6 KiB
Go
package web
|
|
|
|
import (
|
|
"context"
|
|
"log/slog"
|
|
"time"
|
|
|
|
"gitea.mynodes.xyz/ilker/llmgw/internal/promq"
|
|
)
|
|
|
|
// History reads the cluster's past out of a metrics store.
|
|
type History interface {
|
|
RangeQuery(ctx context.Context, expr, nameBy string, window, step time.Duration) ([]promq.Series, error)
|
|
}
|
|
|
|
// FleetCharts are the operational panels: what the cluster is doing, rather
|
|
// than what it has been billed for.
|
|
//
|
|
// The usage charts already on this page come from llmgw's own store and answer
|
|
// a different question — who spent what. These answer whether the thing is
|
|
// healthy, and they need history llmgw does not keep.
|
|
type FleetCharts struct {
|
|
Saturation TimeSeries
|
|
Queue TimeSeries
|
|
KV TimeSeries
|
|
Ready TimeSeries
|
|
Window string
|
|
// Err is set when the metrics store could not be read. Shown as a line of
|
|
// text where the charts would be: an empty panel reads as a healthy cluster
|
|
// with nothing happening, which is the opposite of not knowing.
|
|
Err string
|
|
}
|
|
|
|
// fleetPanel is one chart and the query behind it.
|
|
var fleetPanels = []struct {
|
|
name string
|
|
expr string
|
|
nameBy string
|
|
unit string
|
|
into func(*FleetCharts, TimeSeries)
|
|
}{
|
|
{
|
|
"saturation", "llmgw_pool_saturation", "model", "",
|
|
func(f *FleetCharts, t TimeSeries) { f.Saturation = t },
|
|
},
|
|
{
|
|
"queue depth", "llmgw_endpoint_waiting", "replica", "",
|
|
func(f *FleetCharts, t TimeSeries) { f.Queue = t },
|
|
},
|
|
{
|
|
"kv cache", "llmgw_endpoint_kv_cache_usage", "replica", "%",
|
|
func(f *FleetCharts, t TimeSeries) { f.KV = t },
|
|
},
|
|
{
|
|
"replicas ready", "llmgw_model_replicas_ready", "model", "",
|
|
func(f *FleetCharts, t TimeSeries) { f.Ready = t },
|
|
},
|
|
}
|
|
|
|
// LoadFleetCharts queries each panel and renders it.
|
|
//
|
|
// One failure fails them all, deliberately: four panels where three are drawn
|
|
// and one is blank invites reading the blank one as "nothing is happening
|
|
// there". Either the metrics store answered or it did not.
|
|
func LoadFleetCharts(ctx context.Context, h History, window time.Duration,
|
|
log *slog.Logger,
|
|
) *FleetCharts {
|
|
if h == nil {
|
|
return nil
|
|
}
|
|
if window <= 0 {
|
|
window = time.Hour
|
|
}
|
|
// Sixty points across the window. More is detail nobody can see at this
|
|
// size and a bigger response to parse on every page load.
|
|
step := window / 60
|
|
|
|
out := &FleetCharts{Window: window.String()}
|
|
for _, p := range fleetPanels {
|
|
series, err := h.RangeQuery(ctx, p.expr, p.nameBy, window, step)
|
|
if err != nil {
|
|
if log != nil {
|
|
log.Warn("could not read history for the dashboard",
|
|
"panel", p.name, "err", err)
|
|
}
|
|
out.Err = err.Error()
|
|
return out
|
|
}
|
|
p.into(out, LineChart(series, p.unit, 4))
|
|
}
|
|
return out
|
|
}
|