Say on the chat page whether a GPU or a CPU answered
Release / build (push) Successful in 1m12s
Release / notes (push) Successful in 3s
CI / test (push) Successful in 3m54s

A model with replicas on a card and on CPUs answered at either speed with no
way to tell which. The picker names where each model runs, each reply names
the hardware that served it, and every answer carries x-llmgw-hardware.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-03 18:57:50 +02:00
1 parent 477d0bf19d
commit 1b89d5cdea
7 files changed
+149 -4

No files matched your search

+27
View File
@@ -18,6 +18,33 @@ type ChatModel struct {
// offered anyway: the first message is what asks, so hiding it until it
// runs meant it could never be started from here.
OnDemand bool
// Hardware is what the model's running replicas compute on: "cpu", or a
// GPU class. Several when it runs on both, and empty when nothing runs
// or nothing said.
Hardware []string
}
// Runs names the hardware for the picker: "GPU", "CPU", or "GPU + CPU" for a
// model with replicas on both, where which one answers is the router's choice
// and each reply says which it was.
func (m ChatModel) Runs() string {
var gpu, cpu bool
for _, h := range m.Hardware {
if h == "cpu" {
cpu = true
} else {
gpu = true
}
}
switch {
case gpu && cpu:
return "GPU + CPU"
case gpu:
return "GPU"
case cpu:
return "CPU"
}
return ""
}
// ChatContext is one thing the chat page can spend against: a project, or the