What a replica computes on comes from nvidia-smi; llama.cpp says its layers only at -lv 4

Checked against the installed llama.cpp (b10991): at its default verbosity
it prints neither the devices it uses nor the layers it offloads, so the
start-up output said nothing and every replica's ON column stayed empty.
The agent now takes compute from nvidia-smi - the GPUs the shard's processes
hold memory on, and how much - or CPU, for every engine, falling back to it
when llama.cpp's output says nothing; adopted engines get it too, and each
session reports it once per shard as a compute-only status. engine-check
runs its GPU token at -lv 4, where llama.cpp does say where its layers went.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-07 14:04:08 +02:00
1 parent 25b3e129fe
commit b5d650af16
7 files changed
+147 -13

No files matched your search

+12
View File
@@ -658,6 +658,18 @@ func (s *Supervisor) SetCompute(id string, c engines.Compute) {
}
}
// ComputeOf is where a shard's engine was found to compute, nil when not
// yet known.
func (s *Supervisor) ComputeOf(id string) *engines.Compute {
s.mu.Lock()
defer s.mu.Unlock()
if sh, ok := s.shards[id]; ok && sh.Spec.Compute != nil {
c := *sh.Spec.Compute
return &c
}
return nil
}
// Remove forgets a terminated shard.
func (s *Supervisor) Remove(id string) {
s.mu.Lock()