A v0.14.6 install on fewer than eight cores refused every request until its judge was given cores by hand, and upgrading did not change that. The judge keeper, on the leader, now finds a judge whose entry is exactly what init before v0.14.7 wrote: one llama.cpp CPU profile, an 8192 context and nothing else, 2 or 4 GB, none to two cores. It gives the judge 2 GB, --cache-ram 0 and cores by init's rule through modelsvc.SetProfiles, then restarts its ready replicas. The replica count is left as it is. A judge whose entry differs is left alone, and the log says so once. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
b8274d6278
commit
2545fca174
6 files changed
+228
-4
No files matched your search
@@ -21,6 +21,7 @@ import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"slices"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
@@ -248,6 +249,19 @@ func (s *Service) Resources(ctx context.Context, name, profile string, memoryGB,
|
||||
return final, err
|
||||
}
|
||||
|
||||
// SetProfiles replaces a model's profiles, as a model added with them
|
||||
// would have: for a model whose engines are to run otherwise, its args among
|
||||
// it. Running replicas keep what they were started with until restarted.
|
||||
func (s *Service) SetProfiles(ctx context.Context, name string, profiles []catalog.Profile, by string) (catalog.Model, error) {
|
||||
if len(profiles) == 0 {
|
||||
return catalog.Model{}, errors.New("a model needs a profile")
|
||||
}
|
||||
return s.update(ctx, name, by, "replace its profiles", func(m *catalog.Model) error {
|
||||
m.Profiles = slices.Clone(profiles)
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
// SetClass sets a model's capacity class: critical, standard or
|
||||
// opportunistic, or empty to go back to what its scaling implies.
|
||||
func (s *Service) SetClass(ctx context.Context, name, class, by string) (catalog.Model, error) {
|
||||
|
||||
Reference in new issue
Block a user