Busy is not unfit: rank files by the hardware the cluster has, and let a model wait for its cards
CI / test (push) Successful in 4m43s
CI / test (push) Successful in 4m43s
Ranking what was free first made every file come out on the CPU on a cluster whose one card was in use. The plan now ranks by hardware alone, free now only breaking a tie; a model served onto cards that are busy is added and waits for them, the job ending as waiting rather than failed. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
c2be429554
commit
7cb9d4fa01
7 files changed
+118
-17
No files matched your search
@@ -93,6 +93,12 @@ func followJob(cmd *cobra.Command, client func() *ctlClient, job modelserve.Job,
|
||||
}
|
||||
switch job.State {
|
||||
case modelserve.Done:
|
||||
for _, st := range job.Steps {
|
||||
if st.Name == "placement" && st.State == modelserve.Skipped {
|
||||
fmt.Fprintf(out, "\n%s is in the catalog, waiting for room on the hardware it fits.\n", job.Model)
|
||||
return nil
|
||||
}
|
||||
}
|
||||
fmt.Fprintf(out, "\n%s %s.\n", job.Model, doneSays)
|
||||
return nil
|
||||
case modelserve.Failed:
|
||||
|
||||
Reference in new issue
Block a user