Track the registry's new weights for served models, and move a model to them or back
A model served from the registry records its origin: repository, commit and each file's checksum; one served before is worked out from the mirror's record of where it fetched the file. Every mirror.update_check (6h) the control plane asks whether those files changed there. ctl model updates lists it, ctl model update fetches beside the old files, checks, swaps and replaces replicas one at a time, ctl model rollback goes back. The console shows it on Models and offers the same. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
a96b1a54b2
commit
44425193bd
15 files changed
+1393
-33
No files matched your search
@@ -76,6 +76,7 @@ rather than repeated.
|
||||
| `tls` | `tlsconf.Config` | [Mirror](#mirror) |
|
||||
| `registry` | `string` | [Mirror](#mirror) |
|
||||
| `timeout` | `time.Duration` | [Mirror](#mirror) |
|
||||
| `update_check` | `time.Duration` | [Mirror](#mirror) |
|
||||
| `name` | `string` | [Model](#model) |
|
||||
| `aliases` | `[]string` | [Model](#model) |
|
||||
| `description` | `string` | [Model](#model) |
|
||||
@@ -92,6 +93,7 @@ rather than repeated.
|
||||
| `context_length` | `int` | [Model](#model) |
|
||||
| `max_output` | `int` | [Model](#model) |
|
||||
| `capabilities` | `Capabilities` | [Model](#model) |
|
||||
| `origin` | `*Origin` | [Model](#model) |
|
||||
| `prometheus` | `string` | [Monitoring](#monitoring) |
|
||||
| `window` | `time.Duration` | [Monitoring](#monitoring) |
|
||||
| `timeout` | `time.Duration` | [Monitoring](#monitoring) |
|
||||
@@ -99,6 +101,13 @@ rather than repeated.
|
||||
| `ib_root` | `string` | [Options](#options) |
|
||||
| `prefer_nvml` | `bool` | [Options](#options) |
|
||||
| `skip_gpus` | `bool` | [Options](#options) |
|
||||
| `repo` | `string` | [Origin](#origin) |
|
||||
| `revision` | `string` | [Origin](#origin) |
|
||||
| `files` | `[]OriginFile` | [Origin](#origin) |
|
||||
| `previous` | `*PreviousOrigin` | [Origin](#origin) |
|
||||
| `path` | `string` | [OriginFile](#originfile) |
|
||||
| `sha256` | `string` | [OriginFile](#originfile) |
|
||||
| `bytes` | `int64` | [OriginFile](#originfile) |
|
||||
| `disabled` | `bool` | [Personal](#personal) |
|
||||
| `rpm` | `int` | [Personal](#personal) |
|
||||
| `tpm` | `int` | [Personal](#personal) |
|
||||
@@ -111,6 +120,9 @@ rather than repeated.
|
||||
| `match_ratio` | `float64` | [Prefix](#prefix) |
|
||||
| `balance_abs` | `float64` | [Prefix](#prefix) |
|
||||
| `balance_rel` | `float64` | [Prefix](#prefix) |
|
||||
| `source` | `string` | [PreviousOrigin](#previousorigin) |
|
||||
| `revision` | `string` | [PreviousOrigin](#previousorigin) |
|
||||
| `files` | `[]OriginFile` | [PreviousOrigin](#previousorigin) |
|
||||
| `name` | `string` | [Profile](#profile) |
|
||||
| `engine` | `string` | [Profile](#profile) |
|
||||
| `gpu_class` | `string` | [Profile](#profile) |
|
||||
@@ -668,6 +680,15 @@ Timeout bounds a management call. It does not bound a download: the
|
||||
mirror answers a fetch as soon as it has started one, and the bytes keep
|
||||
arriving long after this deadline has passed.
|
||||
|
||||
### `update_check`
|
||||
|
||||
`time.Duration`
|
||||
|
||||
UpdateCheck is how often the registry is asked whether the files of
|
||||
the models served from it have changed there: a repository can publish
|
||||
new weights under the same name. Only told, never applied — moving a
|
||||
model to them is `llmgw ctl model update`. 0 never asks.
|
||||
|
||||
## Model
|
||||
|
||||
### `name`
|
||||
@@ -791,6 +812,19 @@ Capabilities is what this model can do beyond plain completion.
|
||||
|
||||
Takes a [Capabilities](#capabilities) block.
|
||||
|
||||
### `origin`
|
||||
|
||||
`*Origin`
|
||||
|
||||
Origin is the registry repository the weights came from, at which
|
||||
commit, and each file's checksum there. Written when a model is served
|
||||
from the registry; for one that was not, worked out from the mirror's
|
||||
record of where it fetched the file. It is what lets the control plane
|
||||
say that the repository has published different weights since, and
|
||||
move the model to them.
|
||||
|
||||
Takes a [Origin](#origin) block.
|
||||
|
||||
## Monitoring
|
||||
|
||||
### `prometheus`
|
||||
@@ -845,6 +879,51 @@ is used, is the more exact source of the peers.
|
||||
SkipGPUs omits GPU discovery. A node without nvidia-smi is already
|
||||
found to have none, so this is for one whose driver is broken.
|
||||
|
||||
## Origin
|
||||
|
||||
### `repo`
|
||||
|
||||
`string`
|
||||
|
||||
Repo is the repository, owner/name.
|
||||
|
||||
### `revision`
|
||||
|
||||
`string`
|
||||
|
||||
Revision is the commit the files were fetched at.
|
||||
|
||||
### `files`
|
||||
|
||||
`[]OriginFile`
|
||||
|
||||
Files are the weights fetched from it, with what the registry
|
||||
published as each one's sha256. A new commit that leaves them alone —
|
||||
a README edited — is not an update.
|
||||
|
||||
### `previous`
|
||||
|
||||
`*PreviousOrigin`
|
||||
|
||||
Previous is what the last update replaced, kept so it can be gone back
|
||||
to while its files are still on the mirror.
|
||||
|
||||
Takes a [PreviousOrigin](#previousorigin) block.
|
||||
|
||||
## OriginFile
|
||||
|
||||
### `path`
|
||||
|
||||
`string`
|
||||
|
||||
### `sha256`
|
||||
|
||||
`string`
|
||||
|
||||
### `bytes`
|
||||
|
||||
`int64`
|
||||
|
||||
## Personal
|
||||
|
||||
### `disabled`
|
||||
@@ -910,6 +989,20 @@ before affinity is abandoned.
|
||||
|
||||
`float64`
|
||||
|
||||
## PreviousOrigin
|
||||
|
||||
### `source`
|
||||
|
||||
`string`
|
||||
|
||||
### `revision`
|
||||
|
||||
`string`
|
||||
|
||||
### `files`
|
||||
|
||||
`[]OriginFile`
|
||||
|
||||
## Profile
|
||||
|
||||
### `name`
|
||||
|
||||
Reference in new issue
Block a user