Upgrade the cluster with one command, over the agents' own streams
Every upgrade so far has been a script of mine copying a binary to each machine over ssh, restarting it and polling until it came back — which is the job, but not something the product could do, and which needed ssh into every node. `sudo llmgw ctl upgrade VERSION` on the control plane does it. This machine first: the release for its platform, checked against the release's checksums, the old binary kept beside it, llmgw-server restarted, and a wait until every node that was ready has reconnected. Then each node in turn: the control plane tells its agent the version over the stream it already holds; the agent fetches and checks the release itself, swaps it in with a rename, answers, and exits with status 75 so systemd starts the new binary. Engines keep running, as for any agent restart, and are adopted again. A node is done when it is back on the new version, ready, with as many ready replicas as before; one that is not within the timeout stops the run before the next node, so a bad release reaches one machine. Nodes already on the version are skipped, so running it again resumes. The fetch-and-verify is one package now, shared with onboarding's idea of what a release version looks like. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
3bf4b7b709
commit
116e75f8cb
19 files changed
+1082
-90
No files matched your search
@@ -170,6 +170,7 @@ rather than repeated.
|
||||
| `mirror` | `Mirror` | [Server](#server) |
|
||||
| `engines` | `map[string]string` | [Server](#server) |
|
||||
| `backup_dir` | `string` | [Server](#server) |
|
||||
| `releases` | `string` | [Server](#server) |
|
||||
| `router` | `Router` | [Server](#server) |
|
||||
| `control` | `Control` | [Server](#server) |
|
||||
| `models` | `[]catalog.Model` | [Server](#server) |
|
||||
@@ -1274,6 +1275,14 @@ BackupDir is where `llmgw server backup` writes, and where the newest
|
||||
archive's age is read for llmgw_backup_last_success_timestamp_seconds.
|
||||
Empty means /var/backups/llmgw-server.
|
||||
|
||||
### `releases`
|
||||
|
||||
`string`
|
||||
|
||||
Releases is where `llmgw ctl upgrade` fetches release binaries from,
|
||||
for this machine and — through their agents — every node. Empty means
|
||||
the project's own releases.
|
||||
|
||||
### `router`
|
||||
|
||||
`Router`
|
||||
|
||||
Reference in new issue
Block a user