Onboard a node with a script, and find two examples that never worked
New nodes had been joined by hand, four times, by me. There was no procedure, so there was nothing to be wrong — and nothing to be right either. Two scripts now: provision-node.sh makes the VM on Proxmox, onboard-node.sh gives it an identity. They are separate because they are run by different people with different access, and the second must run where the certificate authority is, which is the one part that cannot be delegated. Running them turned up why phase 4 was never verified. FabricAttachment carried no koanf tags, so the `name:` key the shipped example documents bound to nothing and `rate_gbps` was never read at all. An agent configured exactly as instructed refused to start with "fabrics[0]: name is required" — naming the key the operator had just written. Every fabric in this project had been configured in a Go test, where the tags do not matter. The server example did not load either: its models had no source, so it failed validation the moment anything tried to parse it. Both examples are now parsed by a test. A published example that has never been loaded is a guess, and these two had been guesses since they were written. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
a075ff4120
commit
c1ac2f022f
11 files changed
+687
-9
No files matched your search
@@ -206,15 +206,51 @@ control:
|
||||
port_range: [20000, 20999]
|
||||
mgmt_interface: eth0
|
||||
|
||||
# Each model needs a source — a weights path or a HuggingFace id — and at least
|
||||
# one profile if the control plane is to place it. Without profiles the model is
|
||||
# catalogued but never started, which is the router-only mode below.
|
||||
models:
|
||||
- name: llama-3.3-70b
|
||||
source: meta-llama/Llama-3.3-70B-Instruct
|
||||
aliases: [llama]
|
||||
fallbacks: [qwen2.5-32b]
|
||||
routing_profile: chat
|
||||
scaling: { min: 1, max: 4 }
|
||||
profiles:
|
||||
# Two nodes of four H100s each. A multi-node profile requires a fabric,
|
||||
# and a replica never spans two of them.
|
||||
- name: tp8
|
||||
engine: vllm
|
||||
gpu_class: h100-80g
|
||||
tp: 8
|
||||
nodes: 2
|
||||
gpus_per_node: 4
|
||||
priority: 100
|
||||
weight: 8
|
||||
startup_timeout: 30m
|
||||
# Where a whole node is free, one node is cheaper to schedule and faster
|
||||
# to start than two.
|
||||
- name: tp4-a100
|
||||
engine: vllm
|
||||
gpu_class: a100-40g
|
||||
tp: 4
|
||||
priority: 50
|
||||
weight: 3
|
||||
startup_timeout: 20m
|
||||
|
||||
- name: qwen2.5-32b
|
||||
source: Qwen/Qwen2.5-32B-Instruct
|
||||
aliases: [qwen]
|
||||
routing_profile: batch
|
||||
scaling: { min: 1, max: 2 }
|
||||
profiles:
|
||||
- name: tp2
|
||||
engine: sglang
|
||||
gpu_class: h100-80g
|
||||
tp: 2
|
||||
priority: 100
|
||||
weight: 2
|
||||
startup_timeout: 15m
|
||||
|
||||
# A static replica list, used only when no model declares a profile. That is the
|
||||
# router-only mode for testing against sim-engines; with the control plane
|
||||
|
||||
Reference in new issue
Block a user