Ilker ManapandClaude Opus 5
c1ac2f022f
New nodes had been joined by hand, four times, by me. There was no procedure, so there was nothing to be wrong — and nothing to be right either. Two scripts now: provision-node.sh makes the VM on Proxmox, onboard-node.sh gives it an identity. They are separate because they are run by different people with different access, and the second must run where the certificate authority is, which is the one part that cannot be delegated. Running them turned up why phase 4 was never verified. FabricAttachment carried no koanf tags, so the `name:` key the shipped example documents bound to nothing and `rate_gbps` was never read at all. An agent configured exactly as instructed refused to start with "fabrics[0]: name is required" — naming the key the operator had just written. Every fabric in this project had been configured in a Go test, where the tags do not matter. The server example did not load either: its models had no source, so it failed validation the moment anything tried to parse it. Both examples are now parsed by a test. A published example that has never been loaded is a guess, and these two had been guesses since they were written. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
86 lines
2.8 KiB
Go
86 lines
2.8 KiB
Go
package engines
|
|
|
|
import (
|
|
"fmt"
|
|
"os"
|
|
"strconv"
|
|
)
|
|
|
|
// SimConfig locates the simulated engine.
|
|
type SimConfig struct {
|
|
// Bin is the llmgw binary itself; the engine is one of its subcommands.
|
|
// Empty means whatever started this agent, which is right almost always.
|
|
Bin string `koanf:"bin"`
|
|
}
|
|
|
|
// Sim launches llmgw's own simulated engine.
|
|
//
|
|
// It is in the production registry on purpose. A cluster can be brought up and
|
|
// exercised end to end — placement, fabrics, multi-node sequencing, failure and
|
|
// rebuild — on machines that have no GPUs, which is the only way most of that
|
|
// path gets tested at all. The multi-node launch sequence went four phases
|
|
// without ever running, because the only things exercising it were stub
|
|
// processes that exercised the command line and nothing else.
|
|
//
|
|
// Nothing about it pretends to be real: the model name it serves is whatever it
|
|
// is told, the tokens are invented, and it says so in its own logs.
|
|
type Sim struct{ cfg SimConfig }
|
|
|
|
// Name identifies the engine in the catalog.
|
|
func (*Sim) Name() string { return "sim" }
|
|
|
|
// HealthPath is the simulated engine's readiness endpoint.
|
|
func (*Sim) HealthPath() string { return "/health" }
|
|
|
|
// MetricsPath is its Prometheus endpoint, in vLLM's vocabulary.
|
|
func (*Sim) MetricsPath() string { return "/metrics" }
|
|
|
|
// ServesHTTP reports whether this rank answers the API.
|
|
//
|
|
// Only the leader, exactly as vLLM and SGLang behave — which is the behaviour
|
|
// this engine exists to let us test.
|
|
func (*Sim) ServesHTTP(s ShardSpec) bool { return s.Leader() }
|
|
|
|
// Validate accepts anything the arithmetic allows.
|
|
//
|
|
// No compute-capability checks and no quantization rules: there is no hardware
|
|
// to be wrong about, and inventing refusals would make this a worse stand-in
|
|
// for the engines it stands in for.
|
|
func (*Sim) Validate(p Profile, gpus []GPUInfo) error {
|
|
return validateCommon(p, gpus)
|
|
}
|
|
|
|
// Commands returns the single process that makes up one shard.
|
|
func (s *Sim) Commands(spec ShardSpec) ([]Command, error) {
|
|
bin := s.cfg.Bin
|
|
if bin == "" {
|
|
exe, err := os.Executable()
|
|
if err != nil {
|
|
return nil, fmt.Errorf("locating the llmgw binary for the sim engine: %w", err)
|
|
}
|
|
bin = exe
|
|
}
|
|
p := spec.Profile
|
|
|
|
args := []string{"sim-engine",
|
|
"--listen", "0.0.0.0:" + strconv.Itoa(spec.HTTPPort),
|
|
"--model", p.ServedModelName,
|
|
}
|
|
if spec.NNodes > 1 {
|
|
args = append(args,
|
|
"--nnodes", strconv.Itoa(spec.NNodes),
|
|
"--node-rank", strconv.Itoa(spec.Rank),
|
|
"--dist-init-addr", spec.LeaderAddr+":"+strconv.Itoa(spec.LeaderPort),
|
|
)
|
|
}
|
|
args = append(args, p.Args...)
|
|
|
|
return []Command{{
|
|
Name: "sim-engine", Path: bin, Args: args,
|
|
Env: materializeEnv(baseEnv(spec)),
|
|
// Workers dial the leader, so the leader has to be listening first.
|
|
// Both ranks express the same wait because both know the same address.
|
|
WaitForLeaderPort: spec.NNodes > 1 && !spec.Leader(),
|
|
}}, nil
|
|
}
|