Install a node's engines on the node, instead of trusting what is there
A node ran whatever llama-server somebody had once unpacked on it. Onboarding checked an engine only when told its path, installed none, and a node with nothing on it registered as ready. The one time that went wrong it went wrong invisibly: the binary was there, executable, and could not start for want of libgomp1, while the node reported healthy. The control plane now says which engines every node has, at which version — `engines:` in server.yaml, defaulting to the llama.cpp build this release was checked against — and each agent converges on it. What only the node can know, the node works out: the build for its processor and driver (CUDA with its runtime when an NVIDIA driver is loaded, CPU otherwise), the OS packages that build needs from its own package sources, and whether the result runs. Every archive is checked against the sha256 GitHub publishes for it and unpacked refusing any path or link that leaves the install directory. ldd names a missing library outright; running `--version` proves the rest. Only then is a directory marked installed, and it is checked again every ten minutes, because an unrelated upgrade can remove a library. An engine is reported only once installed and seen to run, or when the agent's configuration names it on purpose — never because something happened to be on PATH. The agent now says when it has probed, so an empty list means "has none" rather than "too old to say": a node rebuilt from nothing is no longer a candidate for every model at the moment it can run none of them. Placement's refusal carries the node's own words, "installing: downloading llama-b10991-bin-ubuntu-x64.tar.gz (16 MB)", and so do `ctl nodes` and the cluster page, along with a warning for a CPU without AVX2 — the VM CPU type that makes everything five times slower and nothing look wrong. Agents also report their OS, architecture, vector extensions and NVIDIA driver. Onboarding installs llmgw itself from the CI release for the node's architecture, checked against the release's checksums, rather than copying whatever binary the operator's machine had. vLLM and SGLang are not installed yet: they are Python environments of several gigabytes, not an archive, and still have to be named in the agent's configuration. Also fixes a race on the weights fingerprint, which two heartbeat loops touch during a failover. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
fa7f90880d
commit
7d97c6b74f
27 files changed
+2155
-192
No files matched your search
+475
-103
@@ -90,7 +90,7 @@ func (x ShardStatus_State) Number() protoreflect.EnumNumber {
|
||||
|
||||
// Deprecated: Use ShardStatus_State.Descriptor instead.
|
||||
func (ShardStatus_State) EnumDescriptor() ([]byte, []int) {
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{9, 0}
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{13, 0}
|
||||
}
|
||||
|
||||
// AgentMessage is anything a node tells the control plane.
|
||||
@@ -202,6 +202,7 @@ type ControlMessage struct {
|
||||
// *ControlMessage_Drain
|
||||
// *ControlMessage_Ack
|
||||
// *ControlMessage_RemoveWeights
|
||||
// *ControlMessage_Provision
|
||||
Msg isControlMessage_Msg `protobuf_oneof:"msg"`
|
||||
unknownFields protoimpl.UnknownFields
|
||||
sizeCache protoimpl.SizeCache
|
||||
@@ -289,6 +290,15 @@ func (x *ControlMessage) GetRemoveWeights() *RemoveWeights {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (x *ControlMessage) GetProvision() *Provision {
|
||||
if x != nil {
|
||||
if x, ok := x.Msg.(*ControlMessage_Provision); ok {
|
||||
return x.Provision
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
type isControlMessage_Msg interface {
|
||||
isControlMessage_Msg()
|
||||
}
|
||||
@@ -313,6 +323,10 @@ type ControlMessage_RemoveWeights struct {
|
||||
RemoveWeights *RemoveWeights `protobuf:"bytes,5,opt,name=remove_weights,json=removeWeights,proto3,oneof"`
|
||||
}
|
||||
|
||||
type ControlMessage_Provision struct {
|
||||
Provision *Provision `protobuf:"bytes,6,opt,name=provision,proto3,oneof"`
|
||||
}
|
||||
|
||||
func (*ControlMessage_Start) isControlMessage_Msg() {}
|
||||
|
||||
func (*ControlMessage_Stop) isControlMessage_Msg() {}
|
||||
@@ -323,6 +337,8 @@ func (*ControlMessage_Ack) isControlMessage_Msg() {}
|
||||
|
||||
func (*ControlMessage_RemoveWeights) isControlMessage_Msg() {}
|
||||
|
||||
func (*ControlMessage_Provision) isControlMessage_Msg() {}
|
||||
|
||||
// Register is the first message on every stream. It carries the node's full
|
||||
// hardware inventory, so the control plane needs no separate discovery call and
|
||||
// a reconnecting agent automatically refreshes what the scheduler knows.
|
||||
@@ -381,7 +397,15 @@ type Register struct {
|
||||
//
|
||||
// Empty means the node was not asked to look, exactly as for weights, and
|
||||
// constrains nothing.
|
||||
Storage []*Storage `protobuf:"bytes,12,rep,name=storage,proto3" json:"storage,omitempty"`
|
||||
Storage []*Storage `protobuf:"bytes,12,rep,name=storage,proto3" json:"storage,omitempty"`
|
||||
// host is what the machine is, so the control plane can have the mirror
|
||||
// fetch software built for it rather than assuming every node matches the
|
||||
// first.
|
||||
Host *Host `protobuf:"bytes,13,opt,name=host,proto3" json:"host,omitempty"`
|
||||
// engines_probed says engine_versions is an answer. Without it an empty map
|
||||
// meant "an agent too old to say" and constrained nothing — so a freshly
|
||||
// built node with no engine installed yet was a candidate for every model.
|
||||
EnginesProbed bool `protobuf:"varint,14,opt,name=engines_probed,json=enginesProbed,proto3" json:"engines_probed,omitempty"`
|
||||
unknownFields protoimpl.UnknownFields
|
||||
sizeCache protoimpl.SizeCache
|
||||
}
|
||||
@@ -500,6 +524,113 @@ func (x *Register) GetStorage() []*Storage {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (x *Register) GetHost() *Host {
|
||||
if x != nil {
|
||||
return x.Host
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (x *Register) GetEnginesProbed() bool {
|
||||
if x != nil {
|
||||
return x.EnginesProbed
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// Host describes the operating system and processor of a node.
|
||||
type Host struct {
|
||||
state protoimpl.MessageState `protogen:"open.v1"`
|
||||
// From /etc/os-release: "ubuntu", "24.04", "noble".
|
||||
OsId string `protobuf:"bytes,1,opt,name=os_id,json=osId,proto3" json:"os_id,omitempty"`
|
||||
OsVersion string `protobuf:"bytes,2,opt,name=os_version,json=osVersion,proto3" json:"os_version,omitempty"`
|
||||
OsCodename string `protobuf:"bytes,3,opt,name=os_codename,json=osCodename,proto3" json:"os_codename,omitempty"`
|
||||
// Debian's spelling, which is what package indexes are keyed by: "amd64",
|
||||
// "arm64".
|
||||
Arch string `protobuf:"bytes,4,opt,name=arch,proto3" json:"arch,omitempty"`
|
||||
// The vector extensions inference cares about, of those the CPU has. A VM
|
||||
// given QEMU's default CPU model has none of them, and runs, and is five
|
||||
// times slower, and nothing says why.
|
||||
CpuFlags []string `protobuf:"bytes,5,rep,name=cpu_flags,json=cpuFlags,proto3" json:"cpu_flags,omitempty"`
|
||||
// nvidia_driver is the driver version, empty when there is none. A card
|
||||
// without a driver is not a GPU anything can use.
|
||||
NvidiaDriver string `protobuf:"bytes,6,opt,name=nvidia_driver,json=nvidiaDriver,proto3" json:"nvidia_driver,omitempty"`
|
||||
unknownFields protoimpl.UnknownFields
|
||||
sizeCache protoimpl.SizeCache
|
||||
}
|
||||
|
||||
func (x *Host) Reset() {
|
||||
*x = Host{}
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[3]
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
ms.StoreMessageInfo(mi)
|
||||
}
|
||||
|
||||
func (x *Host) String() string {
|
||||
return protoimpl.X.MessageStringOf(x)
|
||||
}
|
||||
|
||||
func (*Host) ProtoMessage() {}
|
||||
|
||||
func (x *Host) ProtoReflect() protoreflect.Message {
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[3]
|
||||
if x != nil {
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
if ms.LoadMessageInfo() == nil {
|
||||
ms.StoreMessageInfo(mi)
|
||||
}
|
||||
return ms
|
||||
}
|
||||
return mi.MessageOf(x)
|
||||
}
|
||||
|
||||
// Deprecated: Use Host.ProtoReflect.Descriptor instead.
|
||||
func (*Host) Descriptor() ([]byte, []int) {
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{3}
|
||||
}
|
||||
|
||||
func (x *Host) GetOsId() string {
|
||||
if x != nil {
|
||||
return x.OsId
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func (x *Host) GetOsVersion() string {
|
||||
if x != nil {
|
||||
return x.OsVersion
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func (x *Host) GetOsCodename() string {
|
||||
if x != nil {
|
||||
return x.OsCodename
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func (x *Host) GetArch() string {
|
||||
if x != nil {
|
||||
return x.Arch
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func (x *Host) GetCpuFlags() []string {
|
||||
if x != nil {
|
||||
return x.CpuFlags
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (x *Host) GetNvidiaDriver() string {
|
||||
if x != nil {
|
||||
return x.NvidiaDriver
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// Storage is one filesystem a node keeps weights on.
|
||||
//
|
||||
// Per filesystem rather than per configured directory: free space belongs to
|
||||
@@ -522,7 +653,7 @@ type Storage struct {
|
||||
|
||||
func (x *Storage) Reset() {
|
||||
*x = Storage{}
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[3]
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[4]
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
ms.StoreMessageInfo(mi)
|
||||
}
|
||||
@@ -534,7 +665,7 @@ func (x *Storage) String() string {
|
||||
func (*Storage) ProtoMessage() {}
|
||||
|
||||
func (x *Storage) ProtoReflect() protoreflect.Message {
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[3]
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[4]
|
||||
if x != nil {
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
if ms.LoadMessageInfo() == nil {
|
||||
@@ -547,7 +678,7 @@ func (x *Storage) ProtoReflect() protoreflect.Message {
|
||||
|
||||
// Deprecated: Use Storage.ProtoReflect.Descriptor instead.
|
||||
func (*Storage) Descriptor() ([]byte, []int) {
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{3}
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{4}
|
||||
}
|
||||
|
||||
func (x *Storage) GetPath() string {
|
||||
@@ -614,7 +745,7 @@ type RunningShard struct {
|
||||
|
||||
func (x *RunningShard) Reset() {
|
||||
*x = RunningShard{}
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[4]
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[5]
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
ms.StoreMessageInfo(mi)
|
||||
}
|
||||
@@ -626,7 +757,7 @@ func (x *RunningShard) String() string {
|
||||
func (*RunningShard) ProtoMessage() {}
|
||||
|
||||
func (x *RunningShard) ProtoReflect() protoreflect.Message {
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[4]
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[5]
|
||||
if x != nil {
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
if ms.LoadMessageInfo() == nil {
|
||||
@@ -639,7 +770,7 @@ func (x *RunningShard) ProtoReflect() protoreflect.Message {
|
||||
|
||||
// Deprecated: Use RunningShard.ProtoReflect.Descriptor instead.
|
||||
func (*RunningShard) Descriptor() ([]byte, []int) {
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{4}
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{5}
|
||||
}
|
||||
|
||||
func (x *RunningShard) GetReplicaId() string {
|
||||
@@ -776,7 +907,7 @@ type GPU struct {
|
||||
|
||||
func (x *GPU) Reset() {
|
||||
*x = GPU{}
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[5]
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[6]
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
ms.StoreMessageInfo(mi)
|
||||
}
|
||||
@@ -788,7 +919,7 @@ func (x *GPU) String() string {
|
||||
func (*GPU) ProtoMessage() {}
|
||||
|
||||
func (x *GPU) ProtoReflect() protoreflect.Message {
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[5]
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[6]
|
||||
if x != nil {
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
if ms.LoadMessageInfo() == nil {
|
||||
@@ -801,7 +932,7 @@ func (x *GPU) ProtoReflect() protoreflect.Message {
|
||||
|
||||
// Deprecated: Use GPU.ProtoReflect.Descriptor instead.
|
||||
func (*GPU) Descriptor() ([]byte, []int) {
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{5}
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{6}
|
||||
}
|
||||
|
||||
func (x *GPU) GetIndex() int32 {
|
||||
@@ -923,7 +1054,7 @@ type FabricAttachment struct {
|
||||
|
||||
func (x *FabricAttachment) Reset() {
|
||||
*x = FabricAttachment{}
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[6]
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[7]
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
ms.StoreMessageInfo(mi)
|
||||
}
|
||||
@@ -935,7 +1066,7 @@ func (x *FabricAttachment) String() string {
|
||||
func (*FabricAttachment) ProtoMessage() {}
|
||||
|
||||
func (x *FabricAttachment) ProtoReflect() protoreflect.Message {
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[6]
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[7]
|
||||
if x != nil {
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
if ms.LoadMessageInfo() == nil {
|
||||
@@ -948,7 +1079,7 @@ func (x *FabricAttachment) ProtoReflect() protoreflect.Message {
|
||||
|
||||
// Deprecated: Use FabricAttachment.ProtoReflect.Descriptor instead.
|
||||
func (*FabricAttachment) Descriptor() ([]byte, []int) {
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{6}
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{7}
|
||||
}
|
||||
|
||||
func (x *FabricAttachment) GetFabric() string {
|
||||
@@ -990,7 +1121,7 @@ type LocalWeights struct {
|
||||
|
||||
func (x *LocalWeights) Reset() {
|
||||
*x = LocalWeights{}
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[7]
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[8]
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
ms.StoreMessageInfo(mi)
|
||||
}
|
||||
@@ -1002,7 +1133,7 @@ func (x *LocalWeights) String() string {
|
||||
func (*LocalWeights) ProtoMessage() {}
|
||||
|
||||
func (x *LocalWeights) ProtoReflect() protoreflect.Message {
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[7]
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[8]
|
||||
if x != nil {
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
if ms.LoadMessageInfo() == nil {
|
||||
@@ -1015,7 +1146,7 @@ func (x *LocalWeights) ProtoReflect() protoreflect.Message {
|
||||
|
||||
// Deprecated: Use LocalWeights.ProtoReflect.Descriptor instead.
|
||||
func (*LocalWeights) Descriptor() ([]byte, []int) {
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{7}
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{8}
|
||||
}
|
||||
|
||||
func (x *LocalWeights) GetPath() string {
|
||||
@@ -1070,13 +1201,18 @@ type Heartbeat struct {
|
||||
// somebody restarts the agent — which is a step nobody knows to take.
|
||||
EnginesChanged bool `protobuf:"varint,8,opt,name=engines_changed,json=enginesChanged,proto3" json:"engines_changed,omitempty"`
|
||||
EngineVersions map[string]string `protobuf:"bytes,9,rep,name=engine_versions,json=engineVersions,proto3" json:"engine_versions,omitempty" protobuf_key:"bytes,1,opt,name=key" protobuf_val:"bytes,2,opt,name=value"`
|
||||
unknownFields protoimpl.UnknownFields
|
||||
sizeCache protoimpl.SizeCache
|
||||
// provision is where installing the node's software stands, sent when it
|
||||
// changed. The flag makes an empty list mean "nothing to install" rather
|
||||
// than "nothing to say", as for weights.
|
||||
ProvisionChanged bool `protobuf:"varint,10,opt,name=provision_changed,json=provisionChanged,proto3" json:"provision_changed,omitempty"`
|
||||
Provision []*ProvisionStatus `protobuf:"bytes,11,rep,name=provision,proto3" json:"provision,omitempty"`
|
||||
unknownFields protoimpl.UnknownFields
|
||||
sizeCache protoimpl.SizeCache
|
||||
}
|
||||
|
||||
func (x *Heartbeat) Reset() {
|
||||
*x = Heartbeat{}
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[8]
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[9]
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
ms.StoreMessageInfo(mi)
|
||||
}
|
||||
@@ -1088,7 +1224,7 @@ func (x *Heartbeat) String() string {
|
||||
func (*Heartbeat) ProtoMessage() {}
|
||||
|
||||
func (x *Heartbeat) ProtoReflect() protoreflect.Message {
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[8]
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[9]
|
||||
if x != nil {
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
if ms.LoadMessageInfo() == nil {
|
||||
@@ -1101,7 +1237,7 @@ func (x *Heartbeat) ProtoReflect() protoreflect.Message {
|
||||
|
||||
// Deprecated: Use Heartbeat.ProtoReflect.Descriptor instead.
|
||||
func (*Heartbeat) Descriptor() ([]byte, []int) {
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{8}
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{9}
|
||||
}
|
||||
|
||||
func (x *Heartbeat) GetNodeId() string {
|
||||
@@ -1167,6 +1303,206 @@ func (x *Heartbeat) GetEngineVersions() map[string]string {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (x *Heartbeat) GetProvisionChanged() bool {
|
||||
if x != nil {
|
||||
return x.ProvisionChanged
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func (x *Heartbeat) GetProvision() []*ProvisionStatus {
|
||||
if x != nil {
|
||||
return x.Provision
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Provision is the software a node should have: which engines, at which
|
||||
// versions. Nothing more.
|
||||
//
|
||||
// Everything else is the node's to work out, because only the node knows it:
|
||||
// which build fits its hardware, where that build is published, what its
|
||||
// operating system is missing, and whether the result actually runs. The agent
|
||||
// converges on this — installs what is absent, checks what is there — and
|
||||
// reports each engine through ProvisionStatus. Nothing is taken on trust from
|
||||
// what happens to be on the machine: an engine is offered to placement only
|
||||
// once the agent has installed it and seen it run.
|
||||
type Provision struct {
|
||||
state protoimpl.MessageState `protogen:"open.v1"`
|
||||
Engines []*EngineWant `protobuf:"bytes,1,rep,name=engines,proto3" json:"engines,omitempty"`
|
||||
unknownFields protoimpl.UnknownFields
|
||||
sizeCache protoimpl.SizeCache
|
||||
}
|
||||
|
||||
func (x *Provision) Reset() {
|
||||
*x = Provision{}
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[10]
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
ms.StoreMessageInfo(mi)
|
||||
}
|
||||
|
||||
func (x *Provision) String() string {
|
||||
return protoimpl.X.MessageStringOf(x)
|
||||
}
|
||||
|
||||
func (*Provision) ProtoMessage() {}
|
||||
|
||||
func (x *Provision) ProtoReflect() protoreflect.Message {
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[10]
|
||||
if x != nil {
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
if ms.LoadMessageInfo() == nil {
|
||||
ms.StoreMessageInfo(mi)
|
||||
}
|
||||
return ms
|
||||
}
|
||||
return mi.MessageOf(x)
|
||||
}
|
||||
|
||||
// Deprecated: Use Provision.ProtoReflect.Descriptor instead.
|
||||
func (*Provision) Descriptor() ([]byte, []int) {
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{10}
|
||||
}
|
||||
|
||||
func (x *Provision) GetEngines() []*EngineWant {
|
||||
if x != nil {
|
||||
return x.Engines
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// EngineWant names one engine at one version.
|
||||
type EngineWant struct {
|
||||
state protoimpl.MessageState `protogen:"open.v1"`
|
||||
Engine string `protobuf:"bytes,1,opt,name=engine,proto3" json:"engine,omitempty"` // "llamacpp"
|
||||
Version string `protobuf:"bytes,2,opt,name=version,proto3" json:"version,omitempty"` // "b10991"
|
||||
unknownFields protoimpl.UnknownFields
|
||||
sizeCache protoimpl.SizeCache
|
||||
}
|
||||
|
||||
func (x *EngineWant) Reset() {
|
||||
*x = EngineWant{}
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[11]
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
ms.StoreMessageInfo(mi)
|
||||
}
|
||||
|
||||
func (x *EngineWant) String() string {
|
||||
return protoimpl.X.MessageStringOf(x)
|
||||
}
|
||||
|
||||
func (*EngineWant) ProtoMessage() {}
|
||||
|
||||
func (x *EngineWant) ProtoReflect() protoreflect.Message {
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[11]
|
||||
if x != nil {
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
if ms.LoadMessageInfo() == nil {
|
||||
ms.StoreMessageInfo(mi)
|
||||
}
|
||||
return ms
|
||||
}
|
||||
return mi.MessageOf(x)
|
||||
}
|
||||
|
||||
// Deprecated: Use EngineWant.ProtoReflect.Descriptor instead.
|
||||
func (*EngineWant) Descriptor() ([]byte, []int) {
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{11}
|
||||
}
|
||||
|
||||
func (x *EngineWant) GetEngine() string {
|
||||
if x != nil {
|
||||
return x.Engine
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func (x *EngineWant) GetVersion() string {
|
||||
if x != nil {
|
||||
return x.Version
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// ProvisionStatus is where one engine's installation stands on a node.
|
||||
type ProvisionStatus struct {
|
||||
state protoimpl.MessageState `protogen:"open.v1"`
|
||||
Engine string `protobuf:"bytes,1,opt,name=engine,proto3" json:"engine,omitempty"`
|
||||
Version string `protobuf:"bytes,2,opt,name=version,proto3" json:"version,omitempty"`
|
||||
Variant string `protobuf:"bytes,3,opt,name=variant,proto3" json:"variant,omitempty"`
|
||||
// "installing", "ready", "failed", or "unsupported" when no build fits.
|
||||
State string `protobuf:"bytes,4,opt,name=state,proto3" json:"state,omitempty"`
|
||||
// detail says what is happening or what went wrong, in words for a person.
|
||||
Detail string `protobuf:"bytes,5,opt,name=detail,proto3" json:"detail,omitempty"`
|
||||
unknownFields protoimpl.UnknownFields
|
||||
sizeCache protoimpl.SizeCache
|
||||
}
|
||||
|
||||
func (x *ProvisionStatus) Reset() {
|
||||
*x = ProvisionStatus{}
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[12]
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
ms.StoreMessageInfo(mi)
|
||||
}
|
||||
|
||||
func (x *ProvisionStatus) String() string {
|
||||
return protoimpl.X.MessageStringOf(x)
|
||||
}
|
||||
|
||||
func (*ProvisionStatus) ProtoMessage() {}
|
||||
|
||||
func (x *ProvisionStatus) ProtoReflect() protoreflect.Message {
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[12]
|
||||
if x != nil {
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
if ms.LoadMessageInfo() == nil {
|
||||
ms.StoreMessageInfo(mi)
|
||||
}
|
||||
return ms
|
||||
}
|
||||
return mi.MessageOf(x)
|
||||
}
|
||||
|
||||
// Deprecated: Use ProvisionStatus.ProtoReflect.Descriptor instead.
|
||||
func (*ProvisionStatus) Descriptor() ([]byte, []int) {
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{12}
|
||||
}
|
||||
|
||||
func (x *ProvisionStatus) GetEngine() string {
|
||||
if x != nil {
|
||||
return x.Engine
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func (x *ProvisionStatus) GetVersion() string {
|
||||
if x != nil {
|
||||
return x.Version
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func (x *ProvisionStatus) GetVariant() string {
|
||||
if x != nil {
|
||||
return x.Variant
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func (x *ProvisionStatus) GetState() string {
|
||||
if x != nil {
|
||||
return x.State
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func (x *ProvisionStatus) GetDetail() string {
|
||||
if x != nil {
|
||||
return x.Detail
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// ShardStatus reports a shard's lifecycle transition.
|
||||
type ShardStatus struct {
|
||||
state protoimpl.MessageState `protogen:"open.v1"`
|
||||
@@ -1193,7 +1529,7 @@ type ShardStatus struct {
|
||||
|
||||
func (x *ShardStatus) Reset() {
|
||||
*x = ShardStatus{}
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[9]
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[13]
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
ms.StoreMessageInfo(mi)
|
||||
}
|
||||
@@ -1205,7 +1541,7 @@ func (x *ShardStatus) String() string {
|
||||
func (*ShardStatus) ProtoMessage() {}
|
||||
|
||||
func (x *ShardStatus) ProtoReflect() protoreflect.Message {
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[9]
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[13]
|
||||
if x != nil {
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
if ms.LoadMessageInfo() == nil {
|
||||
@@ -1218,7 +1554,7 @@ func (x *ShardStatus) ProtoReflect() protoreflect.Message {
|
||||
|
||||
// Deprecated: Use ShardStatus.ProtoReflect.Descriptor instead.
|
||||
func (*ShardStatus) Descriptor() ([]byte, []int) {
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{9}
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{13}
|
||||
}
|
||||
|
||||
func (x *ShardStatus) GetNodeId() string {
|
||||
@@ -1292,7 +1628,7 @@ type FetchProgress struct {
|
||||
|
||||
func (x *FetchProgress) Reset() {
|
||||
*x = FetchProgress{}
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[10]
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[14]
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
ms.StoreMessageInfo(mi)
|
||||
}
|
||||
@@ -1304,7 +1640,7 @@ func (x *FetchProgress) String() string {
|
||||
func (*FetchProgress) ProtoMessage() {}
|
||||
|
||||
func (x *FetchProgress) ProtoReflect() protoreflect.Message {
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[10]
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[14]
|
||||
if x != nil {
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
if ms.LoadMessageInfo() == nil {
|
||||
@@ -1317,7 +1653,7 @@ func (x *FetchProgress) ProtoReflect() protoreflect.Message {
|
||||
|
||||
// Deprecated: Use FetchProgress.ProtoReflect.Descriptor instead.
|
||||
func (*FetchProgress) Descriptor() ([]byte, []int) {
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{10}
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{14}
|
||||
}
|
||||
|
||||
func (x *FetchProgress) GetSource() string {
|
||||
@@ -1373,7 +1709,7 @@ type StartShard struct {
|
||||
|
||||
func (x *StartShard) Reset() {
|
||||
*x = StartShard{}
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[11]
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[15]
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
ms.StoreMessageInfo(mi)
|
||||
}
|
||||
@@ -1385,7 +1721,7 @@ func (x *StartShard) String() string {
|
||||
func (*StartShard) ProtoMessage() {}
|
||||
|
||||
func (x *StartShard) ProtoReflect() protoreflect.Message {
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[11]
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[15]
|
||||
if x != nil {
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
if ms.LoadMessageInfo() == nil {
|
||||
@@ -1398,7 +1734,7 @@ func (x *StartShard) ProtoReflect() protoreflect.Message {
|
||||
|
||||
// Deprecated: Use StartShard.ProtoReflect.Descriptor instead.
|
||||
func (*StartShard) Descriptor() ([]byte, []int) {
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{11}
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{15}
|
||||
}
|
||||
|
||||
func (x *StartShard) GetReplicaId() string {
|
||||
@@ -1525,7 +1861,7 @@ type EngineProfile struct {
|
||||
|
||||
func (x *EngineProfile) Reset() {
|
||||
*x = EngineProfile{}
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[12]
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[16]
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
ms.StoreMessageInfo(mi)
|
||||
}
|
||||
@@ -1537,7 +1873,7 @@ func (x *EngineProfile) String() string {
|
||||
func (*EngineProfile) ProtoMessage() {}
|
||||
|
||||
func (x *EngineProfile) ProtoReflect() protoreflect.Message {
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[12]
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[16]
|
||||
if x != nil {
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
if ms.LoadMessageInfo() == nil {
|
||||
@@ -1550,7 +1886,7 @@ func (x *EngineProfile) ProtoReflect() protoreflect.Message {
|
||||
|
||||
// Deprecated: Use EngineProfile.ProtoReflect.Descriptor instead.
|
||||
func (*EngineProfile) Descriptor() ([]byte, []int) {
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{12}
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{16}
|
||||
}
|
||||
|
||||
func (x *EngineProfile) GetName() string {
|
||||
@@ -1646,7 +1982,7 @@ type RemoveWeights struct {
|
||||
|
||||
func (x *RemoveWeights) Reset() {
|
||||
*x = RemoveWeights{}
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[13]
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[17]
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
ms.StoreMessageInfo(mi)
|
||||
}
|
||||
@@ -1658,7 +1994,7 @@ func (x *RemoveWeights) String() string {
|
||||
func (*RemoveWeights) ProtoMessage() {}
|
||||
|
||||
func (x *RemoveWeights) ProtoReflect() protoreflect.Message {
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[13]
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[17]
|
||||
if x != nil {
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
if ms.LoadMessageInfo() == nil {
|
||||
@@ -1671,7 +2007,7 @@ func (x *RemoveWeights) ProtoReflect() protoreflect.Message {
|
||||
|
||||
// Deprecated: Use RemoveWeights.ProtoReflect.Descriptor instead.
|
||||
func (*RemoveWeights) Descriptor() ([]byte, []int) {
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{13}
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{17}
|
||||
}
|
||||
|
||||
func (x *RemoveWeights) GetPaths() []string {
|
||||
@@ -1691,7 +2027,7 @@ type StopShard struct {
|
||||
|
||||
func (x *StopShard) Reset() {
|
||||
*x = StopShard{}
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[14]
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[18]
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
ms.StoreMessageInfo(mi)
|
||||
}
|
||||
@@ -1703,7 +2039,7 @@ func (x *StopShard) String() string {
|
||||
func (*StopShard) ProtoMessage() {}
|
||||
|
||||
func (x *StopShard) ProtoReflect() protoreflect.Message {
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[14]
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[18]
|
||||
if x != nil {
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
if ms.LoadMessageInfo() == nil {
|
||||
@@ -1716,7 +2052,7 @@ func (x *StopShard) ProtoReflect() protoreflect.Message {
|
||||
|
||||
// Deprecated: Use StopShard.ProtoReflect.Descriptor instead.
|
||||
func (*StopShard) Descriptor() ([]byte, []int) {
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{14}
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{18}
|
||||
}
|
||||
|
||||
func (x *StopShard) GetShardId() string {
|
||||
@@ -1736,7 +2072,7 @@ type DrainShard struct {
|
||||
|
||||
func (x *DrainShard) Reset() {
|
||||
*x = DrainShard{}
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[15]
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[19]
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
ms.StoreMessageInfo(mi)
|
||||
}
|
||||
@@ -1748,7 +2084,7 @@ func (x *DrainShard) String() string {
|
||||
func (*DrainShard) ProtoMessage() {}
|
||||
|
||||
func (x *DrainShard) ProtoReflect() protoreflect.Message {
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[15]
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[19]
|
||||
if x != nil {
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
if ms.LoadMessageInfo() == nil {
|
||||
@@ -1761,7 +2097,7 @@ func (x *DrainShard) ProtoReflect() protoreflect.Message {
|
||||
|
||||
// Deprecated: Use DrainShard.ProtoReflect.Descriptor instead.
|
||||
func (*DrainShard) Descriptor() ([]byte, []int) {
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{15}
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{19}
|
||||
}
|
||||
|
||||
func (x *DrainShard) GetShardId() string {
|
||||
@@ -1783,7 +2119,7 @@ type Ack struct {
|
||||
|
||||
func (x *Ack) Reset() {
|
||||
*x = Ack{}
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[16]
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[20]
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
ms.StoreMessageInfo(mi)
|
||||
}
|
||||
@@ -1795,7 +2131,7 @@ func (x *Ack) String() string {
|
||||
func (*Ack) ProtoMessage() {}
|
||||
|
||||
func (x *Ack) ProtoReflect() protoreflect.Message {
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[16]
|
||||
mi := &file_llmgw_v1_agent_proto_msgTypes[20]
|
||||
if x != nil {
|
||||
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
|
||||
if ms.LoadMessageInfo() == nil {
|
||||
@@ -1808,7 +2144,7 @@ func (x *Ack) ProtoReflect() protoreflect.Message {
|
||||
|
||||
// Deprecated: Use Ack.ProtoReflect.Descriptor instead.
|
||||
func (*Ack) Descriptor() ([]byte, []int) {
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{16}
|
||||
return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{20}
|
||||
}
|
||||
|
||||
func (x *Ack) GetMessageId() string {
|
||||
@@ -1841,14 +2177,15 @@ const file_llmgw_v1_agent_proto_rawDesc = "" +
|
||||
"\bregister\x18\x01 \x01(\v2\x12.llmgw.v1.RegisterH\x00R\bregister\x123\n" +
|
||||
"\theartbeat\x18\x02 \x01(\v2\x13.llmgw.v1.HeartbeatH\x00R\theartbeat\x12-\n" +
|
||||
"\x05shard\x18\x03 \x01(\v2\x15.llmgw.v1.ShardStatusH\x00R\x05shardB\x05\n" +
|
||||
"\x03msg\"\x83\x02\n" +
|
||||
"\x03msg\"\xb8\x02\n" +
|
||||
"\x0eControlMessage\x12,\n" +
|
||||
"\x05start\x18\x01 \x01(\v2\x14.llmgw.v1.StartShardH\x00R\x05start\x12)\n" +
|
||||
"\x04stop\x18\x02 \x01(\v2\x13.llmgw.v1.StopShardH\x00R\x04stop\x12,\n" +
|
||||
"\x05drain\x18\x03 \x01(\v2\x14.llmgw.v1.DrainShardH\x00R\x05drain\x12!\n" +
|
||||
"\x03ack\x18\x04 \x01(\v2\r.llmgw.v1.AckH\x00R\x03ack\x12@\n" +
|
||||
"\x0eremove_weights\x18\x05 \x01(\v2\x17.llmgw.v1.RemoveWeightsH\x00R\rremoveWeightsB\x05\n" +
|
||||
"\x03msg\"\xa7\x05\n" +
|
||||
"\x0eremove_weights\x18\x05 \x01(\v2\x17.llmgw.v1.RemoveWeightsH\x00R\rremoveWeights\x123\n" +
|
||||
"\tprovision\x18\x06 \x01(\v2\x13.llmgw.v1.ProvisionH\x00R\tprovisionB\x05\n" +
|
||||
"\x03msg\"\xf2\x05\n" +
|
||||
"\bRegister\x12\x17\n" +
|
||||
"\anode_id\x18\x01 \x01(\tR\x06nodeId\x12\x1a\n" +
|
||||
"\bhostname\x18\x02 \x01(\tR\bhostname\x12\x1b\n" +
|
||||
@@ -1862,13 +2199,24 @@ const file_llmgw_v1_agent_proto_rawDesc = "" +
|
||||
"\x0fmemory_total_mb\x18\n" +
|
||||
" \x01(\x03R\rmemoryTotalMb\x120\n" +
|
||||
"\aweights\x18\v \x03(\v2\x16.llmgw.v1.LocalWeightsR\aweights\x12+\n" +
|
||||
"\astorage\x18\f \x03(\v2\x11.llmgw.v1.StorageR\astorage\x1a9\n" +
|
||||
"\astorage\x18\f \x03(\v2\x11.llmgw.v1.StorageR\astorage\x12\"\n" +
|
||||
"\x04host\x18\r \x01(\v2\x0e.llmgw.v1.HostR\x04host\x12%\n" +
|
||||
"\x0eengines_probed\x18\x0e \x01(\bR\renginesProbed\x1a9\n" +
|
||||
"\vLabelsEntry\x12\x10\n" +
|
||||
"\x03key\x18\x01 \x01(\tR\x03key\x12\x14\n" +
|
||||
"\x05value\x18\x02 \x01(\tR\x05value:\x028\x01\x1aA\n" +
|
||||
"\x13EngineVersionsEntry\x12\x10\n" +
|
||||
"\x03key\x18\x01 \x01(\tR\x03key\x12\x14\n" +
|
||||
"\x05value\x18\x02 \x01(\tR\x05value:\x028\x01\"u\n" +
|
||||
"\x05value\x18\x02 \x01(\tR\x05value:\x028\x01\"\xb1\x01\n" +
|
||||
"\x04Host\x12\x13\n" +
|
||||
"\x05os_id\x18\x01 \x01(\tR\x04osId\x12\x1d\n" +
|
||||
"\n" +
|
||||
"os_version\x18\x02 \x01(\tR\tosVersion\x12\x1f\n" +
|
||||
"\vos_codename\x18\x03 \x01(\tR\n" +
|
||||
"osCodename\x12\x12\n" +
|
||||
"\x04arch\x18\x04 \x01(\tR\x04arch\x12\x1b\n" +
|
||||
"\tcpu_flags\x18\x05 \x03(\tR\bcpuFlags\x12#\n" +
|
||||
"\rnvidia_driver\x18\x06 \x01(\tR\fnvidiaDriver\"u\n" +
|
||||
"\aStorage\x12\x12\n" +
|
||||
"\x04path\x18\x01 \x01(\tR\x04path\x12\x1d\n" +
|
||||
"\n" +
|
||||
@@ -1925,7 +2273,7 @@ const file_llmgw_v1_agent_proto_rawDesc = "" +
|
||||
"\x04path\x18\x01 \x01(\tR\x04path\x12\x1d\n" +
|
||||
"\n" +
|
||||
"size_bytes\x18\x02 \x01(\x03R\tsizeBytes\x12\x16\n" +
|
||||
"\x06shared\x18\x03 \x01(\bR\x06shared\"\xe2\x03\n" +
|
||||
"\x06shared\x18\x03 \x01(\bR\x06shared\"\xc8\x04\n" +
|
||||
"\tHeartbeat\x12\x17\n" +
|
||||
"\anode_id\x18\x01 \x01(\tR\x06nodeId\x12+\n" +
|
||||
"\x12gpu_memory_used_mb\x18\x02 \x03(\x05R\x0fgpuMemoryUsedMb\x12!\n" +
|
||||
@@ -1935,10 +2283,25 @@ const file_llmgw_v1_agent_proto_rawDesc = "" +
|
||||
"\aweights\x18\x06 \x03(\v2\x16.llmgw.v1.LocalWeightsR\aweights\x12&\n" +
|
||||
"\x0fmemory_total_mb\x18\a \x01(\x03R\rmemoryTotalMb\x12'\n" +
|
||||
"\x0fengines_changed\x18\b \x01(\bR\x0eenginesChanged\x12P\n" +
|
||||
"\x0fengine_versions\x18\t \x03(\v2'.llmgw.v1.Heartbeat.EngineVersionsEntryR\x0eengineVersions\x1aA\n" +
|
||||
"\x0fengine_versions\x18\t \x03(\v2'.llmgw.v1.Heartbeat.EngineVersionsEntryR\x0eengineVersions\x12+\n" +
|
||||
"\x11provision_changed\x18\n" +
|
||||
" \x01(\bR\x10provisionChanged\x127\n" +
|
||||
"\tprovision\x18\v \x03(\v2\x19.llmgw.v1.ProvisionStatusR\tprovision\x1aA\n" +
|
||||
"\x13EngineVersionsEntry\x12\x10\n" +
|
||||
"\x03key\x18\x01 \x01(\tR\x03key\x12\x14\n" +
|
||||
"\x05value\x18\x02 \x01(\tR\x05value:\x028\x01\"\xbb\x03\n" +
|
||||
"\x05value\x18\x02 \x01(\tR\x05value:\x028\x01\";\n" +
|
||||
"\tProvision\x12.\n" +
|
||||
"\aengines\x18\x01 \x03(\v2\x14.llmgw.v1.EngineWantR\aengines\">\n" +
|
||||
"\n" +
|
||||
"EngineWant\x12\x16\n" +
|
||||
"\x06engine\x18\x01 \x01(\tR\x06engine\x12\x18\n" +
|
||||
"\aversion\x18\x02 \x01(\tR\aversion\"\x8b\x01\n" +
|
||||
"\x0fProvisionStatus\x12\x16\n" +
|
||||
"\x06engine\x18\x01 \x01(\tR\x06engine\x12\x18\n" +
|
||||
"\aversion\x18\x02 \x01(\tR\aversion\x12\x18\n" +
|
||||
"\avariant\x18\x03 \x01(\tR\avariant\x12\x14\n" +
|
||||
"\x05state\x18\x04 \x01(\tR\x05state\x12\x16\n" +
|
||||
"\x06detail\x18\x05 \x01(\tR\x06detail\"\xbb\x03\n" +
|
||||
"\vShardStatus\x12\x17\n" +
|
||||
"\anode_id\x18\x01 \x01(\tR\x06nodeId\x12\x1d\n" +
|
||||
"\n" +
|
||||
@@ -2035,62 +2398,70 @@ func file_llmgw_v1_agent_proto_rawDescGZIP() []byte {
|
||||
}
|
||||
|
||||
var file_llmgw_v1_agent_proto_enumTypes = make([]protoimpl.EnumInfo, 1)
|
||||
var file_llmgw_v1_agent_proto_msgTypes = make([]protoimpl.MessageInfo, 21)
|
||||
var file_llmgw_v1_agent_proto_msgTypes = make([]protoimpl.MessageInfo, 25)
|
||||
var file_llmgw_v1_agent_proto_goTypes = []any{
|
||||
(ShardStatus_State)(0), // 0: llmgw.v1.ShardStatus.State
|
||||
(*AgentMessage)(nil), // 1: llmgw.v1.AgentMessage
|
||||
(*ControlMessage)(nil), // 2: llmgw.v1.ControlMessage
|
||||
(*Register)(nil), // 3: llmgw.v1.Register
|
||||
(*Storage)(nil), // 4: llmgw.v1.Storage
|
||||
(*RunningShard)(nil), // 5: llmgw.v1.RunningShard
|
||||
(*GPU)(nil), // 6: llmgw.v1.GPU
|
||||
(*FabricAttachment)(nil), // 7: llmgw.v1.FabricAttachment
|
||||
(*LocalWeights)(nil), // 8: llmgw.v1.LocalWeights
|
||||
(*Heartbeat)(nil), // 9: llmgw.v1.Heartbeat
|
||||
(*ShardStatus)(nil), // 10: llmgw.v1.ShardStatus
|
||||
(*FetchProgress)(nil), // 11: llmgw.v1.FetchProgress
|
||||
(*StartShard)(nil), // 12: llmgw.v1.StartShard
|
||||
(*EngineProfile)(nil), // 13: llmgw.v1.EngineProfile
|
||||
(*RemoveWeights)(nil), // 14: llmgw.v1.RemoveWeights
|
||||
(*StopShard)(nil), // 15: llmgw.v1.StopShard
|
||||
(*DrainShard)(nil), // 16: llmgw.v1.DrainShard
|
||||
(*Ack)(nil), // 17: llmgw.v1.Ack
|
||||
nil, // 18: llmgw.v1.Register.LabelsEntry
|
||||
nil, // 19: llmgw.v1.Register.EngineVersionsEntry
|
||||
nil, // 20: llmgw.v1.Heartbeat.EngineVersionsEntry
|
||||
nil, // 21: llmgw.v1.EngineProfile.EnvEntry
|
||||
(*Host)(nil), // 4: llmgw.v1.Host
|
||||
(*Storage)(nil), // 5: llmgw.v1.Storage
|
||||
(*RunningShard)(nil), // 6: llmgw.v1.RunningShard
|
||||
(*GPU)(nil), // 7: llmgw.v1.GPU
|
||||
(*FabricAttachment)(nil), // 8: llmgw.v1.FabricAttachment
|
||||
(*LocalWeights)(nil), // 9: llmgw.v1.LocalWeights
|
||||
(*Heartbeat)(nil), // 10: llmgw.v1.Heartbeat
|
||||
(*Provision)(nil), // 11: llmgw.v1.Provision
|
||||
(*EngineWant)(nil), // 12: llmgw.v1.EngineWant
|
||||
(*ProvisionStatus)(nil), // 13: llmgw.v1.ProvisionStatus
|
||||
(*ShardStatus)(nil), // 14: llmgw.v1.ShardStatus
|
||||
(*FetchProgress)(nil), // 15: llmgw.v1.FetchProgress
|
||||
(*StartShard)(nil), // 16: llmgw.v1.StartShard
|
||||
(*EngineProfile)(nil), // 17: llmgw.v1.EngineProfile
|
||||
(*RemoveWeights)(nil), // 18: llmgw.v1.RemoveWeights
|
||||
(*StopShard)(nil), // 19: llmgw.v1.StopShard
|
||||
(*DrainShard)(nil), // 20: llmgw.v1.DrainShard
|
||||
(*Ack)(nil), // 21: llmgw.v1.Ack
|
||||
nil, // 22: llmgw.v1.Register.LabelsEntry
|
||||
nil, // 23: llmgw.v1.Register.EngineVersionsEntry
|
||||
nil, // 24: llmgw.v1.Heartbeat.EngineVersionsEntry
|
||||
nil, // 25: llmgw.v1.EngineProfile.EnvEntry
|
||||
}
|
||||
var file_llmgw_v1_agent_proto_depIdxs = []int32{
|
||||
3, // 0: llmgw.v1.AgentMessage.register:type_name -> llmgw.v1.Register
|
||||
9, // 1: llmgw.v1.AgentMessage.heartbeat:type_name -> llmgw.v1.Heartbeat
|
||||
10, // 2: llmgw.v1.AgentMessage.shard:type_name -> llmgw.v1.ShardStatus
|
||||
12, // 3: llmgw.v1.ControlMessage.start:type_name -> llmgw.v1.StartShard
|
||||
15, // 4: llmgw.v1.ControlMessage.stop:type_name -> llmgw.v1.StopShard
|
||||
16, // 5: llmgw.v1.ControlMessage.drain:type_name -> llmgw.v1.DrainShard
|
||||
17, // 6: llmgw.v1.ControlMessage.ack:type_name -> llmgw.v1.Ack
|
||||
14, // 7: llmgw.v1.ControlMessage.remove_weights:type_name -> llmgw.v1.RemoveWeights
|
||||
18, // 8: llmgw.v1.Register.labels:type_name -> llmgw.v1.Register.LabelsEntry
|
||||
6, // 9: llmgw.v1.Register.gpus:type_name -> llmgw.v1.GPU
|
||||
7, // 10: llmgw.v1.Register.fabrics:type_name -> llmgw.v1.FabricAttachment
|
||||
19, // 11: llmgw.v1.Register.engine_versions:type_name -> llmgw.v1.Register.EngineVersionsEntry
|
||||
5, // 12: llmgw.v1.Register.running_shards:type_name -> llmgw.v1.RunningShard
|
||||
8, // 13: llmgw.v1.Register.weights:type_name -> llmgw.v1.LocalWeights
|
||||
4, // 14: llmgw.v1.Register.storage:type_name -> llmgw.v1.Storage
|
||||
0, // 15: llmgw.v1.RunningShard.state:type_name -> llmgw.v1.ShardStatus.State
|
||||
4, // 16: llmgw.v1.Heartbeat.storage:type_name -> llmgw.v1.Storage
|
||||
8, // 17: llmgw.v1.Heartbeat.weights:type_name -> llmgw.v1.LocalWeights
|
||||
20, // 18: llmgw.v1.Heartbeat.engine_versions:type_name -> llmgw.v1.Heartbeat.EngineVersionsEntry
|
||||
0, // 19: llmgw.v1.ShardStatus.state:type_name -> llmgw.v1.ShardStatus.State
|
||||
11, // 20: llmgw.v1.ShardStatus.fetch:type_name -> llmgw.v1.FetchProgress
|
||||
13, // 21: llmgw.v1.StartShard.profile:type_name -> llmgw.v1.EngineProfile
|
||||
21, // 22: llmgw.v1.EngineProfile.env:type_name -> llmgw.v1.EngineProfile.EnvEntry
|
||||
1, // 23: llmgw.v1.AgentService.Connect:input_type -> llmgw.v1.AgentMessage
|
||||
2, // 24: llmgw.v1.AgentService.Connect:output_type -> llmgw.v1.ControlMessage
|
||||
24, // [24:25] is the sub-list for method output_type
|
||||
23, // [23:24] is the sub-list for method input_type
|
||||
23, // [23:23] is the sub-list for extension type_name
|
||||
23, // [23:23] is the sub-list for extension extendee
|
||||
0, // [0:23] is the sub-list for field type_name
|
||||
10, // 1: llmgw.v1.AgentMessage.heartbeat:type_name -> llmgw.v1.Heartbeat
|
||||
14, // 2: llmgw.v1.AgentMessage.shard:type_name -> llmgw.v1.ShardStatus
|
||||
16, // 3: llmgw.v1.ControlMessage.start:type_name -> llmgw.v1.StartShard
|
||||
19, // 4: llmgw.v1.ControlMessage.stop:type_name -> llmgw.v1.StopShard
|
||||
20, // 5: llmgw.v1.ControlMessage.drain:type_name -> llmgw.v1.DrainShard
|
||||
21, // 6: llmgw.v1.ControlMessage.ack:type_name -> llmgw.v1.Ack
|
||||
18, // 7: llmgw.v1.ControlMessage.remove_weights:type_name -> llmgw.v1.RemoveWeights
|
||||
11, // 8: llmgw.v1.ControlMessage.provision:type_name -> llmgw.v1.Provision
|
||||
22, // 9: llmgw.v1.Register.labels:type_name -> llmgw.v1.Register.LabelsEntry
|
||||
7, // 10: llmgw.v1.Register.gpus:type_name -> llmgw.v1.GPU
|
||||
8, // 11: llmgw.v1.Register.fabrics:type_name -> llmgw.v1.FabricAttachment
|
||||
23, // 12: llmgw.v1.Register.engine_versions:type_name -> llmgw.v1.Register.EngineVersionsEntry
|
||||
6, // 13: llmgw.v1.Register.running_shards:type_name -> llmgw.v1.RunningShard
|
||||
9, // 14: llmgw.v1.Register.weights:type_name -> llmgw.v1.LocalWeights
|
||||
5, // 15: llmgw.v1.Register.storage:type_name -> llmgw.v1.Storage
|
||||
4, // 16: llmgw.v1.Register.host:type_name -> llmgw.v1.Host
|
||||
0, // 17: llmgw.v1.RunningShard.state:type_name -> llmgw.v1.ShardStatus.State
|
||||
5, // 18: llmgw.v1.Heartbeat.storage:type_name -> llmgw.v1.Storage
|
||||
9, // 19: llmgw.v1.Heartbeat.weights:type_name -> llmgw.v1.LocalWeights
|
||||
24, // 20: llmgw.v1.Heartbeat.engine_versions:type_name -> llmgw.v1.Heartbeat.EngineVersionsEntry
|
||||
13, // 21: llmgw.v1.Heartbeat.provision:type_name -> llmgw.v1.ProvisionStatus
|
||||
12, // 22: llmgw.v1.Provision.engines:type_name -> llmgw.v1.EngineWant
|
||||
0, // 23: llmgw.v1.ShardStatus.state:type_name -> llmgw.v1.ShardStatus.State
|
||||
15, // 24: llmgw.v1.ShardStatus.fetch:type_name -> llmgw.v1.FetchProgress
|
||||
17, // 25: llmgw.v1.StartShard.profile:type_name -> llmgw.v1.EngineProfile
|
||||
25, // 26: llmgw.v1.EngineProfile.env:type_name -> llmgw.v1.EngineProfile.EnvEntry
|
||||
1, // 27: llmgw.v1.AgentService.Connect:input_type -> llmgw.v1.AgentMessage
|
||||
2, // 28: llmgw.v1.AgentService.Connect:output_type -> llmgw.v1.ControlMessage
|
||||
28, // [28:29] is the sub-list for method output_type
|
||||
27, // [27:28] is the sub-list for method input_type
|
||||
27, // [27:27] is the sub-list for extension type_name
|
||||
27, // [27:27] is the sub-list for extension extendee
|
||||
0, // [0:27] is the sub-list for field type_name
|
||||
}
|
||||
|
||||
func init() { file_llmgw_v1_agent_proto_init() }
|
||||
@@ -2109,6 +2480,7 @@ func file_llmgw_v1_agent_proto_init() {
|
||||
(*ControlMessage_Drain)(nil),
|
||||
(*ControlMessage_Ack)(nil),
|
||||
(*ControlMessage_RemoveWeights)(nil),
|
||||
(*ControlMessage_Provision)(nil),
|
||||
}
|
||||
type x struct{}
|
||||
out := protoimpl.TypeBuilder{
|
||||
@@ -2116,7 +2488,7 @@ func file_llmgw_v1_agent_proto_init() {
|
||||
GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
|
||||
RawDescriptor: unsafe.Slice(unsafe.StringData(file_llmgw_v1_agent_proto_rawDesc), len(file_llmgw_v1_agent_proto_rawDesc)),
|
||||
NumEnums: 1,
|
||||
NumMessages: 21,
|
||||
NumMessages: 25,
|
||||
NumExtensions: 0,
|
||||
NumServices: 1,
|
||||
},
|
||||
|
||||
@@ -34,6 +34,7 @@ message ControlMessage {
|
||||
DrainShard drain = 3;
|
||||
Ack ack = 4;
|
||||
RemoveWeights remove_weights = 5;
|
||||
Provision provision = 6;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -101,6 +102,34 @@ message Register {
|
||||
// Empty means the node was not asked to look, exactly as for weights, and
|
||||
// constrains nothing.
|
||||
repeated Storage storage = 12;
|
||||
|
||||
// host is what the machine is, so the control plane can have the mirror
|
||||
// fetch software built for it rather than assuming every node matches the
|
||||
// first.
|
||||
Host host = 13;
|
||||
|
||||
// engines_probed says engine_versions is an answer. Without it an empty map
|
||||
// meant "an agent too old to say" and constrained nothing — so a freshly
|
||||
// built node with no engine installed yet was a candidate for every model.
|
||||
bool engines_probed = 14;
|
||||
}
|
||||
|
||||
// Host describes the operating system and processor of a node.
|
||||
message Host {
|
||||
// From /etc/os-release: "ubuntu", "24.04", "noble".
|
||||
string os_id = 1;
|
||||
string os_version = 2;
|
||||
string os_codename = 3;
|
||||
// Debian's spelling, which is what package indexes are keyed by: "amd64",
|
||||
// "arm64".
|
||||
string arch = 4;
|
||||
// The vector extensions inference cares about, of those the CPU has. A VM
|
||||
// given QEMU's default CPU model has none of them, and runs, and is five
|
||||
// times slower, and nothing says why.
|
||||
repeated string cpu_flags = 5;
|
||||
// nvidia_driver is the driver version, empty when there is none. A card
|
||||
// without a driver is not a GPU anything can use.
|
||||
string nvidia_driver = 6;
|
||||
}
|
||||
|
||||
// Storage is one filesystem a node keeps weights on.
|
||||
@@ -234,6 +263,43 @@ message Heartbeat {
|
||||
// somebody restarts the agent — which is a step nobody knows to take.
|
||||
bool engines_changed = 8;
|
||||
map<string, string> engine_versions = 9;
|
||||
|
||||
// provision is where installing the node's software stands, sent when it
|
||||
// changed. The flag makes an empty list mean "nothing to install" rather
|
||||
// than "nothing to say", as for weights.
|
||||
bool provision_changed = 10;
|
||||
repeated ProvisionStatus provision = 11;
|
||||
}
|
||||
|
||||
// Provision is the software a node should have: which engines, at which
|
||||
// versions. Nothing more.
|
||||
//
|
||||
// Everything else is the node's to work out, because only the node knows it:
|
||||
// which build fits its hardware, where that build is published, what its
|
||||
// operating system is missing, and whether the result actually runs. The agent
|
||||
// converges on this — installs what is absent, checks what is there — and
|
||||
// reports each engine through ProvisionStatus. Nothing is taken on trust from
|
||||
// what happens to be on the machine: an engine is offered to placement only
|
||||
// once the agent has installed it and seen it run.
|
||||
message Provision {
|
||||
repeated EngineWant engines = 1;
|
||||
}
|
||||
|
||||
// EngineWant names one engine at one version.
|
||||
message EngineWant {
|
||||
string engine = 1; // "llamacpp"
|
||||
string version = 2; // "b10991"
|
||||
}
|
||||
|
||||
// ProvisionStatus is where one engine's installation stands on a node.
|
||||
message ProvisionStatus {
|
||||
string engine = 1;
|
||||
string version = 2;
|
||||
string variant = 3;
|
||||
// "installing", "ready", "failed", or "unsupported" when no build fits.
|
||||
string state = 4;
|
||||
// detail says what is happening or what went wrong, in words for a person.
|
||||
string detail = 5;
|
||||
}
|
||||
|
||||
// ShardStatus reports a shard's lifecycle transition.
|
||||
|
||||
Reference in new issue
Block a user