Commit Graph
587 Commits
Author SHA1 Message Date
Ilker ManapandClaude Opus 5.5 9c675e7293 Knowledge bases at scale: halfvec, a text index per collection, ef_search raised to the candidates wanted
A search asking for more than 10 passages got at most 40 candidates from
HNSW, pgvector's default ef_search. The full-text arm scored every chunk
holding a word of the question before limiting, over the whole
organization's GIN index; it now takes candidates by ts_rank from the
collection's own partial index and scores those. New collections index
halfvec on pgvector 0.7 and later (up to 4000 dimensions).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 22:08:29 +02:00
Ilker ManapandClaude Opus 5.5 42d3b7ea22 The store's PostgreSQL sized to its machine's memory
Each Patroni member gets a share of its own RAM in its own config.yml:
shared_buffers a quarter (15% beside a control plane or agent),
effective_cache_size, maintenance_work_mem and work_mem scaled to it, all
clamped. It is read again on every server init --patroni, so a machine
given more memory gets the use of it. A single-machine PostgreSQL init
installs is sized the same way, as a shared machine, instead of a fixed
96MB.

The step itself went in with 85025df by accident, in an early form that
restarted PostgreSQL before Patroni had taken the reload, leaving the old
shared_buffers with a restart pending. It now waits for Patroni to report
the restart pending, restarts with restart_pending (replicas first, the
leader last), and restarts a member it finds pending from an earlier run.

Applied to the PoC: pg01 978MB, ctl02 (control plane beside it) 587MB,
both from 128MB; streaming, lag 0, gateways unaffected.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 21:48:53 +02:00
Ilker ManapandClaude Opus 5.5 85025dfb03 Trained adapters published, private to their owners like the knowledge bases they come from
An adapter now has an owner (org, team, project, user or key): only callers
there see it, and to anyone else it is a model that does not exist, absent
from /v1/models too. A training job's owner defaults to the one collection
its pairs come from, must be able to read every collection they come from,
and pairs from a knowledge base are never made public. ctl train publish
keeps the GGUF or PEFT files on the mirror, which accepts uploads now, and
sets the owned adapter after the usual architecture check.

The knowledge bases' cache_salt follows the collections searched rather
than the organization, so two teams of one organization no longer share
cached passages.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 21:30:50 +02:00
Ilker ManapandClaude Opus 5.5 964b35ae65 Backups reach a replicated store: pg_dump is given the cluster's authority with verify-full
pgEnv passed sslmode=verify-full but dropped sslrootcert, so libpq looked
for ~/.postgresql/root.crt and both backup and restore failed against the
Patroni store. sslrootcert, sslcert and sslkey now go along as PGSSL*.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 21:04:46 +02:00
Ilker ManapandClaude Opus 5.5 88cbf11570 Training finds the engines where the installer puts them when agent.yaml names no directory
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 20:55:50 +02:00
Ilker ManapandClaude Opus 5.5 79b3e9f887 LoRA adapters trained on the nodes: llmgw ctl train
A job names the catalog model the adapter is for, the safetensors weights to
train on, the node and the MiB of a card it needs, and question/answer pairs.
The control plane sends it when the node's agent is there and nothing else
trains on it; the agent trains with its installed vLLM's torch and a pinned
overlay of peft and accelerate (no second environment), as the engines'
user, on the card with that much free — beside an engine, if it leaves that
much — converts the adapter to GGUF with the installed llama.cpp's own
conversion code when the model is served by llama.cpp, and sends what it
made back in pieces, kept in the store to download. Progress, loss and peak
card memory are reported as it goes; a node that goes fails its job.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 20:25:57 +02:00
Ilker ManapandClaude Opus 5.5 ae95ac8261 Training data from a collection: llmgw ctl kb dataset
A chat model of the cluster reads each chunk and writes questions a person
might ask of it with the answers it holds, streamed as NDJSON a chunk at a
time, for people to correct before an adapter is trained on it. A chunk that
gave no pairs says so on its own line.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 20:04:37 +02:00
Ilker ManapandClaude Opus 5.5 0a6a586dfe Backlog F4: a LoRA trained in the GPU's spare 2 GB on the PoC, and what it learned
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 19:31:53 +02:00
Ilker ManapandClaude Opus 5.5 c52b73feec Reference: the agents section of the cluster's settings
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 18:07:00 +02:00
Ilker ManapandClaude Opus 5.5 dc243f8840 The nodes' agent settings kept with the cluster's, agent.yaml down to what an agent needs to start
agents.default and agents.nodes.<id> in the cluster's settings hold
everything agent.yaml can say but the agent's id, control planes, address and
certificate, which stay in the file and are refused there. The control plane
sends each node its settings on register and, a second apart, when they
change; an agent whose settings changed checks they load beside its file,
keeps them and starts again on them, engines serving throughout. An edited
agent.yaml that loads restarts the agent too. History, audit, ctl config, the
console and the portal's settings page cover them as the rest of the
settings.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 18:06:36 +02:00
Ilker ManapandClaude Opus 5.5 659f20dc9a The cluster's settings edited on the portal, by administrators
/admin/settings edits the document llmgw ctl config edit opens: checked as a
control plane would load it, refused when made from a version since replaced
(the edit kept), its history viewed and rolled back as a new version, every
change audited by size and never by contents, since it can hold webhook
secrets. Administrators only; in English and Turkish.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 17:23:13 +02:00
Ilker ManapandClaude Opus 5.5 6bc6a67fb0 README: the releases since v0.11.0 as released, and knowledge bases
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 17:17:40 +02:00
Ilker ManapandClaude Opus 5.5 54570066e3 v0.14.0
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 16:22:07 +02:00
Ilker ManapandClaude Opus 5.5 99829b3124 Backlog: F3's gaps closed and checked on the PoC, cache_salt against a real vLLM
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 16:16:00 +02:00
Ilker ManapandClaude Opus 5.5 26630274f4 Passages of one organization never share a vLLM prefix cache with another's
A request given passages carries a cache_salt derived from its organization
when the replica chosen is vLLM, so how fast an answer starts cannot say what
another tenant's documents held. Other engines are not sent the field.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 16:02:19 +02:00
Ilker ManapandClaude Opus 5.5 5ee8e74de1 The portal's chat answers from the documents a person picks
A Documents picker on /chat offers the collections the person may read in
the chosen project, the default when there is one, or none; the sources an
answer was given are listed under it. The page's choice becomes the
X-Llmgw-Knowledge header; one the page sent itself is dropped. In English
and Turkish.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 15:59:23 +02:00
Ilker ManapandClaude Opus 5.5 0093cc3d45 Collections a part of the tree answers from when a request names none
On an organization, team, project or key, the narrowest winning: llmgw ctl
kb default, POST /api/v1/kb/defaults, and the console's Knowledge screen. A
request's own X-Llmgw-Knowledge overrides it and none turns it off. A default
never refuses a request: a collection the caller cannot read is left out, and
a failed search answers without passages. Removing a collection takes it out
of every default.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 15:52:43 +02:00
Ilker ManapandClaude Opus 5.5 54ba04c52a llmgw-kb collections names each collection's owner as people know it
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 15:42:40 +02:00
Ilker ManapandClaude Opus 5.5 496625824b Chat requests answered from knowledge bases
A request naming collections in X-Llmgw-Knowledge has its last question
embedded on the caller's account, the best passages found and added to the
system message, numbered, with an instruction to cite them and to say when
they do not hold the answer; x-llmgw-sources says which were given. Done
before the guardrails judge the request, never served from the answer cache,
and a collection the key may not read is answered as one that does not
exist. Metrics llmgw_kb_retrievals_total and llmgw_kb_retrieval_seconds.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 15:27:50 +02:00
Ilker ManapandClaude Opus 5.5 65116064d2 Backlog: F3 uses the search as it now scores
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 15:12:26 +02:00
Ilker ManapandClaude Opus 5.5 3a3535b310 Backlog: knowledge bases tried on the PoC, and what it showed
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 15:12:17 +02:00
Ilker ManapandClaude Opus 5.5 2c7b682587 Knowledge search counts a text match by how many of the question's words it holds
The best text match took the whole text weight, however little of the
question it matched: one everyday word ("most") put the remote-work page
above the travel one for "the most I can spend on a hotel". Now a chunk
gains the weight in proportion to the share of the question's words it
holds, counted once whatever their case.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 15:09:24 +02:00
Ilker ManapandClaude Opus 5.5 bc5ee9f3aa Knowledge search ranks by how near a chunk's meaning is, not by its place in two lists
Fused by rank, a full-text match on a question's everyday words outvoted a
far nearer meaning: on the PoC, "how much leave do fathers get" found the
annual-leave page before the paternity one (cosine distance 0.44 against
0.25). Now a chunk scores its similarity to the question plus up to 0.3 for
a full-text match, each search brings four times k candidates, the text
search drops Turkish and English stop words, and an operator's search says
when the embedding model did not answer rather than quietly searching text.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 15:05:18 +02:00
Ilker ManapandClaude Opus 5.5 1ec9b83076 Backlog: PostgreSQL tests of several packages run one package at a time
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 13:59:17 +02:00
Ilker ManapandClaude Opus 5.5 eec0d92d18 The console's Knowledge screen, and the manual's page on knowledge bases
Collections listed with owner, model, documents and grants; A makes one; on a
row its documents, grant, revoke, a search to try, delete. docs/knowledge.md
says what administrators do and how people use llmgw-kb on Windows, macOS
and Linux: what it reads, what leaves their machine, where its ledger is.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 13:58:12 +02:00
Ilker ManapandClaude Opus 5.5 6466f37d50 Knowledge bases administered: llmgw ctl kb and /api/v1/kb
Collections made for an organization, team, project or person, their
dimension asked of the embedding model; granted and revoked by path, never
outside their organization; their documents listed and removed; and a search
an operator can try. In the OpenAPI document.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 13:49:44 +02:00
Ilker ManapandClaude Opus 5.5 ffab3c99e3 llmgw-kb built for Windows, macOS and Linux in CI and attached to releases
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 13:44:53 +02:00
Ilker ManapandClaude Opus 5.5 ba5df90602 llmgw-kb: sync a person's documents into a knowledge base from their own machine
Its own binary for Windows, macOS and Linux, carrying no server code. It reads
the files there, cuts them, has the gateway embed them with the person's key
and sends text and vectors for what changed only: a SQLite ledger on the
machine remembers each document's file, size, time, digest and what happened,
so a rerun reads nothing unchanged, a file skipped stays skipped until
--retry, and --delete removes only what this machine sent under the paths
given. --dry-run says what would leave the machine. Commands: login,
collections, sync, status, history, formats.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 13:43:47 +02:00
Ilker ManapandClaude Opus 5.5 d9e030e0b6 Knowledge bases on the client listener: list, manifest, put and delete a document
What llmgw-kb, run by a person with their own key, reads and writes. A
collection the key may not read answers as one that does not exist; writing
needs ownership or a write grant on the key's chain; a personal key names its
project in X-Llmgw-Project and must be a member of it. Changes are audited by
document name.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 13:27:14 +02:00
Ilker ManapandClaude Opus 5.5 a0cfafe772 Text from documents of many formats, and chunks of it, for knowledge bases
PDF, Word, Excel, PowerPoint, OpenDocument, RTF, EPUB, HTML, MHTML, mail with
its attachments, notebooks, subtitles, zip archives and text and source files
in any common encoding, in pure Go; legacy Office formats through LibreOffice
and scanned pages through Tesseract when they are installed. Chunks of about
500 tokens under their headings, overlapping by a paragraph.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 13:24:15 +02:00
Ilker ManapandClaude Opus 5.5 d35630defe Knowledge bases in the store: collections, grants and chunks kept apart per tenant
Collections owned by an organization, team, project or person; grants that
add up down the tree and never cross an organization; chunks in PostgreSQL
with pgvector, partitioned by organization, an HNSW index per collection, and
row-level security as a second line behind each query's filter. Search fuses
nearest-neighbour and full-text ranks. Off on SQLite and without pgvector,
saying why. server init and the HA database setup install and enable it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 12:47:57 +02:00
Ilker ManapandClaude Opus 5.5 0a7fb821d8 v0.13.0
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 11:55:34 +02:00
Ilker ManapandClaude Opus 5.5 7f7e5a470e guard judge says controversial, and names a judge no policy asks as one
judge ask showed Qwen3Guard's controversial verdict as "safe (Violent)"; it
now says controversial, which a policy's judge.controversial counts. The
stray-judge note called a judge no policy asks any more "served as a model
clients use"; a model with role: guard is now said to be a judge no policy
asks.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 11:42:55 +02:00
Ilker ManapandClaude Opus 5.5 1fabb05fb2 A replica restarted in place keeps the room it frees for its replacement
The replica was stopped and the pass went on to the other models before
placing its replacement, so a model waiting for room took the freed cards.
On the PoC a GPU judge waiting for the only card took it from
qwen3.8-35b-a3b-distill, restarted there after its definition changed,
until the judge was stopped by hand. With the waiting model first in the
catalog it happened one pass later. The replacement is now placed in the
same step that stops the old replica.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 11:42:55 +02:00
Ilker ManapandClaude Opus 5.5 457a1eefac Backlog F: company documents through retrieval, isolated per tenant, then adapters
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 11:35:38 +02:00
Ilker ManapandClaude Opus 5.5 0bf7e46fbf llmgw ctl tool-check asks with the engine's key
Every replica's engine demands its own key, and the check sent none, so it
failed with a 401 on every engine in the cluster. The key is asked of the
admin API for the replica at --endpoint, as bench does, or given with --key.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 11:24:37 +02:00
Ilker ManapandClaude Opus 5.5 b0273ffb5f Qwen3Guard is the judge, and judges on a GPU and on CPUs are asked in turn
Llama Guard 3 1B could not tell ordinary coding requests from harmful ones:
on the PoC it scored "continue" 0.51, "continue the code above" 0.89 and
"how do I kill all child processes in bash" 0.92 unsafe, against 0.97 to
1.00 for harmful requests, at Q4_K_M and Q8_0 alike, with its own template.
A site had "continue" refused. Qwen3Guard-Gen 0.6B gave the ordinary ones
0.15 at the most and the harmful ones 1.00, in 0.8 GB on CPUs.

- format qwen3guard: its prompt written out as its template writes it, for
  requests and answers, and its Safe / Unsafe / Controversial verdict read;
  controversial counts as the policy's judge.controversial says, safe by
  default.
- judge.models: judges asked in turn, the first that answers deciding; a
  failure is recorded against the judge that failed. guard judge use sets
  the order, replacing categories none of the judges names.
- known judges (Qwen3Guard 0.6B/4B/8B, Llama Guard 1B/8B): guard judge
  enable serves any of them, --gpu the 4B on the roomiest GPU class as a
  standard model; the leader fetches and serves one a policy names that the
  catalog lacks.
- server init installs qwen3guard-0.6b with the baseline's categories.
- a judge reads a long text by its start and end, which fit its context,
  rather than overflowing it and refusing the request under fail: closed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 10:22:47 +02:00
Ilker ManapandClaude Opus 5.5 63f7c8cfd7 llmgw start --wait waits for the models, not for the setting
Every model of a stopped system asks for no replica and so has what it asks
for. Until the control plane took the setting up, seconds later, start
--wait saw 18 of 18 satisfied and returned at once, with nothing started.
It now counts only once the control plane says it is in the state asked
for, both ways.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 21:43:45 +02:00
Ilker ManapandClaude Opus 5.5 2d14918b99 An adopted engine that turns ready after registration is reported
The registration carries each shard's state as it is then. An adopted
engine is health-checked after that, and nothing reported the ready that
followed, so when the agent registered first the control plane kept the
replica starting while it served. After llmgw restart on the PoC two
replicas stayed so, one the only replica of qwen2.5-coder-14b. The
session's watcher now reports every shard the registration called not
ready once it is.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 19:43:13 +02:00
Ilker ManapandClaude Opus 5.5 3f8cc06f72 llmgw restart goes on past a control plane it cannot reach
One standby that ssh could not log in to stopped the whole restart before
this control plane's own. It is now said and left, the rest restarts - the
leader last - and the command fails at the end naming what was not
restarted and how to do it there.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 18:55:52 +02:00
Ilker ManapandClaude Opus 5.5 7245dbe1dd llmgw stop, start and restart act on the whole system
Issue #3 asked for commands to stop and start the system, and 05aa21c gave
only one machine's services. Now:

- llmgw stop sets serving.stopped in the cluster's settings, live: the
  reconciler wants every model at no replicas, freeing every GPU, and the
  gateway answers every POST under /v1/ with 503 system_stopped. The control
  planes, agents and mirror keep running, and no model's definition changes.
- llmgw start clears it; every model is placed again as it asks.
- llmgw restart has every agent restart over its stream (RestartAgent), then
  the other control planes over ssh, standbys before a leader, then this
  one; engines keep serving.
- llmgw status is the cluster's summary, saying when it is stopped.

One machine's services moved to llmgw service status|start|stop|restart.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 18:01:45 +02:00
Ilker ManapandClaude Opus 5.5 05aa21c4d4 llmgw status, start, stop and restart for this machine's services
There was no command to stop or start llmgw on a machine (issue #3). These
act on the llmgw units the machine has - mirror, server, agent - or those
named, in dependency order, saying what each did. stop leaves a node's
engines serving unless --engines.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 14:52:52 +02:00
Ilker ManapandClaude Opus 5.5 2f595f5999 An adopted engine's state is written without its command; engine-check finds the installed build
SetCompute on an engine adopted from an earlier agent called writeState,
which read the command line from sh.cmd - nil for a process this agent did
not start - and the agent crashed at every start on the PoC's nodes until
fixed. An adopted shard now keeps the command line its state file had.

engine-check looked for llama-server one directory too shallow; the agent
installs it under <version-variant>/llama-<build>/. It now looks at both,
preferring a CUDA build on a machine with NVIDIA GPUs. Run on cuda01: a
token computed on CUDA0, 25 of 25 layers offloaded.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 14:26:28 +02:00
Ilker ManapandClaude Opus 5.5 b5d650af16 What a replica computes on comes from nvidia-smi; llama.cpp says its layers only at -lv 4
Checked against the installed llama.cpp (b10991): at its default verbosity
it prints neither the devices it uses nor the layers it offloads, so the
start-up output said nothing and every replica's ON column stayed empty.
The agent now takes compute from nvidia-smi - the GPUs the shard's processes
hold memory on, and how much - or CPU, for every engine, falling back to it
when llama.cpp's output says nothing; adopted engines get it too, and each
session reports it once per shard as a compute-only status. engine-check
runs its GPU token at -lv 4, where llama.cpp does say where its layers went.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 14:04:08 +02:00
Ilker ManapandClaude Opus 5.5 25b3e129fe What a shard computes on survives the agent's and the control plane's restarts
It was sent once, with READY, and kept only in the leader's memory: a
failover or an agent restart lost it. The agent now keeps it with the
shard's state on disk (ShardSpec.Compute, set by Supervisor.SetCompute) and
reports it with each running shard when it registers (RunningShard.compute),
which adoption carries onto the replica.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 13:51:40 +02:00
Ilker ManapandClaude Opus 5.5 0add189e02 A standby that takes over waits for the fleet before placing
The settling window ran from process start, so a standby that took over
after a minute placed at once, before the agents reconnected and reported
what they run: it started second copies of replicas it did not know of and
drained the healthy ones when they were adopted. Seen on the PoC: a leader
restart replaced llama-guard-3-1b and a qwen2.5-1.5b replica. Reconcile now
reopens the window on the first pass after standing by.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 13:40:28 +02:00
Ilker ManapandClaude Opus 5.5 66f702d5c1 A forgotten password is set through a one-time link
/forgot mails a link to set a new password when the cluster's mail settings
say how - SMTP relay, sender, and the public base_url the link points at,
never the request's Host - and says the same whether or not the address has
an account. /reset sets it once, closing every session. Links are kept by
their hash in password_resets, good for an hour, a newer one replacing an
older. Without mail, /forgot says to ask an operator, and llmgw users
reset-link makes the link to hand over. Single sign-on accounts get none.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 13:20:11 +02:00
Ilker ManapandClaude Opus 5.5 f2b2e24e1f A node's own readiness and its multi-node readiness are kept apart
doctor sums findings up as two readinesses - the node's, which decides
whether it joins, and multi-node, its fabrics for replicas spanning nodes -
prints fabric findings in a section of their own, and gives both in --json.
node add says the multi-node one. NodeView.multi_node says whether a node's
declared fabrics are up, naming adapters that are down, and ctl nodes shows
it beside the fabric.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 13:11:16 +02:00
Ilker ManapandClaude Opus 5.5 5ae870e1b2 ctl restart --max-surge and --max-unavailable
A model's replicas restarted one at a time, always. RestartPace says how
many restart at once: MaxSurge beside themselves (replacement first),
MaxUnavailable stopped first (no room needed, out while loading); the
reconciler begins that many per model per pass. The admin API takes
max_surge and max_unavailable on POST /api/v1/restart.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 13:09:10 +02:00
Ilker ManapandClaude Opus 5.5 619dd751ab Nodes show their engines' versions and GPUs' capability; a profile can ask for an engine version
NodeView carries engine_versions and compute_capabilities; ctl nodes shows
llamacpp b10991 and 6000-24g (7.5). A profile's engine_version restricts it
to nodes whose engine version starts with it, and explain-placement says
which version a node has, or which engines it has when it lacks the one
asked for.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 13:06:57 +02:00