Commit Graph
194 Commits
Author SHA1 Message Date
Ilker ManapandClaude Opus 5.5 ac25a9136a Sign-in is set up from the console, and a directory tried before it is kept
People → Sign-in shows single sign-on, Active Directory and SCIM, and
the directory's last sync. E sets up Entra ID from the tenant and client
IDs; D sets up Active Directory, tried from the control plane first:
each server bound, people who may sign in, members of each mapped group.
The secrets go into the store as store:auth/..., sealed by the master
key, and the CA into the settings, so no control plane needs a file.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-19 12:55:27 +02:00
Ilker ManapandClaude Opus 5.5 178b168bfd A broken directory setting, or a second server, no longer takes the cluster down
Found trying auth.ldap against a Samba DC on the PoC. A CA file a
control plane could not use made every control plane exit, and no
ctl config could then put the setting right: directory sign-in is now
turned off there, and said. `llmgw server check` ran a whole server
that, its ports taken, held the leader's lease for three hours: server
refuses words it does not know and takes its ports before anything else.
The guides now say auth is set with ctl config, not server.yaml.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-19 11:10:43 +02:00
Ilker ManapandClaude Opus 5.5 f27d722111 People sign in with their Active Directory account
auth.ldap checks a username and password against the domain controllers
over LDAPS or StartTLS: found with a service account, checked by binding
as the person. Groups, nested ones included, give staff roles and places
in the tree. Accounts are tied to the objectGUID; the leader reads the
directory every 15 minutes, suspends whom it disabled or removed and
reinstates them after, and refuses a sync that would suspend most.

Entra ID sends no groups for someone in more than 200, only where to
fetch them; that was read as none and took an administrator's role away.
It is now left as it was, and logged. An account made by SCIM or OIDC
can be reinstated again.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-18 10:59:50 +02:00
Ilker ManapandClaude Opus 5.5 fe310576e7 Several keys for one outside provider
An external model takes api_key_files, each a file and an optional weight,
in place of api_key_file. Each key is an endpoint of its own
(external:MODEL#1, #2), so the router's scoring, retries and breakers work
per key. A key answering 429 is passed over while its breaker is open. A
key answering 401 is passed over too, and external_key_refused is sent,
naming the endpoint and never the key; with a single key the 401 is still
the client's. A key whose file cannot be read is left out, and usage
records which key answered by its endpoint.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-17 08:56:30 +02:00
Ilker ManapandClaude Opus 5.5 0b984a92bc A judge is asked again only on another machine
Measured under CPU load on the three-VM cluster, with both judge replicas on
ha1: 37 of 60 requests failed with the hedge, 38 of 60 without it. Two
replicas on one saturated machine are slow together, and the second
question doubled the work there. The hedge now goes only to a replica whose
address names another host.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-16 10:27:50 +02:00
Ilker ManapandClaude Opus 5.5 bc4c5d5ac9 A judge replica that hangs is overtaken by another at half the budget
On the three-VM cluster about one answer in twenty-five came back as
content_filter: both judge replicas timed out within the same millisecond.
The tries shared the policy's 5 s budget, so the first replica, busy on an
oversubscribed machine, used it all and the retry on the second had nothing
left. A judge request now goes to a second replica too when the first has
not answered by half the budget, and the first answer is taken. The first
is not cut short.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-16 08:35:28 +02:00
Ilker ManapandClaude Opus 5.5 edeeb9f8e1 MCP takes pictures of real size, by id too, and waits for a cold model
The game's bridge found three things on the PoC. /mcp held its body to
64 KiB — the edit meant for it had raised the memory notes' limit instead —
so a 1024 px concept sent as base64 failed as "expected one JSON-RPC
request object". The notes are back to 64 KiB, /mcp takes 64 MB, and a
call past it is a 413 saying so.

/v1/files took batch files only, so a picture could not be sent once and
named after. purpose vision and user_data are kept too, as long as job
results, and generate_mesh hands a workflow model the id as it is.

generate_image and review_images failed at once while their model was
cold. They wait out an answer that says when to come back — loading,
another model holding the GPU, a full queue — for as long as the call may,
saying who holds the GPU and about how long; a refusal that says no
Retry-After, a spent budget, is said at once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 07:24:59 +02:00
Ilker Manap a7cb3aac3c Merge remote-tracking branch 'origin/mcp-media' into shared-replica
# Conflicts:
#	CHANGELOG.md
2026-10-14 14:01:24 +02:00
Ilker ManapandClaude Opus 5.5 aa531b1053 ComfyUI models of one environment share one replica
flux and sf3d on one card handed it to each other, each starting ComfyUI
and loading its weights from nothing (BACKLOG H3, placement side). An
environment of two or more models now gets a hidden catalog entry,
comfyui-env-<name>: on its first member's profiles with the most memory
any member needs, on demand for the longest idle timeout, carrying every
member's weights, repositories and the environment's custom nodes. The
members get no replicas of their own; asking for one wants the entry.

The agent fetches each member's weights; the gate links all of them,
refusing two weights files of one name, links each member's Hugging Face
repository, and answers /v1/models for every member. The control plane
offers the replica under the members' names with each one's workflow and
checkpoint, which the router sends by the model asked for. A member's
sibling never evicts the replica serving it, and the members' load counts
as the entry's.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 13:55:34 +02:00
Ilker ManapandClaude Opus 5.5 55d5e13534 Pictures, 3D models, a vision model and workflows as tools at /mcp
An agent driving Blender made its concepts, meshes and render reviews
through a tool server of the game's own, which called this gateway; the
gateway's MCP server offered only the memory notes. It now offers
generate_image, generate_mesh, review_images, run_workflow, job_status and
gateway_status beside them, each only where a model for it is here.

Each tool makes its request of the client API as the caller, through the
handler Handler built, key check and all: allowlists, limits, licences
left out, budgets and accounting apply as to any request, and a job is
the caller's. What a tool makes is kept as the caller's file and answered
by id and download URL, since a server over HTTP cannot write to the
caller's disk; a picture comes with a 512 px preview, and whole where no
file can be kept. A picture to work from is a file id, a data URL or
base64; one sent to the vision model is shrunk to 1280 px.

A mesh takes minutes. A client that takes a stream is answered as
server-sent events, with notifications/progress for its token while the
job is queued, waiting for the GPU and running, and waits up to ten
minutes; one that does not waits 50 seconds and gets the job id, which
job_status asks after.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 13:11:09 +02:00
Ilker ManapandClaude Opus 5.5 7c53380e68 Extended thinking on /v1/messages: a model's reasoning as thinking blocks
Claude Code and the Anthropic SDKs asking for extended thinking got the
answer alone: a reasoning model's reasoning_content was dropped. Asked for,
it comes back as a thinking block before the text — streamed as
thinking_delta events and closed with a signature before the text begins —
and the model is told to think through its chat template
(enable_thinking); disabled tells it not to. Not asked for, it is left out
as before.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 11:05:02 +02:00
Ilker ManapandClaude Opus 5.5 0460f52dd2 Jobs run side by side, one per model at a time
One job at a time, a 3D job waiting for the card held up another model's
job that could have run (seen on the PoC, BACKLOG H7). The leader's worker
claims jobs while fewer than eight run, passing over the models that have
one running, and starts each in its own goroutine; one ending wakes it for
the next. Lease and leader-only rules are as they were.

BACKLOG: H5–H7 marked, H8 (presets) and H9 (what the PoC taught) written.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 10:06:20 +02:00
Ilker ManapandClaude Opus 5.5 2eb2a6d8fe A mesh carries its whole licence, not only its name
GLBs carried the licence's name, URL and notice in asset.extras.license.
A pipeline that sorts assets by licence needs what a machine matches on:
the id, whether commercial use is allowed, the regions it excludes, the
revenue cap and the attribution. The whole licence is written, as
/v1/models gives it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 01:12:47 +02:00
Ilker ManapandClaude Opus 5.5 848db08469 The cluster page and the console say which picture and 3D models are here, and add them
A site whose catalog had no flux-schnell or triposr showed no Images or 3D
page, and nothing said they were one click away. The cluster page now has
a Pictures and 3D models card under the models: each preset, what it
makes, the classes of card here it fits, whether and by whom it was
added, and why the leader has not added it by itself; an Add button, which
says first when it would install ComfyUI on the GPU nodes and restart the
control planes; and the switch for presets.auto. The console's models
screen shows the same under its table: A adds one, asking first in the
same case, and P turns automatic adding on or off.

`ctl model add PRESET` and `ctl model presets` go through the presets API,
so the control plane keeps who added one, and `ctl model presets auto
on|off` flips the setting; against a control plane from before, add sends
the entry as v0.14.4 did.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 20:48:40 +02:00
Ilker ManapandClaude Opus 5.5 3858114ada A picture in a prompt counts as what a model makes of it, not its base64
A 1280x720 PNG sent to qwen3-vl-8b was estimated at 138 thousand prompt
tokens and refused against its 8k context before reaching the engine, which
makes about a thousand of it. Parts are counted by kind: text by its text,
a picture or a sound as a thousand tokens.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-11 20:45:06 +02:00
Ilker ManapandClaude Opus 5.5 587d8d5059 A licence left out by its name stays out once its entry gives an id
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-11 16:28:40 +02:00
Ilker ManapandClaude Opus 5.5 4bbac0833d A person leaves licences out, and no key of theirs reaches a model under one
A game sold in the EU cannot ship what a licence excluding the EU made. On
the profile page a person leaves out licences one by one, or by rule:
every licence forbidding commercial use, every licence granting nothing in
a region. The rules cover models added later. The choice reaches every
principal through the resolved limits, so permits() leaves such models out
of /v1/models, requests, jobs, fallbacks and the portal's pickers, and a
request for one is refused 403 licence_excluded saying which choice did.

Kept in user_excluded_licences (migration 0046), copied with the store and
gone with the person.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-11 16:10:22 +02:00
Ilker ManapandClaude Opus 5.5 b6bfc33ef7 Any ComfyUI workflow runs as a job, from the inputs its catalog entry declares
A model's catalog entry now says what its workflow takes and makes:
comfyui.inputs, each a placeholder of its name with a type (text, int,
float, bool, choice, image, mask, audio, video, file), bounds, choices
and a default; and comfyui.output, which gains video, audio and files
beside pictures and meshes. POST /v1/workflows/runs with {model, inputs}
checks a request against them before a job is written — a required input
left out, an unknown one, a number out of bounds, a song sent as a
picture, another owner's file id are each refused with what was wrong —
and the job worker replays it to a replica, where the files go into
ComfyUI's input directory, the workflow is filled and run, and the first
file of the model's output kind is the job's result. /v1/models lists
each ComfyUI model's workflow, so a client can build its form.

A PNG result carries the licence in an iTXt chunk, as a GLB does in
asset.extras. A file whose server named no type is typed by its name
before its bytes, so a WAV is audio/wav, not audio/wave.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-11 14:28:15 +02:00
Ilker ManapandClaude Opus 5.5 95c61daf1b Jobs are served by the leader, and a moment of fitting nowhere is waited out
A 3D job leased by the standby control plane, whose router has no endpoints
and which a job replayed in the process never leaves, was told "loading"
for seven minutes while the leader served the jobs behind it. Jobs and
batches are claimed only while leading.

Five jobs asked for while cuda01's agent restarted failed for good as
"cannot be placed": re-registering, the node had no engines for a few
seconds. A job asks again for five minutes before that is its end.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 23:53:33 +02:00
Ilker ManapandClaude Opus 5.5 b1490dbf30 A job waiting for the card says what holds it and about how long
A 3D job waiting for the picture model to give up the one card had an empty
progress; its client could not tell waiting from stuck. The control plane
says which model holds the cards and when it lets go, plus the waiting
model's last start; the 503 carries it as headers, and a job keeps it as
progress {state: waiting_gpu, held_by, eta_seconds}.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 22:25:27 +02:00
Ilker ManapandClaude Opus 5.5 b743b11c54 A picture is made ready for a 3D model; a model asleep is listed asleep
LLMGWPrepareForMesh, llmgw's own ComfyUI node, written by the gate into
every replica's custom_nodes from the binary: TripoSR's own preparation
(tsr/utils.py) — the background taken away, by the picture's alpha where
it has one and by rembg (MIT; u2net, Apache 2.0) where it does not, the
object cropped to its bounds, set square at a ratio of the side (0.85),
on grey. A picture sent as it was had a white or black shell round the
object, whatever was behind it made into the mesh.

/v1/models listed only models with a replica up, so one that loads when
asked — triposr, flux-schnell — was invisible to the clients that would
have asked for it. Those are listed now, with loaded: false; every entry
says loaded.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 20:43:20 +02:00
Ilker ManapandClaude Opus 5.5 fd3c7bda32 The client API described in OpenAPI, and both APIs in Swagger UI
/v1/openapi.json is the API clients use — OpenAI's endpoints, Anthropic's,
and what llmgw adds — in OpenAPI 3.1, generated from internal/clientapi's
route table by the generator the admin API's document comes from, now
told each document's title, server, token and error. A test reads the
router's mux.HandleFunc calls and refuses a route described and not
mounted, or mounted and not described. Asked for before there is a key,
so it needs none.

/docs/api shows it in Swagger UI 5.33.1 (Apache 2.0, served from the
gateway as the page's policy requires): Authorize takes a key, Try it out
sends to this gateway. Staff are offered the admin API's document too,
from /docs/api/admin.json, which nobody else is given.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 20:22:48 +02:00
Ilker ManapandClaude Opus 5.5 7f320c5f65 v0.14.3: changelog, README, docs for 3D models; a finished job says done
A finished job kept the last word heard from ComfyUI, the node running;
it is now said done before the job is marked completed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 19:22:58 +02:00
Ilker ManapandClaude Opus 5.5 2f54780802 G4: a page that makes 3D models, and ctl 3d
/3d beside /images, while some model makes meshes: a picture chosen, the
job asked for through the same path an API client's takes, its progress
shown (waiting for the card, n ahead; making it, the node and percent),
and the mesh in a viewer — model-viewer 4.4.0 (Apache 2.0, with three.js
and lit), served from the gateway since the page's policy loads scripts
from nowhere else — with a download and the model's licence. The mesh is
served to its owner from /3d/jobs/{id}/file; another project's person is
told there is no such job.

A job made from the portal acted for a key that does not exist: the
worker now rebuilds a session's principal from the account that made it,
while it is active and still in the project the job is charged to.

ctl 3d list|get|cancel, over /api/v1/client-jobs (/api/v1/jobs is the
control plane's own work, which ctl jobs shows).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 19:16:20 +02:00
Ilker ManapandClaude Opus 5.5 4b68328e3e A mesh's resolution is the model's own: a multiple of 32 from 64 to 2048
TripoSR takes 256 by default; 512, 1024 and 1536 were TRELLIS.2's alone.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 18:47:52 +02:00
Ilker ManapandClaude Opus 5.5 88781cfcfe A job says how far along it is; a ComfyUI model brings its custom nodes
G2: the gate keeps a websocket open to its ComfyUI under a client id of its
own, puts that id on every workflow it lets through — ComfyUI tells only
the client that queued a workflow how it goes — and keeps the last word on
each: the node running, its step of how many. It answers at
/progress/{prompt_id}. While a job's request waits, the router reads the
queue (queued, n ahead) and the gate (running, node, step, percent) and
writes it into the job, which a client reads back. Only into a job of the
same key: a tag naming another's job writes nothing.

G3, the rest of the plumbing: comfyui.nodes names custom nodes by https
repository and full commit, with the PyPI packages they need. The gate puts
each in the replica's own custom_nodes at that commit and installs the
packages beside the replica, constrained to the versions ComfyUI's
environment already has, which it never changes. The catalog refuses a
branch for a commit and anything but package specs for requirements. An
OBJ a node saves leaves the gateway as a GLB, colours kept, so every mesh
can carry its licence.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 18:46:44 +02:00
Ilker ManapandClaude Opus 5.5 c87e413360 A ComfyUI model may make meshes: from a picture, with its licence inside
G3, the part that needs no model: a ComfyUI model's comfyui.output (or
external.output) says pictures or meshes. A request for a 3D model — a
job at /v1/3d/generations — carries the picture as base64; at the replica
it is uploaded to ComfyUI's input directory (the gate lets /upload/image
through), the workflow is filled in with its name, the seed and the
resolution, and the mesh the workflow saves comes back with its type,
which is the file the job keeps. comfy.Client reads every list of files an
output names, not "images" only, and types a file by its extension where
ComfyUI sends octet-stream.

A model's license (name, url, notice) is said in /v1/models and in the
job, and written into a GLB's asset.extras, since the file travels without
the API; a mesh it cannot be written into is refused. per_mesh in
auth.pricing, usage.meshes in the rows, the sums and the export
(migration 0045). A picture asked of a mesh model, or a mesh of a picture
model, is refused saying which it makes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 18:30:52 +02:00
Ilker ManapandClaude Opus 5.5 ec233d3586 Merge admission-bulk into jobs-3d
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 17:35:39 +02:00
Ilker ManapandClaude Opus 5.5 497e4b24a1 A band may wait longer than the rest; a busy refusal says when to return
A knowledge-base import sends far more than the cluster can take at once,
and every band waited the same 30 s. A band now has its own queue_ttl, so
one for bulk work waits behind people instead of being turned away.

And a request turned away because the cluster stayed busy — the queue
full (503) or waited as long as its band may (504) — now carries
Retry-After: 10. Without it llmgw-kb, which retries only what says when
to, counted each such passage as failed; that is what turned parts of the
metallurgy import away.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 16:11:09 +02:00
Ilker ManapandClaude Fable 5.1 b51cc3cf92 A 3D model is a job: queued with an id, asked after, its result a file
POST /v1/3d/generations answers 202 with a job; GET says its status,
progress and result file; cancel stops it in the engine's queue too. The
request is kept as sent and replayed by whichever gateway holds the lease
through the path a client's request takes, so it is charged and judged
like any request (tagged job=<id>), and a model still loading is waited
out for as long as the job has. Migration 0044. Nothing makes 3D models
yet; the door refuses every model until the ComfyUI adapter learns meshes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-10 16:04:11 +02:00
Ilker ManapandClaude Opus 5.5 fde1501e8a ComfyUI as an engine of the cluster's own
A model whose profiles run engine comfyui is placed on a node's card like
any other, and makes pictures there. ComfyUI asks for no key, so the agent
starts it behind llmgw comfyui-gate: ComfyUI on the loopback, the gate on
the replica's port demanding the replica's key and letting through only
what the router asks — queueing, waiting, fetching, cancelling. The gate
answers the agent's /health and /v1/models, shows ComfyUI the fetched
weights by their own names, and gives each replica its own base directory.
ComfyUI runs as the gate's child, so its card memory is the replica's.

A node offers the engine when agent.yaml names its ComfyUI
(engines.comfyui.dir). The model's comfyui: block names the workflow and,
for a source that is a set of files, the checkpoint.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 00:39:10 +02:00
Ilker ManapandClaude Opus 5.5 698f82cc12 The pictures page is a session page, not asked for an API key
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-09 22:32:57 +02:00
Ilker ManapandClaude Opus 5.5 62db7c5586 A page that makes pictures, beside the chat
/images offers the models that make pictures, and the chat no longer
offers them. Like the chat, the browser holds a session and never a key;
the page's request is rebuilt from its own fields and sent down the path
every /v1/images/generations request takes. Size, how many, seed, steps
and a negative prompt; each picture shown with its seed, a download link
and a way to make it again. Nothing is kept but the usage. The menu
offers the page while some model makes pictures.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-09 22:24:43 +02:00
Ilker ManapandClaude Opus 5.5 4a4fe8fa16 Pictures are priced by the picture
auth.pricing takes per_image, charged on top of a prompt's tokens; a model
priced only per picture is not free, so a spent budget still stops it.
Each usage row keeps how many pictures were made (migration 0041), and
ctl request and the admin API's request view show them.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-09 21:48:47 +02:00
Ilker ManapandClaude Opus 5.5 2f06c0e513 Pictures from ComfyUI at /v1/images/generations
An external model with api: comfyui is a ComfyUI server. A request for
pictures goes through the same key, allowlist, limit, admission and guard
checks as any other; at the replica it becomes a workflow — a built-in
(flux-schnell, flux-dev) or a JSON file in API format, with {{prompt}},
{{width}}, {{seed}} and the like filled in — queued, waited on, and its
pictures sent back as b64_json. A client that leaves takes its workflow
off the queue or interrupts it. ComfyUI is never scraped for metrics.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-09 21:15:50 +02:00
Ilker ManapandClaude Opus 5.5 6b7750a0dd An embedding model's query_prefix, put before questions and never before documents
Qwen3-Embedding and models like it are trained with an instruction on the
query side only. The prefix goes before the question a knowledge-base
search embeds, at answer time and in the admin API's search; documents
reach /v1/embeddings from llmgw-kb as they are.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-09 18:03:00 +02:00
Ilker ManapandClaude Opus 5.5 92733317a9 A batch of embeddings is checked text by text against the model's context
llmgw-kb sends many passages in one /v1/embeddings request. Their sizes
were added together and compared with the context, so a batch of passages
that each fit was refused as one over-long prompt. The longest input is
what has to fit; the total still counts for usage and limits.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-09 11:43:21 +02:00
Ilker ManapandClaude Opus 5.5 85025dfb03 Trained adapters published, private to their owners like the knowledge bases they come from
An adapter now has an owner (org, team, project, user or key): only callers
there see it, and to anyone else it is a model that does not exist, absent
from /v1/models too. A training job's owner defaults to the one collection
its pairs come from, must be able to read every collection they come from,
and pairs from a knowledge base are never made public. ctl train publish
keeps the GGUF or PEFT files on the mirror, which accepts uploads now, and
sets the owned adapter after the usual architecture check.

The knowledge bases' cache_salt follows the collections searched rather
than the organization, so two teams of one organization no longer share
cached passages.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 21:30:50 +02:00
Ilker ManapandClaude Opus 5.5 659f20dc9a The cluster's settings edited on the portal, by administrators
/admin/settings edits the document llmgw ctl config edit opens: checked as a
control plane would load it, refused when made from a version since replaced
(the edit kept), its history viewed and rolled back as a new version, every
change audited by size and never by contents, since it can hold webhook
secrets. Administrators only; in English and Turkish.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 17:23:13 +02:00
Ilker ManapandClaude Opus 5.5 26630274f4 Passages of one organization never share a vLLM prefix cache with another's
A request given passages carries a cache_salt derived from its organization
when the replica chosen is vLLM, so how fast an answer starts cannot say what
another tenant's documents held. Other engines are not sent the field.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 16:02:19 +02:00
Ilker ManapandClaude Opus 5.5 5ee8e74de1 The portal's chat answers from the documents a person picks
A Documents picker on /chat offers the collections the person may read in
the chosen project, the default when there is one, or none; the sources an
answer was given are listed under it. The page's choice becomes the
X-Llmgw-Knowledge header; one the page sent itself is dropped. In English
and Turkish.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 15:59:23 +02:00
Ilker ManapandClaude Opus 5.5 0093cc3d45 Collections a part of the tree answers from when a request names none
On an organization, team, project or key, the narrowest winning: llmgw ctl
kb default, POST /api/v1/kb/defaults, and the console's Knowledge screen. A
request's own X-Llmgw-Knowledge overrides it and none turns it off. A default
never refuses a request: a collection the caller cannot read is left out, and
a failed search answers without passages. Removing a collection takes it out
of every default.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 15:52:43 +02:00
Ilker ManapandClaude Opus 5.5 54ba04c52a llmgw-kb collections names each collection's owner as people know it
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 15:42:40 +02:00
Ilker ManapandClaude Opus 5.5 496625824b Chat requests answered from knowledge bases
A request naming collections in X-Llmgw-Knowledge has its last question
embedded on the caller's account, the best passages found and added to the
system message, numbered, with an instruction to cite them and to say when
they do not hold the answer; x-llmgw-sources says which were given. Done
before the guardrails judge the request, never served from the answer cache,
and a collection the key may not read is answered as one that does not
exist. Metrics llmgw_kb_retrievals_total and llmgw_kb_retrieval_seconds.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 15:27:50 +02:00
Ilker ManapandClaude Opus 5.5 ba5df90602 llmgw-kb: sync a person's documents into a knowledge base from their own machine
Its own binary for Windows, macOS and Linux, carrying no server code. It reads
the files there, cuts them, has the gateway embed them with the person's key
and sends text and vectors for what changed only: a SQLite ledger on the
machine remembers each document's file, size, time, digest and what happened,
so a rerun reads nothing unchanged, a file skipped stays skipped until
--retry, and --delete removes only what this machine sent under the paths
given. --dry-run says what would leave the machine. Commands: login,
collections, sync, status, history, formats.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 13:43:47 +02:00
Ilker ManapandClaude Opus 5.5 d9e030e0b6 Knowledge bases on the client listener: list, manifest, put and delete a document
What llmgw-kb, run by a person with their own key, reads and writes. A
collection the key may not read answers as one that does not exist; writing
needs ownership or a write grant on the key's chain; a personal key names its
project in X-Llmgw-Project and must be a member of it. Changes are audited by
document name.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 13:27:14 +02:00
Ilker ManapandClaude Opus 5.5 7245dbe1dd llmgw stop, start and restart act on the whole system
Issue #3 asked for commands to stop and start the system, and 05aa21c gave
only one machine's services. Now:

- llmgw stop sets serving.stopped in the cluster's settings, live: the
  reconciler wants every model at no replicas, freeing every GPU, and the
  gateway answers every POST under /v1/ with 503 system_stopped. The control
  planes, agents and mirror keep running, and no model's definition changes.
- llmgw start clears it; every model is placed again as it asks.
- llmgw restart has every agent restart over its stream (RestartAgent), then
  the other control planes over ssh, standbys before a leader, then this
  one; engines keep serving.
- llmgw status is the cluster's summary, saying when it is stopped.

One machine's services moved to llmgw service status|start|stop|restart.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 18:01:45 +02:00
Ilker ManapandClaude Opus 5.5 66f702d5c1 A forgotten password is set through a one-time link
/forgot mails a link to set a new password when the cluster's mail settings
say how - SMTP relay, sender, and the public base_url the link points at,
never the request's Host - and says the same whether or not the address has
an account. /reset sets it once, closing every session. Links are kept by
their hash in password_resets, good for an hour, a newer one replacing an
older. Without mail, /forgot says to ask an operator, and llmgw users
reset-link makes the link to hand over. Single sign-on accounts get none.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 13:20:11 +02:00
Ilker ManapandClaude Opus 5.5 249a91aeec Guardrails turn off and on; a judge's refusal says which hazard; ctl guard judge ask
guard.enabled (unset is on) makes every policy check nothing while false,
live: llmgw ctl guard off/on set it in the cluster's settings, and ctl
status, ctl guard list and the guardrails page say when they are off.

A refusal by a judge names the hazard it saw - judge: S6,Specialized
Advice - and the verdict is kept with the request in a new usage column,
guard_rule, shown by ctl request, the request log and the guardrails page.

llmgw ctl guard judge ask has a judge read a text, past the cache, and shows
its raw answer and how it is read; the endpoint is kept out of the audit so
no text is recorded.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 12:41:08 +02:00
Ilker ManapandClaude Opus 5.5 6289b01ed6 The cluster's settings live in the store, and every change takes effect
Everything in server.yaml that is not one machine's own is now a versioned
document in the store (cluster_settings), the same on every control plane.
The first start against a store imports the file's cluster keys, with the
baseline guard policy when there is none; after that those keys in a file
are ignored, with a warning.

Each control plane polls the document's version and its own server.yaml's
hash. A change is validated and applied: guard live (Gate.Replace), anything
else by a graceful restart that systemd brings back, standbys first and the
leader 30s later. A version that does not load on a machine is not applied;
at start the newest one that loads is used, so a bad edit cannot keep every
control plane down.

llmgw ctl config show/get/set/unset/edit/history/rollback, the admin API
under /api/v1/settings, and the console's Settings screen, which layers the
store's document over the machine's file, write it. Commands beside the
control plane that read cluster keys (init, node add, server add, backup,
upgrade) read the layered configuration; init --bundle writes offline to
the store once it holds the settings.

A standby that becomes the leader re-reads models, engine choices and
facility holds from the store.

Retires appendGuardIfMissing: the import does it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 15:25:23 +02:00