Ilker ManapandClaude Opus 5.5 94eec557ad Serve a model from an external OpenAI-compatible provider ...

A catalog model with external {url, model, api_key_file} and no profiles
is published as an endpoint by the control plane, the key read from its
file each time; the router rewrites the request's model to the provider's
name and sends its key. It is never scraped nor stale, and nothing is
placed for it. Named in fallbacks, it serves when the cluster has no room.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

2026-09-30 23:29:01 +02:00

..

agent.go

Add llmgw doctor: what a node's GPUs, NVLink and fabric need, checked

2026-09-29 03:51:09 +02:00

backup_test.go

Close what a second security review found still open

2026-09-30 02:25:07 +02:00

backup.go

Close what a second security review found still open

2026-09-30 02:25:07 +02:00

bench.go

Give every engine a key only the router holds

2026-09-30 00:38:29 +02:00

capacity.go

Keep what a benchmark found out, instead of leaving it in a terminal

2026-09-27 11:51:58 +02:00

cert.go

Add a private CA and mutual TLS for the agent stream

2026-09-16 07:28:56 +02:00

ctl_fabrictest.go

Measure two nodes' fabric rail by rail: llmgw ctl fabric-test

2026-09-29 21:29:23 +02:00

ctl_logs.go

Show a node's logs on the admin page

2026-09-28 12:27:47 +02:00

ctl_request.go

Follow one request from its id to the replica that served it

2026-09-30 02:36:45 +02:00

ctl_restart.go

Park a model that fails every time it starts, and say so everywhere

2026-09-30 01:21:57 +02:00

ctl_status_test.go

Read a queued change as a success in ctl

2026-09-30 01:14:12 +02:00

ctl_upgrade_test.go

Upgrade the cluster with one command, over the agents' own streams

2026-09-30 02:54:36 +02:00

ctl_upgrade.go

Release v0.10.5

2026-09-30 22:26:19 +02:00

ctl.go

Serve a model from an external OpenAI-compatible provider

2026-09-30 23:29:01 +02:00

ctlweights.go

Carry a download's progress to where somebody is looking at it

2026-09-27 07:49:08 +02:00

dispatcherref_test.go

Measure two nodes' fabric rail by rail: llmgw ctl fabric-test

2026-09-29 21:29:23 +02:00

doctor.go

Install vLLM on every GPU node that can run it, as llama.cpp is

2026-09-29 13:15:54 +02:00

enginecheck_test.go

Ship binaries with releases, and ask engines whether our flags still exist

2026-09-16 16:44:29 +02:00

enginecheck.go

Check llmgw's engine flags against real vLLM and llama.cpp

2026-09-16 16:52:11 +02:00

filter_test.go

Let ctl ask a question instead of printing the whole fleet

2026-09-27 07:30:51 +02:00

filter.go

Let ctl ask a question instead of printing the whole fleet

2026-09-27 07:30:51 +02:00

keys.go

Let a person belong to many places, or to none

2026-09-28 19:03:46 +02:00

memory.go

Manage people and the tenancy tree, from the web and the CLI

2026-09-28 16:26:39 +02:00

mirror_gate_test.go

Release v0.10.4: CPU threads, tracing, and the rest of the review

2026-09-30 04:00:35 +02:00

mirror_test.go

Say when a replica is running an edited profile, and check a source from the web too

2026-09-27 22:42:26 +02:00

mirror.go

Release v0.10.4: CPU threads, tracing, and the rest of the review

2026-09-30 04:00:35 +02:00

mirrorinit.go

Put the mirror on its own machine in two commands

2026-09-29 22:17:24 +02:00

model_serve.go

Reserve CPU cores and require memory for CPU models

2026-09-30 17:43:13 +02:00

model.go

Scale and place by what a replica was measured to serve

2026-09-30 18:21:08 +02:00

modelplan.go

Make the planned entry use an engine that can serve the precision

2026-09-17 07:48:20 +02:00

movestore.go

Move a store to another one without reissuing every key

2026-09-25 16:28:38 +02:00

node.go

Let server init ask for what it needs, and node add take several nodes

2026-09-29 21:54:18 +02:00

org_test.go

Let a person belong to many places, or to none

2026-09-28 19:03:46 +02:00

org.go

Let a person belong to many places, or to none

2026-09-28 19:03:46 +02:00

quota.go

Set capacity quotas, class, owner and reservation from the admin pages

2026-09-30 22:40:45 +02:00

release.go

Sign releases, and install only signed ones once a key is trusted

2026-09-30 23:03:48 +02:00

resolve.go

Manage people and the tenancy tree, from the web and the CLI

2026-09-28 16:26:39 +02:00

root.go

Sign releases, and install only signed ones once a key is trusted

2026-09-30 23:03:48 +02:00

selfupgrade.go

Release v0.10.5

2026-09-30 22:26:19 +02:00

servable_test.go

Refuse weights the nodes' llama.cpp cannot load, before fetching them

2026-09-28 20:44:36 +02:00

server.go

Scale and place by what a replica was measured to serve

2026-09-30 18:21:08 +02:00

serverinit_cluster_test.go

Put the mirror on its own machine in two commands

2026-09-29 22:17:24 +02:00

serverinit_cluster.go

Put the mirror on its own machine in two commands

2026-09-29 22:17:24 +02:00

serverinit_postgres_test.go

Find the PostgreSQL a machine has, rather than assume its names

2026-09-28 14:47:55 +02:00

serverinit_postgres.go

Find the PostgreSQL a machine has, rather than assume its names

2026-09-28 14:47:55 +02:00

serverinit_test.go

Let a person belong to many places, or to none

2026-09-28 19:03:46 +02:00

serverinit.go

Put the mirror on its own machine in two commands

2026-09-29 22:17:24 +02:00

simengine.go

Onboard a node with a script, and find two examples that never worked

2026-09-16 12:28:36 +02:00

sourcenote_test.go

Say when a replica is running an edited profile, and check a source from the web too

2026-09-27 22:42:26 +02:00

sourcenote.go

Say when a replica is running an edited profile, and check a source from the web too

2026-09-27 22:42:26 +02:00

status_test.go

Make the planned entry use an engine that can serve the precision

2026-09-17 07:48:20 +02:00

status.go

Park a model that fails every time it starts, and say so everywhere

2026-09-30 01:21:57 +02:00

storeflag_test.go

Stop calling the PostgreSQL store SQLite, and stop guessing where it is

2026-09-27 12:20:40 +02:00

storeflag.go

Stop calling the PostgreSQL store SQLite, and stop guessing where it is

2026-09-27 12:20:40 +02:00

threads.go

Manage people and the tenancy tree, from the web and the CLI

2026-09-28 16:26:39 +02:00

toolcheck_test.go

Stop tool-check blaming the engine for what the model did

2026-09-18 08:31:09 +02:00

toolcheck.go

Stop tool-check blaming the engine for what the model did

2026-09-18 08:31:09 +02:00

tui_init_test.go

Put the mirror on its own machine in two commands

2026-09-29 22:17:24 +02:00

tui_init.go

Release v0.10.4: CPU threads, tracing, and the rest of the review

2026-09-30 04:00:35 +02:00

tui_node_test.go

Let server init ask for what it needs, and node add take several nodes

2026-09-29 21:54:18 +02:00

tui_node.go

Print only what must outlive init's screen, and drop bubbles

2026-09-29 22:37:28 +02:00

tui_widgets.go

Print only what must outlive init's screen, and drop bubbles

2026-09-29 22:37:28 +02:00

usage.go

Show what a group spent, and fix the reports that never ran on PostgreSQL

2026-09-27 12:26:14 +02:00

users.go

Give an organization's managers a page of their own

2026-09-28 19:23:12 +02:00