This website requires JavaScript.
5bc71b3d0a
Mirrors kept in step: llmgw mirror serve --peer
Ilker Manap and Claude Opus 5.5
2026-10-23 09:59:59 +02:00
c09d5ad4ee
Outside services read answers redacted and whole; a card's own lanes
Ilker Manap and Claude Opus 5.5
2026-10-29 13:22:39 +01:00
1948a787ae
Fixes from the second review: grants, flags, updates, setup
Ilker Manap and Claude Opus 5.5
2026-10-29 12:31:07 +01:00
ecf5545a7a
VS Code's sign-in keeps its place; recordings checked for secrets
Ilker Manap and Claude Opus 5.5
2026-10-29 11:40:19 +01:00
258591b3f2
What an engine downloads by a Hugging Face id is asked about too
Ilker Manap and Claude Opus 5.5
2026-10-29 11:06:11 +01:00
5eac62d556
Guardrail services merged the stricter way; the request log keeps no restored value
Ilker Manap and Claude Opus 5.5
2026-10-29 11:03:57 +01:00
f94d902cc3
MCP servers: fields that can be emptied, addresses without passwords
Ilker Manap and Claude Opus 5.5
2026-10-29 10:03:27 +01:00
721705b29d
A card in the wrong PCIe slot is said so
Ilker Manap and Claude Opus 5.5
2026-10-29 08:49:16 +01:00
6a870a8c15
The explorer's card figures: MIG, NVLink, GH200, FP8 and FP4
Ilker Manap and Claude Opus 5.5
2026-10-29 08:38:38 +01:00
b986ea180d
Fixes from review: setup's memory check, MCP secrets, VS Code, bandwidth
Ilker Manap and Claude Opus 5.5
2026-10-29 08:23:00 +01:00
54c6b66cb9
Fixes from review: the similar cache, restore, outside services, latency
Ilker Manap and Claude Opus 5.5
2026-10-29 08:15:43 +01:00
fd94265e08
Pickles refused however they come; a string temperature passes again
Ilker Manap and Claude Opus 5.5
2026-10-28 22:19:06 +01:00
44451bacd9
Granting or taking back a server's tools leaves the rest as it was
Ilker Manap and Claude Opus 5.5
2026-10-28 22:11:07 +01:00
7a04248148
A reused answer never carries another caller's values
Ilker Manap and Claude Opus 5.5
2026-10-28 22:03:32 +01:00
03d3bcfd2d
The company's MCP servers on a web page
Ilker Manap and Claude Opus 5.5
2026-10-28 19:26:51 +01:00
98c45ed28b
The company's MCP servers in the console
Ilker Manap and Claude Opus 5.5
2026-10-28 13:23:25 +01:00
89435735de
BACKLOG: the open list as it stands after v0.15.0
Ilker Manap and Claude Opus 5.5
2026-10-28 11:16:49 +01:00
a583359ebf
Agents started together measure their memory apart
Ilker Manap and Claude Opus 5.5
2026-10-28 09:56:02 +01:00
f2c6f12cfb
How fast a model would answer on the cards, as this cluster would lay it out
Ilker Manap and Claude Opus 5.5
2026-10-28 00:19:36 +01:00
e70d6caa2f
The memory is measured after the agent reaches its control plane
Ilker Manap and Claude Opus 5.5
2026-10-27 15:31:21 +01:00
84444e71c8
How fast a model would answer on a CPU, before it is fetched
Ilker Manap and Claude Opus 5.5
2026-10-27 00:09:47 +01:00
2a45068461
The explorer lists what this cluster's hardware holds
Ilker Manap and Claude Opus 5.5
2026-10-26 18:30:16 +01:00
e84631ce3a
The explorer lists only what a node loads
Ilker Manap and Claude Opus 5.5
2026-10-26 13:00:28 +01:00
4466d2d15a
Weights that run code are refused
Ilker Manap and Claude Opus 5.5
2026-10-25 23:17:22 +01:00
6752938e6d
v0.15.0: changelog and README
v0.15.0
Ilker Manap and Claude Opus 5.5
2026-10-25 17:59:44 +01:00
3a3db8cd20
Answers to similar questions, not only the same one
Ilker Manap and Claude Opus 5.5
2026-10-25 17:47:11 +01:00
ed0d1ec0ad
A cached prompt costs the cached rate
Ilker Manap and Claude Opus 5.5
2026-10-25 17:15:48 +01:00
73e1bb6733
Chat goes to the replica that has been answering soonest
Ilker Manap and Claude Opus 5.5
2026-10-25 17:01:30 +01:00
0b37796067
llmgw and Kong, written down after trying Kong 3.9 in front
Ilker Manap and Claude Opus 5.5
2026-10-25 16:55:22 +01:00
9e7de57816
The reference describes a guardrail policy's services
Ilker Manap and Claude Opus 5.5
2026-10-25 16:50:58 +01:00
783dc9e9f6
A guardrail policy can ask someone else's service
Ilker Manap and Claude Opus 5.5
2026-10-25 16:49:56 +01:00
81685ecd75
A request's redacted values are put back in its answer
Ilker Manap and Claude Opus 5.5
2026-10-25 16:41:53 +01:00
d110968243
A machine too small for the models it would serve is said so first
Ilker Manap and Claude Opus 5.5
2026-10-25 16:19:00 +01:00
06baa38858
Traces say model calls in the OpenTelemetry GenAI conventions
Ilker Manap and Claude Opus 5.5
2026-10-25 12:59:20 +01:00
38dfe268c9
VS Code gets the gateway's models from an extension the gateway serves
Ilker Manap and Claude Opus 5.5
2026-10-20 23:12:57 +02:00
fea5774fe5
The install guide shows every screen of a real install
Ilker Manap and Claude Opus 5.5
2026-10-24 01:30:04 +02:00
f8d18efaef
Turkish pages are Turkish to the end
Ilker Manap and Claude Opus 5.5
2026-10-21 09:22:42 +02:00
3a4488b41e
The company's MCP tools under the guardrails, a rate and a breaker
Ilker Manap and Claude Opus 5.5
2026-10-20 23:00:22 +02:00
6bef138c23
The judge check gives each judgment a text of its own
Ilker Manap and Claude Opus 5.5
2026-10-20 22:45:58 +02:00
85f135311e
How fast the judge is, measured, and said at install
Ilker Manap and Claude Opus 5.5
2026-10-20 22:35:19 +02:00
d1bf26528b
keys limits changes only the limits it is given
Ilker Manap and Claude Opus 5.5
2026-10-20 17:32:43 +02:00
4eb49d0e4c
The cluster as a read-only MCP server, llmgw:admin
Ilker Manap and Claude Opus 5.5
2026-10-20 17:18:28 +02:00
4453f469e1
Company MCP servers are registered with ctl mcp, and tools granted with keys limits --tools
Ilker Manap and Claude Opus 5.5
2026-10-20 12:50:55 +02:00
ca77637a71
/mcp serves the company's own MCP servers' tools to keys granted them
Ilker Manap and Claude Opus 5.5
2026-10-20 10:53:24 +02:00
765b6f5603
An MCP client for the company's tool servers
Ilker Manap and Claude Opus 5.5
2026-10-20 10:43:35 +02:00
6dfdfbf2fe
The store keeps the company's MCP servers, and which tools a scope may call
Ilker Manap and Claude Opus 5.5
2026-10-20 10:36:08 +02:00
8ffde2b80c
A big model's download no longer runs out its startup timeout
Ilker Manap and Claude Opus 5.5
2026-10-20 09:45:31 +02:00
4fe9b46451
A plan for a big model: experts spread, any number of nodes, memory said
Ilker Manap and Claude Opus 5.5
2026-10-20 08:45:33 +02:00
314cc5e6a3
The judge init installs is no longer replaced by the leader's own
Ilker Manap and Claude Opus 5.5
2026-10-19 20:37:10 +02:00
34764f5ef0
A node given a role takes only what asks for it
Ilker Manap and Claude Opus 5.5
2026-10-19 17:52:30 +02:00
51ca09287f
A three-machine install can give the judges machines of their own
Ilker Manap and Claude Opus 5.5
2026-10-19 17:45:00 +02:00
8b122aae09
A judge the leader serves by itself is sized as init's is
Ilker Manap and Claude Opus 5.5
2026-10-19 16:32:04 +02:00
99c34fd68f
v0.14.9: changelog and README
v0.14.9
Ilker Manap and Claude Opus 5.5
2026-10-19 16:12:01 +02:00
cfc38d35cc
A node's console opens on what the machine is doing
Ilker Manap and Claude Opus 5.5
2026-10-19 12:05:13 +02:00
ac25a9136a
Sign-in is set up from the console, and a directory tried before it is kept
Ilker Manap and Claude Opus 5.5
2026-10-19 12:55:27 +02:00
4b6b3eeddb
A node's page says which llmgw it runs and which model is on each GPU
Ilker Manap and Claude Opus 5.5
2026-10-19 11:31:52 +02:00
178b168bfd
A broken directory setting, or a second server, no longer takes the cluster down
Ilker Manap and Claude Opus 5.5
2026-10-19 11:10:43 +02:00
2005cea326
v0.14.8: changelog and README
v0.14.8
Ilker Manap and Claude Opus 5.5
2026-10-18 11:00:28 +02:00
f27d722111
People sign in with their Active Directory account
Ilker Manap and Claude Opus 5.5
2026-10-18 10:59:50 +02:00
fe310576e7
Several keys for one outside provider
Ilker Manap and Claude Opus 5.5
2026-10-17 08:56:30 +02:00
688b205604
What llmgw adds to a request, measured: ctl bench overhead
Ilker Manap and Claude Opus 5.5
2026-10-17 00:23:26 +02:00
35ccabac28
modelserve: a job's request is set under the lock Get reads it with
Ilker Manap and Claude Opus 5.5
2026-10-16 22:40:09 +02:00
2545fca174
An upgrade refits the judge an earlier server init installed
Ilker Manap and Claude Opus 5.5
2026-10-16 22:06:56 +02:00
b8274d6278
A restart is not refused for room another restart is giving back
Ilker Manap and Claude Opus 5.5
2026-10-16 21:33:20 +02:00
8164260e3b
Backlog: test VMs removed, Apertus on demand
Ilker Manap and Claude Opus 5.5
2026-10-16 21:18:46 +02:00
fbb5df05b8
Backlog: what is open, in order; J, what the HA and single-machine tests found
Ilker Manap and Claude Opus 5.5
2026-10-16 21:07:57 +02:00
af942aca27
v0.14.7: changelog and README
v0.14.7
Ilker Manap and Claude Opus 5.5
2026-10-16 20:11:26 +02:00
3e7dd20bad
One machine runs one judge, on up to four cores
Ilker Manap and Claude Opus 5.5
2026-10-16 19:35:42 +02:00
f8206c3540
A judge reserves two cores and 2 GB, with no prompt cache
Ilker Manap and Claude Opus 5.5
2026-10-16 19:22:33 +02:00
0def6eaba1
CI: run on maas-packer, the runner that serves this Gitea, given 8 cores and 32 GB
Ilker Manap and Claude Opus 5.5
2026-10-16 18:15:58 +02:00
645be2d278
CI: run again on the runner given 8 cores and 32 GB
Ilker Manap and Claude Opus 5.5
2026-10-16 17:49:44 +02:00
25c4ec9f2a
CI: a package that hangs says which test it hung in
Ilker Manap and Claude Opus 5.5
2026-10-16 16:54:48 +02:00
7eb50c5eb6
v0.14.6: changelog and README
v0.14.6
Ilker Manap and Claude Opus 5.5
2026-10-16 13:33:13 +02:00
b8fc1de822
Backlog I: what other gateways do that llmgw does not
Ilker Manap and Claude Opus 5.5
2026-10-16 13:29:20 +02:00
63b2c6d9a4
The judges and the starter divide a machine's CPUs instead of each taking all
Ilker Manap and Claude Opus 5.5
2026-10-16 13:00:39 +02:00
0b984a92bc
A judge is asked again only on another machine
Ilker Manap and Claude Opus 5.5
2026-10-16 10:27:50 +02:00
bc4c5d5ac9
A judge replica that hangs is overtaken by another at half the budget
Ilker Manap and Claude Opus 5.5
2026-10-16 08:35:28 +02:00
b09922b29b
Leadership stays put while the store's primary is replaced
Ilker Manap and Claude Opus 5.5
2026-10-15 23:17:58 +02:00
5a4b022dcd
A lost store machine is given up on in seconds, and a new leader waits for its judge
Ilker Manap and Claude Opus 5.5
2026-10-15 20:33:13 +02:00
eeaffd764d
A floating address keeps to its own network, and Caddy follows the leader
Ilker Manap and Claude Opus 5.5
2026-10-15 17:35:10 +02:00
94401e21e6
Setup lets every part be chosen, each with its settings; three machines each run the gateway
Ilker Manap and Claude Opus 5.5
2026-10-15 11:22:36 +02:00
733e20924c
Merge remote-tracking branch 'origin/mcp-wait' into sd-dedupe
Ilker Manap
2026-10-15 07:53:22 +02:00
fc60481e40
Every control plane lists the others alike to Prometheus
Ilker Manap and Claude Opus 5.5
2026-10-15 07:53:22 +02:00
edeeb9f8e1
MCP takes pictures of real size, by id too, and waits for a cold model
mcp-wait
Ilker Manap and Claude Opus 5.5
2026-10-15 07:24:59 +02:00
c22528c26e
A model that does not fit its card is not added to fail; weights in use are not removed
Ilker Manap and Claude Opus 5.5
2026-10-15 00:22:01 +02:00
bda988b78f
Merge remote-tracking branch 'origin/setup-webhooks' into shared-replica
Ilker Manap
2026-10-14 14:26:27 +02:00
a7cb3aac3c
Merge remote-tracking branch 'origin/mcp-media' into shared-replica
Ilker Manap
2026-10-14 14:01:24 +02:00
d1e91868ee
Setup asks where alerts go, and sends each a test
setup-webhooks
Ilker Manap and Claude Opus 5.5
2026-10-14 13:55:48 +02:00
aa531b1053
ComfyUI models of one environment share one replica
Ilker Manap and Claude Opus 5.5
2026-10-14 13:55:34 +02:00
55d5e13534
Pictures, 3D models, a vision model and workflows as tools at /mcp
mcp-media
Ilker Manap and Claude Opus 5.5
2026-10-14 13:11:09 +02:00
7c53380e68
Extended thinking on /v1/messages: a model's reasoning as thinking blocks
Ilker Manap and Claude Opus 5.5
2026-10-14 11:05:02 +02:00
0460f52dd2
Jobs run side by side, one per model at a time
Ilker Manap and Claude Opus 5.5
2026-10-14 10:06:20 +02:00
efffdb66b7
An embedding model is served for embedding, with many slots
Ilker Manap and Claude Opus 5.5
2026-10-14 09:19:57 +02:00
f5eb6963af
An upgrade that leaves nodes behind says which
Ilker Manap and Claude Opus 5.5
2026-10-14 09:01:09 +02:00
1e7086e13e
How long an on-demand model holds a card is a live setting
Ilker Manap and Claude Opus 5.5
2026-10-14 08:58:38 +02:00
a9a2360f44
pages.js is loaded by its versioned address
Ilker Manap and Claude Opus 5.5
2026-10-14 08:15:44 +02:00
e78075876b
A form being filled in is not refreshed away
Ilker Manap and Claude Opus 5.5
2026-10-14 08:13:08 +02:00
45666e72db
v0.14.5: changelog and README
v0.14.5
Ilker Manap and Claude Opus 5.5
2026-10-14 01:33:59 +02:00
2eb2a6d8fe
A mesh carries its whole licence, not only its name
Ilker Manap and Claude Opus 5.5
2026-10-14 01:12:47 +02:00
cfc636e328
A mesh's base colour can be brought to the picture it was made from
Ilker Manap and Claude Opus 5.5
2026-10-14 00:51:06 +02:00