This website requires JavaScript.
e12996f7dd
Serve and update models in the background from the console, and follow them under Jobs
Ilker Manap and Claude Opus 5.5
2026-10-04 01:27:02 +02:00
d311a3ca4a
Show the version top right on every screen
Ilker Manap and Claude Opus 5.5
2026-10-04 01:18:45 +02:00
7cb9d4fa01
Busy is not unfit: rank files by the hardware the cluster has, and let a model wait for its cards
Ilker Manap and Claude Opus 5.5
2026-10-04 01:17:32 +02:00
c2be429554
Tab or down from a form's Apply goes round to the first field instead of crashing the console
Ilker Manap and Claude Opus 5.5
2026-10-04 01:12:19 +02:00
3b22ccce72
Merge fit-planner: plan a repository's files against the fleet, and serve the one that suits it
Ilker Manap and Claude Opus 5.5
2026-10-04 01:09:40 +02:00
3b66784f22
Name each file by its quantisation first in the console's plan
Ilker Manap and Claude Opus 5.5
2026-10-04 01:09:05 +02:00
0ea3008f64
On a CPU prefer a middling quantisation, on a card eight bits; past them a bigger file ranks lower
Ilker Manap and Claude Opus 5.5
2026-10-04 01:05:55 +02:00
54db7aecb7
Plan a repository's files against the fleet, and serve the one that suits it the way it fits
Ilker Manap and Claude Opus 5.5
2026-10-04 01:03:48 +02:00
630d2788c0
Merge model updates, the manual, and models of several files
Ilker Manap and Claude Opus 5.5
2026-10-04 00:43:15 +02:00
a16097cfe8
Serve vision models with their projector and original weights on vLLM; group the mirror by version
Ilker Manap and Claude Opus 5.5
2026-10-04 00:42:59 +02:00
7aefb236d6
Publish llmgw_model_update_available, and alert on it
Ilker Manap and Claude Opus 5.5
2026-10-04 00:36:27 +02:00
f36c46a881
Trace weights fetched before origins were kept, and let an origin be given by hand
Ilker Manap and Claude Opus 5.5
2026-10-04 00:24:41 +02:00
dac7ff0e99
The manual in the binary: llmgw docs, and Help in the console
Ilker Manap and Claude Opus 5.5
2026-10-04 00:19:50 +02:00
44425193bd
Track the registry's new weights for served models, and move a model to them or back
Ilker Manap and Claude Opus 5.5
2026-10-04 00:12:20 +02:00
a96b1a54b2
Hugging Face in the console: models that chat when browsing, ages read at a glance, long menus scroll
Ilker Manap and Claude Opus 5.5
2026-10-03 23:47:26 +02:00
2bf43ed90c
Search Hugging Face from the console and serve what is found
Ilker Manap and Claude Opus 5.5
2026-10-03 23:44:51 +02:00
6a7b4bb6a0
Merge tui-actions: everything settable on a running cluster, from the console
Ilker Manap and Claude Opus 5.5
2026-10-03 23:35:14 +02:00
eb7dac8a67
The console manages people, keys, organizations and the mirror, and scrolls a tall sidebar
Ilker Manap and Claude Opus 5.5
2026-10-03 23:30:24 +02:00
c7d5ee16b5
Act on the console's rows: scale, class, owner, resources, adapters, restart, remove; cordon, drain, engines, upgrade; quotas; cluster tasks
Ilker Manap and Claude Opus 5.5
2026-10-03 23:23:12 +02:00
e0384fb148
Release v0.11.1
v0.11.1
Ilker Manap and Claude Opus 5.5
2026-10-03 23:10:20 +02:00
6e815ca21a
Draw the console's overview in its panel, not over the screen
Ilker Manap and Claude Opus 5.5
2026-10-03 23:09:40 +02:00
36cdbce125
Merge ha: replicated store, second control plane, join codes, setup and console screens
v0.11.0
Ilker Manap and Claude Opus 5.5
2026-10-03 22:59:49 +02:00
211688fe7c
Merge usage-rollups: usage sums, per-tenant stats, alerts and dashboard
Ilker Manap and Claude Opus 5.5
2026-10-03 22:56:50 +02:00
9ab93a0aa6
Watch the cluster and edit every setting from the console
Ilker Manap and Claude Opus 5.5
2026-10-03 22:54:09 +02:00
aa3efcebbe
Add a standby control plane with server add, and draw the screens as windows
Ilker Manap and Claude Opus 5.5
2026-10-03 22:29:12 +02:00
f5c79052b1
Let a machine join a setup by code, dialling out, with nothing logging in
Ilker Manap and Claude Opus 5.5
2026-10-03 21:49:15 +02:00
38f2671ed9
Open setup screens or a console when llmgw runs bare in a terminal
Ilker Manap and Claude Opus 5.5
2026-10-03 21:33:59 +02:00
975c93994f
Set up a replicated store with server init --patroni, and watch it
Ilker Manap and Claude Opus 5.5
2026-10-03 20:57:41 +02:00
7a562ac1e7
Keep the store under Patroni, and back up from a store of several hosts
Ilker Manap and Claude Opus 5.5
2026-10-03 20:31:19 +02:00
17cc2e379b
Add a Grafana dashboard per tenant
Ilker Manap and Claude Opus 5.5
2026-10-03 19:58:39 +02:00
ce006a6e6f
Publish per-tenant request and budget series, and alert on them
Ilker Manap and Claude Opus 5.5
2026-10-03 19:55:24 +02:00
9f479b9a70
Show each team's first-token and latency percentiles, errors and refusals
Ilker Manap and Claude Opus 5.5
2026-10-03 19:50:16 +02:00
4454e1322c
Read usage from sums kept as it is written, not from every request
Ilker Manap and Claude Opus 5.5
2026-10-03 19:43:01 +02:00
febcf251e4
Send an administrator from /org to the people page's panel
v0.10.7-rc.11
Ilker Manap and Claude Opus 5.5
2026-10-03 19:15:25 +02:00
fffed716be
Make people, limits and usage one page, with a panel per branch
v0.10.7-rc.10
Ilker Manap and Claude Opus 5.5
2026-10-03 19:10:48 +02:00
1b89d5cdea
Say on the chat page whether a GPU or a CPU answered
v0.10.7-rc.9
Ilker Manap and Claude Opus 5.5
2026-10-03 18:57:50 +02:00
477d0bf19d
Judge a card shared with another process every heartbeat, not once
v0.10.7-rc.8
Ilker Manap and Claude Opus 5.5
2026-10-03 18:46:23 +02:00
372ef64889
Count the GPU time each owner's models held, on the usage page
v0.10.7-rc.7
Ilker Manap and Claude Opus 5.5
2026-10-01 16:07:59 +02:00
82a63cfc6e
Read what wears a GPU out: XIDs, remapped rows, PCIe and NVLink errors
Ilker Manap and Claude Opus 5.5
2026-10-01 16:02:07 +02:00
ff554c7062
Show what each model gets from its GPUs, per card and per watt-hour
Ilker Manap and Claude Opus 5.5
2026-10-01 15:56:42 +02:00
b6a802c03b
Map the fleet's GPUs, and name the cards held and idle
Ilker Manap and Claude Opus 5.5
2026-10-01 15:54:58 +02:00
bd36798b15
Sample GPU utilisation and memory every heartbeat, and chart them
v0.10.7-rc.6
Ilker Manap and Claude Opus 5.5
2026-10-01 15:42:17 +02:00
dc483d8c68
Choose each engine's build from the node's hardware, for every GPU generation
v0.10.7-rc.5
Ilker Manap and Claude Opus 5.5
2026-10-01 14:35:58 +02:00
43d7c4eac1
Let the upgrade repair what the field's catalog already holds
v0.10.7-rc.4
Ilker Manap and Claude Opus 5.5
2026-10-01 14:20:11 +02:00
86e9a1cb83
Give V100s the last vLLM built for them, not a build without their kernels
v0.10.7-rc.3
Ilker Manap and Claude Opus 5.5
2026-10-01 14:04:45 +02:00
4f280a3546
Give each GPU node a vLLM build it can run, chosen on the cluster page
v0.10.7-rc.2
Ilker Manap and Claude Opus 5.5
2026-10-01 12:39:14 +02:00
2735c5a318
Mirror a many-file model whole, with its checksums, for vLLM
v0.10.7-rc.1
Ilker Manap and Claude Opus 5.5
2026-10-01 12:21:04 +02:00
8957db30a2
Recognise a duplicate sign-up on PostgreSQL, and say what to do
Ilker Manap and Claude Opus 5.5
2026-10-01 12:15:37 +02:00
2ef4220643
Fail a node only for the fabrics it declares
Ilker Manap and Claude Opus 5.5
2026-10-01 12:13:18 +02:00
6014d1fee8
Say how a restart with no room beside it goes
Ilker Manap and Claude Opus 5.5
2026-10-01 12:11:46 +02:00
a3ee48aced
Restart in place at once when only the old replica is in the way
Ilker Manap and Claude Opus 5.5
2026-10-01 12:11:37 +02:00
e313f32696
Decide engines per node, and let a node join without one it cannot run
Ilker Manap and Claude Opus 5.5
2026-10-01 12:07:15 +02:00
cba3e329fc
Let the mirror's serve form place a model on a GPU
Ilker Manap and Claude Opus 5.5
2026-10-01 11:59:22 +02:00
cbd5521f4f
Place llama.cpp across several GPUs of a node with gpus_per_node
Ilker Manap and Claude Opus 5.5
2026-10-01 11:56:34 +02:00
057be16a92
Do not call a GPU shard ready while it computes on the CPU
Ilker Manap and Claude Opus 5.5
2026-10-01 11:54:39 +02:00
0c7934ba96
v0.10.6
v0.10.6
Ilker Manap and Claude Opus 5.5
2026-10-01 08:52:01 +02:00
e40b9ac202
Load LoRA adapters at scale zero
v0.10.6-rc.8
Ilker Manap and Claude Opus 5.5
2026-10-01 07:39:12 +02:00
323d172538
Attach LoRA adapters from the command line
v0.10.6-rc.7
Ilker Manap and Claude Opus 5.5
2026-10-01 07:30:13 +02:00
6b1fa65f59
Format kind_live_test.go
v0.10.6-rc.6
Ilker Manap and Claude Opus 5.5
2026-10-01 07:20:35 +02:00
663770226e
Find LoRA adapters on Hugging Face and attach them from the mirror page
v0.10.6-rc.5
Ilker Manap and Claude Opus 5.5
2026-10-01 07:19:08 +02:00
cf53904cc5
Serve LoRA adapters on llama.cpp models, asked for by name
v0.10.6-rc.4
Ilker Manap and Claude Opus 5.5
2026-09-30 23:40:47 +02:00
94eec557ad
Serve a model from an external OpenAI-compatible provider
v0.10.6-rc.3
Ilker Manap and Claude Opus 5.5
2026-09-30 23:29:01 +02:00
d5f02c0a4b
Spill a saturated pool's requests to the next pool
Ilker Manap and Claude Opus 5.5
2026-09-30 23:23:32 +02:00
3d3c96e422
Displace the newest lowest-band request when the gateway queue is full
v0.10.6-rc.2
Ilker Manap and Claude Opus 5.5
2026-09-30 23:07:53 +02:00
623cdb5ba3
Sign releases, and install only signed ones once a key is trusted
Ilker Manap and Claude Opus 5.5
2026-09-30 23:03:48 +02:00
561667c9be
Name only the replicas that would make room in headroom
Ilker Manap and Claude Opus 5.5
2026-09-30 23:01:40 +02:00
cf0178bf05
Test that a restart borrows a lower class's room and returns it
Ilker Manap and Claude Opus 5.5
2026-09-30 22:47:12 +02:00
f288f3c423
Set capacity quotas, class, owner and reservation from the admin pages
v0.10.6-rc.1
Ilker Manap and Claude Opus 5.5
2026-09-30 22:40:45 +02:00
5cf0003e74
Release v0.10.5
v0.10.5
Ilker Manap and Claude Opus 5.5
2026-09-30 22:26:19 +02:00
d10fad064e
Mark facility steps 1-5 deferred; node sensors collected
Ilker Manap and Claude Opus 5.5
2026-09-30 20:25:50 +02:00
cd10e086c8
Read nodes' heat and power: GPUs, CPUs, board inlet, RAPL
v0.10.5-rc.14
Ilker Manap and Claude Opus 5.5
2026-09-30 19:49:08 +02:00
8724dc027a
Plan facility-aware placement: rack power, temperature and cooling
Ilker Manap and Claude Opus 5.5
2026-09-30 19:25:31 +02:00
e7a8105aa4
Log a preempted replica's stop with the grace it had
v0.10.5-rc.13
Ilker Manap and Claude Opus 5.5
2026-09-30 18:41:43 +02:00
d0e9813ce9
Of equally idle victims, try the smallest first
v0.10.5-rc.12
Ilker Manap and Claude Opus 5.5
2026-09-30 18:23:08 +02:00
0e73f37128
Stop only the replicas that make the room
v0.10.5-rc.11
Ilker Manap and Claude Opus 5.5
2026-09-30 18:21:58 +02:00
78e41460f9
Scale and place by what a replica was measured to serve
v0.10.5-rc.10
Ilker Manap and Claude Opus 5.5
2026-09-30 18:21:08 +02:00
9dc1b62fdc
Guarantee and limit capacity per part of the tenancy tree
v0.10.5-rc.9
Ilker Manap and Claude Opus 5.5
2026-09-30 18:15:38 +02:00
ac24e204a6
Add capacity classes and preempt lower ones to place a model
v0.10.5-rc.8
Ilker Manap and Claude Opus 5.5
2026-09-30 18:02:59 +02:00
990d82d801
Reserve CPU cores and require memory for CPU models
v0.10.5-rc.7
Ilker Manap and Claude Opus 5.5
2026-09-30 17:43:13 +02:00
26151ec25a
Format web.go
v0.10.5-rc.6
Ilker Manap and Claude Opus 5.5
2026-09-30 11:02:32 +02:00
235bf33dcd
Keep every admin table to one line per record
v0.10.5-rc.5
Ilker Manap and Claude Opus 5.5
2026-09-30 11:02:18 +02:00
7099043c75
Keep each held row on one line
v0.10.5-rc.4
Ilker Manap and Claude Opus 5.5
2026-09-30 10:50:02 +02:00
a32ea0e5ab
Say on each held row how its model runs and where
v0.10.5-rc.3
Ilker Manap and Claude Opus 5.5
2026-09-30 10:38:49 +02:00
d7f1c1d545
Open a held model to its replicas, nodes and room for more
v0.10.5-rc.2
Ilker Manap and Claude Opus 5.5
2026-09-30 10:19:31 +02:00
2ae12ae1fe
Split the mirror page into Held, Explore and Fetch tabs
v0.10.5-rc.1
Ilker Manap and Claude Opus 5.5
2026-09-30 08:31:28 +02:00
62657f4c2a
Release v0.10.4: CPU threads, tracing, and the rest of the review
v0.10.4
Ilker Manap and Claude Opus 5.5
2026-09-30 04:00:35 +02:00
adb1b3cf2a
Add replicas under load, up to a model's max, and take them back when quiet
v0.10.4-rc.9
Ilker Manap and Claude Opus 5.5
2026-09-30 03:18:07 +02:00
bd12b0909a
Serve a model from the registry with one command
v0.10.4-rc.8
Ilker Manap and Claude Opus 5.5
2026-09-30 03:05:13 +02:00
116e75f8cb
Upgrade the cluster with one command, over the agents' own streams
v0.10.4-rc.7
Ilker Manap and Claude Opus 5.5
2026-09-30 02:54:36 +02:00
3bf4b7b709
Tell clients how much of their rate limit is left
v0.10.4-rc.6
Ilker Manap and Claude Opus 5.5
2026-09-30 02:42:21 +02:00
4a652fdcc1
Size a llama.cpp model's memory from its own shape, not its file size
Ilker Manap and Claude Opus 5.5
2026-09-30 02:40:00 +02:00
f8572fb5ee
Follow one request from its id to the replica that served it
Ilker Manap and Claude Opus 5.5
2026-09-30 02:36:45 +02:00
91fe24a105
Close what a second security review found still open
v0.10.4-rc.5
Ilker Manap and Claude Opus 5.5
2026-09-30 02:25:07 +02:00
9bf8256577
Keep backups where the control plane can see them
v0.10.4-rc.4
Ilker Manap and Claude Opus 5.5
2026-09-30 01:50:33 +02:00
1da4a2e1ef
Back up the store and the certificate authority, and alert when it stops
v0.10.4-rc.3
Ilker Manap and Claude Opus 5.5
2026-09-30 01:44:32 +02:00
b661bfead8
Wait for what tests check instead of sleeping and hoping
Ilker Manap and Claude Opus 5.5
2026-09-30 01:36:55 +02:00
37a80a9b44
Park a model that fails every time it starts, and say so everywhere
v0.10.4-rc.2
Ilker Manap and Claude Opus 5.5
2026-09-30 01:21:57 +02:00
5e1b6ecea0
Read a queued change as a success in ctl
Ilker Manap and Claude Opus 5.5
2026-09-30 01:14:12 +02:00
5dac1854c2
Publish small fixes as release candidates of the next version
v0.10.4-rc.1
Ilker Manap and Claude Opus 5.5
2026-09-30 01:00:11 +02:00
ad6d2d348d
Restart replicas without taking them out of service first
Ilker Manap and Claude Opus 5.5
2026-09-30 00:59:26 +02:00