-
7a562ac1e7
Keep the store under Patroni, and back up from a store of several hosts

Ilker ManapandClaude Opus 5.5
2026-10-03 20:31:19 +02:00
-
-
-
17cc2e379b
Add a Grafana dashboard per tenant

Ilker ManapandClaude Opus 5.5
2026-10-03 19:58:39 +02:00
-
ce006a6e6f
Publish per-tenant request and budget series, and alert on them

Ilker ManapandClaude Opus 5.5
2026-10-03 19:55:24 +02:00
-
9f479b9a70
Show each team's first-token and latency percentiles, errors and refusals

Ilker ManapandClaude Opus 5.5
2026-10-03 19:50:16 +02:00
-
4454e1322c
Read usage from sums kept as it is written, not from every request

Ilker ManapandClaude Opus 5.5
2026-10-03 19:43:01 +02:00
-
-
febcf251e4
Send an administrator from /org to the people page's panel

Ilker ManapandClaude Opus 5.5
2026-10-03 19:15:25 +02:00
-
fffed716be
Make people, limits and usage one page, with a panel per branch

Ilker ManapandClaude Opus 5.5
2026-10-03 19:10:48 +02:00
-
1b89d5cdea
Say on the chat page whether a GPU or a CPU answered

Ilker ManapandClaude Opus 5.5
2026-10-03 18:57:50 +02:00
-
477d0bf19d
Judge a card shared with another process every heartbeat, not once

Ilker ManapandClaude Opus 5.5
2026-10-03 18:46:23 +02:00
-
372ef64889
Count the GPU time each owner's models held, on the usage page

Ilker ManapandClaude Opus 5.5
2026-10-01 16:07:59 +02:00
-
82a63cfc6e
Read what wears a GPU out: XIDs, remapped rows, PCIe and NVLink errors

Ilker ManapandClaude Opus 5.5
2026-10-01 16:02:07 +02:00
-
ff554c7062
Show what each model gets from its GPUs, per card and per watt-hour

Ilker ManapandClaude Opus 5.5
2026-10-01 15:56:42 +02:00
-
b6a802c03b
Map the fleet's GPUs, and name the cards held and idle

Ilker ManapandClaude Opus 5.5
2026-10-01 15:54:58 +02:00
-
bd36798b15
Sample GPU utilisation and memory every heartbeat, and chart them

Ilker ManapandClaude Opus 5.5
2026-10-01 15:42:17 +02:00
-
dc483d8c68
Choose each engine's build from the node's hardware, for every GPU generation

Ilker ManapandClaude Opus 5.5
2026-10-01 14:35:58 +02:00
-
43d7c4eac1
Let the upgrade repair what the field's catalog already holds

Ilker ManapandClaude Opus 5.5
2026-10-01 14:20:11 +02:00
-
86e9a1cb83
Give V100s the last vLLM built for them, not a build without their kernels

Ilker ManapandClaude Opus 5.5
2026-10-01 14:04:45 +02:00
-
4f280a3546
Give each GPU node a vLLM build it can run, chosen on the cluster page

Ilker ManapandClaude Opus 5.5
2026-10-01 12:39:14 +02:00
-
2735c5a318
Mirror a many-file model whole, with its checksums, for vLLM

Ilker ManapandClaude Opus 5.5
2026-10-01 12:21:04 +02:00
-
8957db30a2
Recognise a duplicate sign-up on PostgreSQL, and say what to do

Ilker ManapandClaude Opus 5.5
2026-10-01 12:15:37 +02:00
-
2ef4220643
Fail a node only for the fabrics it declares

Ilker ManapandClaude Opus 5.5
2026-10-01 12:13:18 +02:00
-
6014d1fee8
Say how a restart with no room beside it goes

Ilker ManapandClaude Opus 5.5
2026-10-01 12:11:46 +02:00
-
a3ee48aced
Restart in place at once when only the old replica is in the way

Ilker ManapandClaude Opus 5.5
2026-10-01 12:11:37 +02:00
-
e313f32696
Decide engines per node, and let a node join without one it cannot run

Ilker ManapandClaude Opus 5.5
2026-10-01 12:07:15 +02:00
-
cba3e329fc
Let the mirror's serve form place a model on a GPU

Ilker ManapandClaude Opus 5.5
2026-10-01 11:59:22 +02:00
-
cbd5521f4f
Place llama.cpp across several GPUs of a node with gpus_per_node

Ilker ManapandClaude Opus 5.5
2026-10-01 11:56:34 +02:00
-
057be16a92
Do not call a GPU shard ready while it computes on the CPU

Ilker ManapandClaude Opus 5.5
2026-10-01 11:54:39 +02:00
-
0c7934ba96
v0.10.6

Ilker ManapandClaude Opus 5.5
2026-10-01 08:52:01 +02:00
-
e40b9ac202
Load LoRA adapters at scale zero

Ilker ManapandClaude Opus 5.5
2026-10-01 07:39:12 +02:00
-
323d172538
Attach LoRA adapters from the command line

Ilker ManapandClaude Opus 5.5
2026-10-01 07:30:13 +02:00
-
6b1fa65f59
Format kind_live_test.go

Ilker ManapandClaude Opus 5.5
2026-10-01 07:20:35 +02:00
-
663770226e
Find LoRA adapters on Hugging Face and attach them from the mirror page

Ilker ManapandClaude Opus 5.5
2026-10-01 07:19:08 +02:00
-
cf53904cc5
Serve LoRA adapters on llama.cpp models, asked for by name

Ilker ManapandClaude Opus 5.5
2026-09-30 23:40:47 +02:00
-
94eec557ad
Serve a model from an external OpenAI-compatible provider

Ilker ManapandClaude Opus 5.5
2026-09-30 23:29:01 +02:00
-
d5f02c0a4b
Spill a saturated pool's requests to the next pool

Ilker ManapandClaude Opus 5.5
2026-09-30 23:23:32 +02:00
-
3d3c96e422
Displace the newest lowest-band request when the gateway queue is full

Ilker ManapandClaude Opus 5.5
2026-09-30 23:07:53 +02:00
-
623cdb5ba3
Sign releases, and install only signed ones once a key is trusted

Ilker ManapandClaude Opus 5.5
2026-09-30 23:03:48 +02:00
-
561667c9be
Name only the replicas that would make room in headroom

Ilker ManapandClaude Opus 5.5
2026-09-30 23:01:40 +02:00
-
cf0178bf05
Test that a restart borrows a lower class's room and returns it

Ilker ManapandClaude Opus 5.5
2026-09-30 22:47:12 +02:00
-
f288f3c423
Set capacity quotas, class, owner and reservation from the admin pages

Ilker ManapandClaude Opus 5.5
2026-09-30 22:40:45 +02:00
-
5cf0003e74
Release v0.10.5

Ilker ManapandClaude Opus 5.5
2026-09-30 22:26:19 +02:00
-
d10fad064e
Mark facility steps 1-5 deferred; node sensors collected

Ilker ManapandClaude Opus 5.5
2026-09-30 20:25:50 +02:00
-
cd10e086c8
Read nodes' heat and power: GPUs, CPUs, board inlet, RAPL

Ilker ManapandClaude Opus 5.5
2026-09-30 19:49:08 +02:00
-
8724dc027a
Plan facility-aware placement: rack power, temperature and cooling

Ilker ManapandClaude Opus 5.5
2026-09-30 19:25:31 +02:00
-
e7a8105aa4
Log a preempted replica's stop with the grace it had

Ilker ManapandClaude Opus 5.5
2026-09-30 18:41:43 +02:00
-
d0e9813ce9
Of equally idle victims, try the smallest first

Ilker ManapandClaude Opus 5.5
2026-09-30 18:23:08 +02:00
-
0e73f37128
Stop only the replicas that make the room

Ilker ManapandClaude Opus 5.5
2026-09-30 18:21:58 +02:00
-
78e41460f9
Scale and place by what a replica was measured to serve

Ilker ManapandClaude Opus 5.5
2026-09-30 18:21:08 +02:00
-
9dc1b62fdc
Guarantee and limit capacity per part of the tenancy tree

Ilker ManapandClaude Opus 5.5
2026-09-30 18:15:38 +02:00
-
ac24e204a6
Add capacity classes and preempt lower ones to place a model

Ilker ManapandClaude Opus 5.5
2026-09-30 18:02:59 +02:00
-
990d82d801
Reserve CPU cores and require memory for CPU models

Ilker ManapandClaude Opus 5.5
2026-09-30 17:43:13 +02:00
-
26151ec25a
Format web.go

Ilker ManapandClaude Opus 5.5
2026-09-30 11:02:32 +02:00
-
235bf33dcd
Keep every admin table to one line per record

Ilker ManapandClaude Opus 5.5
2026-09-30 11:02:18 +02:00
-
7099043c75
Keep each held row on one line

Ilker ManapandClaude Opus 5.5
2026-09-30 10:50:02 +02:00
-
a32ea0e5ab
Say on each held row how its model runs and where

Ilker ManapandClaude Opus 5.5
2026-09-30 10:38:49 +02:00
-
d7f1c1d545
Open a held model to its replicas, nodes and room for more

Ilker ManapandClaude Opus 5.5
2026-09-30 10:19:31 +02:00
-
2ae12ae1fe
Split the mirror page into Held, Explore and Fetch tabs

Ilker ManapandClaude Opus 5.5
2026-09-30 08:31:28 +02:00
-
62657f4c2a
Release v0.10.4: CPU threads, tracing, and the rest of the review

Ilker ManapandClaude Opus 5.5
2026-09-30 04:00:35 +02:00
-
adb1b3cf2a
Add replicas under load, up to a model's max, and take them back when quiet

Ilker ManapandClaude Opus 5.5
2026-09-30 03:18:07 +02:00
-
bd12b0909a
Serve a model from the registry with one command

Ilker ManapandClaude Opus 5.5
2026-09-30 03:05:13 +02:00
-
116e75f8cb
Upgrade the cluster with one command, over the agents' own streams

Ilker ManapandClaude Opus 5.5
2026-09-30 02:54:36 +02:00
-
3bf4b7b709
Tell clients how much of their rate limit is left

Ilker ManapandClaude Opus 5.5
2026-09-30 02:42:21 +02:00
-
4a652fdcc1
Size a llama.cpp model's memory from its own shape, not its file size

Ilker ManapandClaude Opus 5.5
2026-09-30 02:40:00 +02:00
-
f8572fb5ee
Follow one request from its id to the replica that served it

Ilker ManapandClaude Opus 5.5
2026-09-30 02:36:45 +02:00
-
91fe24a105
Close what a second security review found still open

Ilker ManapandClaude Opus 5.5
2026-09-30 02:25:07 +02:00
-
9bf8256577
Keep backups where the control plane can see them

Ilker ManapandClaude Opus 5.5
2026-09-30 01:50:33 +02:00
-
1da4a2e1ef
Back up the store and the certificate authority, and alert when it stops

Ilker ManapandClaude Opus 5.5
2026-09-30 01:44:32 +02:00
-
b661bfead8
Wait for what tests check instead of sleeping and hoping

Ilker ManapandClaude Opus 5.5
2026-09-30 01:36:55 +02:00
-
37a80a9b44
Park a model that fails every time it starts, and say so everywhere

Ilker ManapandClaude Opus 5.5
2026-09-30 01:21:57 +02:00
-
5e1b6ecea0
Read a queued change as a success in ctl

Ilker ManapandClaude Opus 5.5
2026-09-30 01:14:12 +02:00
-
5dac1854c2
Publish small fixes as release candidates of the next version

Ilker ManapandClaude Opus 5.5
2026-09-30 01:00:11 +02:00
-
ad6d2d348d
Restart replicas without taking them out of service first

Ilker ManapandClaude Opus 5.5
2026-09-30 00:59:26 +02:00
-
611fddd236
Describe v0.10.3

Ilker ManapandClaude Opus 5.5
2026-09-30 00:38:55 +02:00
-
9853936e0b
Give every engine a key only the router holds

Ilker ManapandClaude Opus 5.5
2026-09-30 00:38:29 +02:00
-
d167686cf1
Stop a browser page, or a model definition, from running code on the nodes

Ilker ManapandClaude Opus 5.5
2026-09-30 00:29:52 +02:00
-
97dc49cbeb
Keep the mirror's management API and its downloads off the network behind it

Ilker ManapandClaude Opus 5.5
2026-09-30 00:27:27 +02:00
-
3a26277c32
Refuse names that leave their directory, on the mirror and the agent

Ilker ManapandClaude Opus 5.5
2026-09-30 00:24:48 +02:00
-
7062645bcb
Describe v0.10.2

Ilker ManapandClaude Opus 5.5
2026-09-29 22:37:28 +02:00
-
bb90a927aa
Print only what must outlive init's screen, and drop bubbles

Ilker ManapandClaude Opus 5.5
2026-09-29 22:37:28 +02:00
-
49c446271c
Describe v0.10.1

Ilker ManapandClaude Opus 5.5
2026-09-29 22:17:33 +02:00
-
b5f412f9df
Put the mirror on its own machine in two commands

Ilker ManapandClaude Opus 5.5
2026-09-29 22:17:24 +02:00
-
a1b5daceb8
Let server init ask for what it needs, and node add take several nodes

Ilker ManapandClaude Opus 5.5
2026-09-29 21:54:18 +02:00
-
7071c67487
Show which requests each node's agent answers

Ilker ManapandClaude Opus 5.5
2026-09-29 21:31:05 +02:00
-
7bb360fc78
Measure two nodes' fabric rail by rail: llmgw ctl fabric-test

Ilker ManapandClaude Opus 5.5
2026-09-29 21:29:23 +02:00
-
af6f8e023e
Describe v0.10.0

Ilker ManapandClaude Opus 5.5
2026-09-29 21:06:42 +02:00
-
590e5b9f3e
Ask a node only for engines it can install; doctor says when AVX2 is hidden

Ilker ManapandClaude Opus 5.5
2026-09-29 21:03:03 +02:00
-
2a16c26575
Add a node with one command: llmgw node add, with a terminal screen

Ilker ManapandClaude Opus 5.5
2026-09-29 19:33:47 +02:00
-
c8da06988f
Add a quick install guide: six steps from empty servers to a model

Ilker ManapandClaude Opus 5.5
2026-09-29 18:13:05 +02:00
-
4c5a2fd6da
Install vLLM on every GPU node that can run it, as llama.cpp is

Ilker ManapandClaude Opus 5.5
2026-09-29 13:15:54 +02:00
-
e751c89937
Serve the project's website at /about, with its fonts in the binary

Ilker ManapandClaude Opus 5.5
2026-09-29 10:44:45 +02:00
-
80e0dd252f
Document what changed since v0.9.2, and fabrics for the sites that have them

Ilker ManapandClaude Opus 5.5
2026-09-29 10:14:58 +02:00
-
f989e67e1a
Connect an editor from the account page, and serve the Responses API

Ilker ManapandClaude Opus 5.5
2026-09-29 10:04:17 +02:00
-
b7e8e1a3a1
Let the chat be stopped, timed, tuned and kept across a reload

Ilker ManapandClaude Opus 5.5
2026-09-29 09:54:23 +02:00
-
0823e5d743
Fit the pages to a phone, and drop an empty Fabrics heading

Ilker ManapandClaude Opus 5.5
2026-09-29 09:24:56 +02:00
-
b18c9c3378
Start a mirror download where it was asked for

Ilker ManapandClaude Opus 5.5
2026-09-29 09:21:21 +02:00
-
4dba2a54ea
Tighten the People page

Ilker ManapandClaude Opus 5.5
2026-09-29 09:18:38 +02:00
-
ac7f5063b6
Give every choice-link one pill, and stop the NVLink page breaking them

Ilker ManapandClaude Opus 5.5
2026-09-29 09:15:44 +02:00
-
45e3f52da3
Launch multi-node vLLM its own way by default, not through Ray

Ilker ManapandClaude Opus 5.5
2026-09-29 09:13:59 +02:00
-
20479a2d5f
Find multi-node NVLink domains, place across them, and show them as racks

Ilker ManapandClaude Opus 5.5
2026-09-29 09:07:59 +02:00
-
5bbc619a31
Map every InfiniBand fabric, draw it, and place replicas by it

Ilker ManapandClaude Opus 5.5
2026-09-29 08:46:23 +02:00