The plan's phase 5 says what is built, what is built but unproven, and what is not

The precise KV index and vLLM's LoRA affinity were built in v0.11.2 and
never run against a real vLLM; facility-aware placement in v0.11.2 less
temperature and cooling: marked [~], a mark now explained at the top of
the roadmap. Control-plane HA is done, by leader election on PostgreSQL
rather than Raft, and the admin UI changes things since staff roles
(v0.11.2) and the settings page (v0.14.1).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-10 19:35:15 +02:00
1 parent cf41adbf25
commit 41601bc5ef
1 file changed
+21 -8
+21 -8
View File
@@ -1286,6 +1286,9 @@ work_dir: /var/lib/llmgw-agent
## 11. Roadmap
`[x]` is built and shown to work, `[ ]` is not built, and `[~]` is built
but not yet run where it matters — a real engine, real hardware.
### Phase 0 — Skeleton (1 week)
- [x] `go mod init`, cobra commands, koanf config, slog, Makefile, golangci-lint
- [x] `sim-engine`: OpenAI-compatible fake engine (tunable latency, token rate, queue, vLLM-format `/metrics`)
@@ -1476,7 +1479,10 @@ The launch path is now built too:
`docs/reference.md` generated from the configuration structs
### Phase 6 — v2 ideas
- [ ] Precise KV index (§7.5 tier 2) — vLLM KV events plus tokenizer integration
- [~] Precise KV index (§7.5 tier 2) — vLLM KV events plus tokenizer integration.
Built in v0.11.2 (`kv_events` on a profile, the agent forwarding block
changes, the router's index and scorer); never run against a real vLLM,
which is what is left (BACKLOG C24)
- [ ] Dynamo-style block cost function, tuning the `softmax` picker
- [x] Admission displacement: a total queue bound, and the newest request of
the lowest band turned away (503, Retry-After) for a higher one
@@ -1489,17 +1495,23 @@ The launch path is now built too:
- [x] CPU cores reserved like memory, and memory required of CPU models
- [x] Node sensors collected: GPU temperature, power and throttling, CPU
temperature and RAPL power, board inlet air; published and shown
- [ ] Rack power, temperature and cooling as placement signals: planned in
[docs/facility-aware-placement.md](docs/facility-aware-placement.md),
deferred — it needs each site's rack, PDU and power data first
- [~] Rack power, temperature and cooling as placement signals: planned in
[docs/facility-aware-placement.md](docs/facility-aware-placement.md).
Built in v0.11.2: location labels, spreading across racks and PDUs,
`keep_within`, held domains, power caps from `facility.budgets`. Left:
temperature and cooling, which need each site's sensor data
- [ ] Defragmentation for large tensor-parallel replicas across NVLink domains
- [ ] Control plane HA (Raft)
- [x] Control plane HA, by leader election on PostgreSQL (v0.8.0) and from
setup (v0.11.0) rather than by Raft: the store is replicated already, so
a second consensus layer would only have to agree with it
- [x] Weight downloading and cache management: the mirror, agent fetches
checked against their digest, and reclaiming what nothing uses
- [ ] Prefill/decode disaggregation (§12)
- [x] LoRA adapters on llama.cpp: every replica loads a model's adapters and
applies one per request, so any replica serves any adapter
- [ ] LoRA affinity for engines that load adapters on demand (vLLM)
- [~] LoRA affinity for engines that load adapters on demand (vLLM): built in
v0.11.2 (`max_loaded_adapters`, the router's adapter tracker, the
affinity filter); never run against a real vLLM
- [x] External provider support: a catalog model served by an OpenAI-compatible
API elsewhere, usable as a fallback, under the same allowlists and budgets
- [x] Read-only dashboard on the admin listener (nodes, replicas, models,
@@ -1511,8 +1523,9 @@ The launch path is now built too:
promoting, resetting passwords and issuing keys stay on the CLI
- [x] The dashboard reachable over the web at `/admin`, behind a session and an
admin flag, without exposing the listener that carries pprof
- [ ] An admin UI that can *change* things, which needs roles this deliberately
does not have
- [x] An admin UI that can *change* things: staff roles (viewer, operator,
billing, admin) and the audit log in v0.11.2, the cluster's settings on
the portal in v0.14.1
---