CHANGELOG and BACKLOG for the rest of C24 and the engine fixes; README

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-05 01:30:01 +02:00
1 parent 33334c8f33
commit 803d1d5fb6
8 files changed
+227 -206

No files matched your search

+59 -2
View File
@@ -679,11 +679,11 @@ portal is not done — the page shows; the console and ctl change.
**More engines (M each).** TensorRT-LLM/Triton adapter; Ollama/GGUF
import. `agent/engines/engine.go` interface and the metric contract in
`router/scrape`.
24. Lower: precise KV-index routing (plan phase 6), LoRA affinity for
24. **Done (branch `features-c2`), all six parts.** Lower: precise KV-index routing (plan phase 6), LoRA affinity for
vLLM, multi-cluster federation via External providers, facility-aware
placement (`docs/facility-aware-placement.md`), portal i18n, SDK/docs
portal.
Done so far (branch `features-c2`):
What was built, and what is left:
- 24 (portal i18n): built — English/Turkish catalogue with fallback, per-
language templates, cookie/Accept-Language/`landing.language` choice
and a footer switcher, Turkish numbers and dates, every template and
@@ -701,6 +701,63 @@ portal is not done — the page shows; the console and ctl change.
console's generated "Commands" page is not in the portal (it is built
from the cobra tree, which internal/web cannot see); no search across
documents.
- [~] 24. Precise KV index (tier 2) — built: vLLM KV events via the agent
(loopback ZMQ, pure-Go zmq4, msgpack in both of vLLM's encodings)
forwarded as net block-hash changes over the agent stream; router
index, tokenize via the replica's `/tokenize`, sha256_cbor block-hash
chain checked against cbor2/hashlib golden values; scorer by leading-
block share with tier 1 fallback; opt-in `kv_events` per profile;
metrics; docs. Left: validation against a real vLLM (event shape, hash
agreement, `/tokenize` matching the served prompt, cost of
tokenizing); vLLM's replay endpoint instead of resetting on a gap;
data-parallel ranks (one publisher each, ports offset); hybrid models'
cache groups and CPU-offloaded blocks; LoRA/multimodal/cache_salt
requests (extra keys); a router-only deployment (no agents)
subscribing directly; standby control planes have no index until they
lead; a per-replica fallback when only some candidates have an index;
caching tokenizations of repeated prompts.
- **24 LoRA affinity (branch `features-c2`):** built `max_loaded_adapters`
(catalog), vLLM adapter support at start and on demand
(`engines/lora.go`, `vllm.go`), the agent linking fetched adapters as
`llmgw-lora/<name>` in the engines' working directory, the router's
adapter tracker (`router/lora.go`: loads/unloads with the engine key,
LRU eviction never of an adapter in use, one load per adapter per
replica, `/v1/models` sync, 404 reload) and the `adapter-affinity`
scheduler filter in the chat, batch and long-context profiles, with
metrics and docs; tested against a fake of vLLM's HTTP interface.
**Left:** never run against a real vLLM — in particular that vLLM
resolves the relative `llmgw-lora/<name>` path against its working
directory on every node (multi-node too) and that unloading an adapter
mid-request is harmless; the agent still fetches every adapter at
replica start, so a new adapter needs a restart (fetching on first
load would need an agent command and a proto change); load timeout and
resync interval are constants, not config; SGLang's runtime LoRA API
is not driven; no per-replica "adapters held" view in the console or
admin API; the debug profile has no affinity.
- 24. Multi-cluster federation — built: peers via `ctl federation` kept in
the store, their models as PEER/MODEL synced from /v1/models
(withdrawn-while-referenced), /ready health with latency and routing
withdrawal, `ctl model fallbacks` for overflow, hop header loop
refusal, residency/no_store hold, optional attribution tags, console
view, metrics, docs. Left: server.yaml-declared models cannot name a
peer model as a fallback (config catalog is validated before peers are
read); per-peer CA / mTLS for peers behind a private CA; transitive
federation (A→B→C models) deliberately not offered; latency is /ready
RTT only (no per-request peer TTFT used in routing or weighting); the
peer's adapters are not imported (listed as aliases); no console form
to add/remove peers (CLI only); key file must exist on every control
plane.
- 24 (facility) — built: location labels (`ctl label`, agent.yaml),
spreading across racks/PDUs, `keep_within`, held domains with drain
(`ctl facility hold/release`, kept in the store), power caps from
`facility.budgets` against declared/derived/measured draw, shown in
explain-placement, `ctl nodes`, `ctl facility`, the console and the
API. Left: temperature conditions with hysteresis and their
enforcement (steps 3–4), a Prometheus/DCIM source, power headroom as a
rack-levelling score, spreading multi-node replicas across racks,
`llmgw_domain_*` metrics and alerts (near cap, over cap, stale data),
shedding and tokens per watt (step 5), and a drained node's state
surviving a restart.
---