Tick the phase 1 and 1.5 boxes their releases already earned
The code shipped in v0.2.0 and v0.3.0 and both exit criteria were measured; only the checkboxes were left behind. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
f84e3b642a
commit
41cf894c72
1 file changed
+7
-5
+7
-5
@@ -1266,15 +1266,17 @@ work_dir: /var/lib/llmgw-agent
|
||||
- [x] `queue`, `kv`, `inflight` scorers; `max-score` and `p2c` pickers
|
||||
- [x] Session affinity (§7.5 tier 0) — cheap, an early win
|
||||
- [x] Metrics scraping (the §7.6 contract), staleness detection, retry, cooldown, breaker
|
||||
- [ ] Prometheus metrics plus decision metrics (§8)
|
||||
- [x] Prometheus metrics plus decision metrics (§8) — `llmgw_requests_total`,
|
||||
`llmgw_ttft_seconds`, `llmgw_scorer_value`, `llmgw_candidates_considered`,
|
||||
`llmgw_candidates_filtered_total`, `llmgw_tokens_total`
|
||||
- **Exit criterion:** 1000 concurrent streams behind 3 sim-engines; upstream cancellation on
|
||||
client disconnect verified; router overhead p99 < 2 ms.
|
||||
|
||||
### Phase 1.5 — Admission control (1 week) ⭐
|
||||
- [ ] Saturation detector (§7.2.1)
|
||||
- [ ] Priority bands, round-robin tenant fairness within a band, queue capacity and TTL
|
||||
- [ ] `Admission` interface and lease, cancellation while queued
|
||||
- [ ] `llmgw_admission_*` metrics
|
||||
- [x] Saturation detector (§7.2.1)
|
||||
- [x] Priority bands, round-robin tenant fairness within a band, queue capacity and TTL
|
||||
- [x] `Admission` interface and lease, cancellation while queued
|
||||
- [x] `llmgw_admission_*` metrics
|
||||
- **Exit criterion:** while 500 concurrent batch requests flow through the bronze band, the TTFT
|
||||
p99 of interactive requests in the platinum band stays under 1.5× its idle value. **Do not
|
||||
move on before writing this test — it is the entire reason phase 1.5 exists.**
|
||||
|
||||
Reference in new issue
Block a user