Files
llmgw/docs.go
T
Ilker ManapandClaude Opus 5.5 f0477f140d Quality suites, and new weights held to one: ctl eval, model update/promote --eval, model gate
A suite is a set of cases in YAML or JSON lines, each scored by exact,
contains, regex, a number within a tolerance, a JSON Schema subset, or a
judge model with a rubric; weighed into one score against a threshold.
internal/bench/eval runs one on the control plane through Router.Replay, as
the gateway's own requests, and keeps every run and its cases in the store
(migration 0028: eval_runs, eval_cases).

- llmgw ctl eval MODEL --suite FILE|NAME, --history, --run, --suites; exits
  non-zero below the score. Suites by name from control.eval_dir, or the
  two built in from deploy/eval (arithmetic, json-format).
- model update and promote take --eval SUITE --min-score N, or a model's
  standing gate (ctl model gate, eval: in the catalog): a promote asks the
  canary first and leaves it below the score; an update tries the new
  weights as a canary given no share, then promotes or removes it.
- The console's model menu: ask a suite with its progress, its runs, its
  gate.
- The audit middleware no longer hands a handler a body cut off at 64 KiB.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 00:04:43 +02:00

18 lines
542 B
Go

// Package llmgw holds what the binary carries besides code: its manual,
// and the example quality suites.
package llmgw
import "embed"
// Docs is the documentation, read on any machine by `llmgw docs` and the
// console's manual without a checkout or the internet.
//
//go:embed README.md CHANGELOG.md docs/*.md
var Docs embed.FS
// EvalSuites are the example quality suites under deploy/eval, which the
// control plane runs by name when its eval directory has none of the name.
//
//go:embed deploy/eval/*.yaml
var EvalSuites embed.FS