Merge C21 quality suites and gating
Conflicts: docs.go (manual, OpenAPI and suites all kept), modelserve Job (trace and eval both kept). The eval routes are added to the OpenAPI route table, update and promote documented with their optional body, and the document regenerated. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
commit
d41104a452
51 files changed
+5204
-49
No files matched your search
@@ -65,6 +65,7 @@ rather than repeated.
|
||||
| `port_range` | `[]int` | [Control](#control) |
|
||||
| `mgmt_interface` | `string` | [Control](#control) |
|
||||
| `timezone` | `string` | [Control](#control) |
|
||||
| `eval_dir` | `string` | [Control](#control) |
|
||||
| `enabled` | `bool` | [Control](#control) |
|
||||
| `lease_ttl` | `time.Duration` | [Control](#control) |
|
||||
| `id` | `string` | [Endpoint](#endpoint) |
|
||||
@@ -79,6 +80,9 @@ rather than repeated.
|
||||
| `nodes` | `[]string` | [EngineOverride](#engineoverride) |
|
||||
| `labels` | `map[string]string` | [EngineOverride](#engineoverride) |
|
||||
| `engines` | `map[string]string` | [EngineOverride](#engineoverride) |
|
||||
| `suite` | `string` | [EvalGate](#evalgate) |
|
||||
| `min_score` | `float64` | [EvalGate](#evalgate) |
|
||||
| `judge` | `string` | [EvalGate](#evalgate) |
|
||||
| `url` | `string` | [External](#external) |
|
||||
| `model` | `string` | [External](#external) |
|
||||
| `api_key_file` | `string` | [External](#external) |
|
||||
@@ -112,6 +116,7 @@ rather than repeated.
|
||||
| `adapters` | `[]Adapter` | [Model](#model) |
|
||||
| `fallbacks` | `[]string` | [Model](#model) |
|
||||
| `canary` | `*Canary` | [Model](#model) |
|
||||
| `eval` | `*EvalGate` | [Model](#model) |
|
||||
| `routing_profile` | `string` | [Model](#model) |
|
||||
| `slo` | `SLO` | [Model](#model) |
|
||||
| `context_length` | `int` | [Model](#model) |
|
||||
@@ -688,6 +693,15 @@ MgmtInterface is the ethernet interface NCCL bootstraps over.
|
||||
Timezone is where models' scaling schedules are read, an IANA name
|
||||
like Europe/Istanbul; empty is the control plane's own.
|
||||
|
||||
### `eval_dir`
|
||||
|
||||
`string`
|
||||
|
||||
EvalDir is where quality suites are read from by name, for `llmgw ctl
|
||||
eval MODEL --suite NAME`, a model's eval gate and the console. The
|
||||
suites the binary carries (arithmetic, json-format) are found when it
|
||||
has none of the name; a suite here of the same name replaces one.
|
||||
|
||||
### `enabled`
|
||||
|
||||
`bool`
|
||||
@@ -781,6 +795,28 @@ Labels picks nodes carrying all of these.
|
||||
|
||||
Engines are laid over the cluster's: a version, or "" for none.
|
||||
|
||||
## EvalGate
|
||||
|
||||
### `suite`
|
||||
|
||||
`string`
|
||||
|
||||
Suite is a suite's name in control.eval_dir, or one the binary
|
||||
carries (arithmetic, json-format).
|
||||
|
||||
### `min_score`
|
||||
|
||||
`float64`
|
||||
|
||||
MinScore is the score, 0 to 1, to pass at; zero is the suite's own
|
||||
threshold.
|
||||
|
||||
### `judge`
|
||||
|
||||
`string`
|
||||
|
||||
Judge grades the suite's rubric cases, instead of the judge it names.
|
||||
|
||||
## External
|
||||
|
||||
### `url`
|
||||
@@ -1058,6 +1094,16 @@ moving to it (`llmgw ctl model canary`).
|
||||
|
||||
Takes a [Canary](#canary) block.
|
||||
|
||||
### `eval`
|
||||
|
||||
`*EvalGate`
|
||||
|
||||
Eval is a quality suite new weights must pass before they replace
|
||||
this model's: `llmgw ctl model update` and `model promote` run it
|
||||
against them first, and stop below its score.
|
||||
|
||||
Takes a [EvalGate](#evalgate) block.
|
||||
|
||||
### `routing_profile`
|
||||
|
||||
`string`
|
||||
|
||||
Reference in new issue
Block a user