The guardrails' judge has its own command, and its trouble is shown
llmgw ctl guard judge status lists each judge model the policies ask: in the catalog, as a judge, replicas ready, its last failure to answer, and what is wrong in words. enable serves the missing one through model serve - Llama Guard 3 1B on CPUs, role guard, under the name the policies give it - with --replace for a model of that name that is not a judge; the console has it under Do. GET /api/v1/guard/judges, and the summary's judges, put the problem on ctl status and the console's Overview. A Llama Guard file served from the mirror page is added as a judge, under the name the policies ask, rather than as a model clients use the policies never find. ctl says plainly when the API answers with a page. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
2a942409ea
commit
a8bb417618
24 files changed
+667
-13
No files matched your search
@@ -116,6 +116,13 @@ func (c *ctlClient) do(ctx context.Context, method, path string, body, out any)
|
||||
if out == nil {
|
||||
return nil
|
||||
}
|
||||
// An older control plane answers a path it does not know with its
|
||||
// dashboard: a page, which read as JSON said only "invalid character '<'".
|
||||
if strings.HasPrefix(resp.Header.Get("Content-Type"), "text/html") {
|
||||
return fmt.Errorf("%s %s answered with a page, not the API: the control plane answering "+
|
||||
"(the leader, through a standby) does not know it — is it on an older release? "+
|
||||
"llmgw ctl status says which it runs", method, path)
|
||||
}
|
||||
return json.NewDecoder(resp.Body).Decode(out)
|
||||
}
|
||||
|
||||
|
||||
Reference in new issue
Block a user