Fixes from review: the similar cache, restore, outside services, latency
CI / test (push) Successful in 9m48s
CI / test (push) Successful in 9m48s
- The similar-question cache embeds a question as its project would: a project kept in the cluster has external fallbacks refused, and an embedding model outside the cluster is not asked at all. The vectors are compared outside the cache's lock. The question's bound is in characters. - The restorer puts values back in streamed tool calls' arguments (escaped for the JSON they are), completions' text and reasoning, each held across events on its own, and releases a held tail with the finish event. A whole answer is walked as JSON, so a value with a quote no longer breaks tool-call arguments. - An outside guardrail service reads the user's turn as redacted, and a streamed answer once rather than at the finish chunk and again at the end. guardrails.md says it can stop a stream only under buffer_all. - A replica's smoothed time to first token is forgotten after five minutes without an answer: one found slow and then given nothing was never tried again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
fd94265e08
commit
54c6b66cb9
11 files changed
+372
-71
No files matched your search
+9
-4
@@ -301,10 +301,15 @@ policies:
|
||||
string; empty sends `{"input": …}`). `blocked` and `reason` are dotted paths
|
||||
into the JSON answer, numbers for list items; true, a number above zero, a
|
||||
non-empty list or a string other than "false" is unsafe. On input the
|
||||
service reads the latest user turn; on output the whole answer, once, at its
|
||||
end — a streamed answer released a sentence at a time is then stopped or
|
||||
flagged as the judge's is. The rule recorded is `service:<name>:<reason>`.
|
||||
The service is told the text only, never who sent it.
|
||||
service reads the latest user turn as the model will, with what `redact`
|
||||
took out still out; on output the whole answer, once, at its end, as the
|
||||
client reads it — with `restore`, the values put back. Being at the end, a
|
||||
service can stop a streamed answer only where it is held back
|
||||
(`stream: buffer_all`): released a sentence at a time, the client has read
|
||||
it by then, and what the service finds ends the stream with
|
||||
`content_filter`, flags it and is recorded. The rule recorded is
|
||||
`service:<name>:<reason>`. The service is told the text only, never who
|
||||
sent it.
|
||||
|
||||
## Where a policy applies
|
||||
|
||||
|
||||
Reference in new issue
Block a user