Fixes from review: the similar cache, restore, outside services, latency
CI / test (push) Successful in 9m48s

- The similar-question cache embeds a question as its project would: a
  project kept in the cluster has external fallbacks refused, and an
  embedding model outside the cluster is not asked at all. The vectors are
  compared outside the cache's lock. The question's bound is in characters.
- The restorer puts values back in streamed tool calls' arguments (escaped
  for the JSON they are), completions' text and reasoning, each held across
  events on its own, and releases a held tail with the finish event. A
  whole answer is walked as JSON, so a value with a quote no longer breaks
  tool-call arguments.
- An outside guardrail service reads the user's turn as redacted, and a
  streamed answer once rather than at the finish chunk and again at the
  end. guardrails.md says it can stop a stream only under buffer_all.
- A replica's smoothed time to first token is forgotten after five minutes
  without an answer: one found slow and then given nothing was never tried
  again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-29 08:15:43 +01:00
1 parent fd94265e08
commit 54c6b66cb9
11 files changed
+372 -71

No files matched your search

+9 -4
View File
@@ -301,10 +301,15 @@ policies:
string; empty sends `{"input": …}`). `blocked` and `reason` are dotted paths
into the JSON answer, numbers for list items; true, a number above zero, a
non-empty list or a string other than "false" is unsafe. On input the
service reads the latest user turn; on output the whole answer, once, at its
end — a streamed answer released a sentence at a time is then stopped or
flagged as the judge's is. The rule recorded is `service:<name>:<reason>`.
The service is told the text only, never who sent it.
service reads the latest user turn as the model will, with what `redact`
took out still out; on output the whole answer, once, at its end, as the
client reads it — with `restore`, the values put back. Being at the end, a
service can stop a streamed answer only where it is held back
(`stream: buffer_all`): released a sentence at a time, the client has read
it by then, and what the service finds ends the stream with
`content_filter`, flags it and is recorded. The rule recorded is
`service:<name>:<reason>`. The service is told the text only, never who
sent it.
## Where a policy applies