A path escaped as it came reaches the leader through a standby; F5 in the backlog
CI / test (push) Successful in 9m30s

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-29 22:54:27 +01:00
1 parent cacaa35c3f
commit 29d7f54cfd
4 files changed
+34 -8

No files matched your search

+13 -5
View File
@@ -35,7 +35,9 @@ What is left, most needed first. The detail is in the item each names.
146's address names was given 152's; what was added meanwhile reached 146 when it came back; a removal was
carried both ways. Found and fixed there: add-peer on a mirror machine of its own, files without checksums never
copied, and `mirror remove` neither recorded nor seen by the running mirror.
6. **F5**: large imports limited by the embedding engine's slots. (M)
6. **F5**: large imports. Done 2026-10-29: the gateway embeds what `llmgw-kb` sends as text; the slots and
halfvec were done before. Left: a larger store machine before a million chunks (the operator's), and the
embedding model on a GPU for a real import.
**From Kong** (section L): L1–L5 and L9 done (v0.15.0); left of L4 is routing by cost between outside
providers, of L5 cache-write and audio prices. L6 and L7 are llmgw-80's; L8 (A2A) when asked.
@@ -1128,13 +1130,19 @@ four texts per pass, while llmgw-kb sends 32 a request from 6 workers. The
usage rows show 4.6 s a request on average, 1.8 s of it in the gateway's
queue and 0.04 s in the guard. The GPU sat at 18–50% and the store machine
at a third of its CPUs.
- An embedding model's profile with many slots by default (`--parallel`
- Done for `ctl model serve` (32 slots of 1024 tokens; `ctl model context
MODEL 1024 --slots 32` resizes one served before): an embedding model's profile with many slots by default (`--parallel`
32–64, `--ctx-size` to match, `--batch-size`/`--ubatch-size` at least the
longest chunk), or `ctl model serve` choosing that for a model with
`capabilities.embeddings`. With the default 512, one 516-token chunk failed.
- Embedding on the gateway's side of a document PUT, so the client sends
text only: today an embeddings answer is ~21 KB of JSON a chunk, and
the text and vector go up again (~11 KB).
- Done (2026-10-29): embedding on the gateway's side of a document PUT.
`llmgw-kb` sends text only to a gateway that says `server_embeds`; the
gateway embeds through its own `/v1/embeddings` as the caller, 32 a batch,
four batches at once, waiting out a busy engine. On 40 DTIC files the
upload went from 25.7 to 3.5 MiB, and the ~21 KB a chunk that came down
is gone; on 6 files both ways gave the same 32 chunks and the same search
answer. Speed is the engine's either way: the PoC's bge-m3 on node02's two
CPU cores embeds ~0.2 chunks/s, so a real import still wants it on a GPU.
- The store next: HNSW inserts took half of one of pg01's two cores; at a
million chunks the vectors alone outgrow its 4 GB. halfvec (pgvector ≥
0.7) and a larger store machine.
+6
View File
@@ -35,6 +35,12 @@ APIs change between minors without a deprecation period.
of a vector down and up again, some thirty times more. `llmgw-kb` does so with a gateway that says it can
(`server_embeds` in a collection's answer); `--embed-here` keeps the old way.
### Fixed
- **A name with a slash in it, asked through a standby control plane.** `llmgw ctl kb delete Demo/hr` on a standby
was answered 405: the standby passed the request on to the leader with `Demo%2Fhr` unescaped, two segments instead
of one. The path now reaches the leader as it was sent.
## v0.15.3 — 2026-10-29
A model stopped and kept, or removed with its files; the second mirror fixed where it was tried.
+4 -2
View File
@@ -73,8 +73,10 @@ func (f *leaderForward) standby(next http.Handler) http.Handler {
p := &httputil.ReverseProxy{
Rewrite: func(pr *httputil.ProxyRequest) {
pr.Out.URL.Scheme, pr.Out.URL.Host = "https", target.Host
// Escaped as it came: a name with a slash in it, Demo/hr
// as Demo%2Fhr, is one segment, not two.
pr.Out.URL.Path = forwardPrefix + pr.In.URL.Path
pr.Out.URL.RawPath = ""
pr.Out.URL.RawPath = forwardPrefix + pr.In.URL.EscapedPath()
pr.Out.Host = target.Host
// The leader's own token is not this one's, and the
// certificate is what it believes.
@@ -221,7 +223,7 @@ func (f *leaderForward) leaderSide(admin http.Handler) http.Handler {
return
}
r2 := r.Clone(r.Context())
r2.URL.Path, r2.URL.RawPath = path, ""
r2.URL.Path, r2.URL.RawPath = path, strings.TrimPrefix(r.URL.EscapedPath(), forwardPrefix)
w.Header().Set(hdrAnsweredBy, f.self)
admin.ServeHTTP(w, r2)
})
+11 -1
View File
@@ -77,7 +77,7 @@ func newForwardCluster(t *testing.T) *forwardCluster {
// The leader's admin API: says whose it is, and who it was asked by.
admin := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_ = json.NewEncoder(w).Encode(map[string]string{"path": r.URL.Path, "by": r.Header.Get(hdrForwardedBy),
_ = json.NewEncoder(w).Encode(map[string]string{"path": r.URL.Path, "raw": r.URL.EscapedPath(), "by": r.Header.Get(hdrForwardedBy),
"auth": r.Header.Get("Authorization"), "actor": r.Header.Get("X-Llmgw-Actor")})
})
mux := http.NewServeMux()
@@ -242,3 +242,13 @@ func TestAStandbysPortalIsTheLeaders(t *testing.T) {
t.Errorf("a forwarded request was forwarded again: %q", body)
}
}
// A name with a slash in it, escaped, stays one segment on the leader: a
// collection Demo/hr removed through a standby was a 405 there.
func TestAnEscapedSlashReachesTheLeaderEscaped(t *testing.T) {
c := newForwardCluster(t)
resp, got := c.ask(t, "/api/v1/kb/collections/Demo%2Fhr")
if resp.StatusCode != http.StatusOK || got["raw"] != "/api/v1/kb/collections/Demo%2Fhr" {
t.Fatalf("%d %v", resp.StatusCode, got)
}
}