A path escaped as it came reaches the leader through a standby; F5 in the backlog
CI / test (push) Successful in 9m30s
CI / test (push) Successful in 9m30s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
cacaa35c3f
commit
29d7f54cfd
4 files changed
+34
-8
No files matched your search
+13
-5
@@ -35,7 +35,9 @@ What is left, most needed first. The detail is in the item each names.
|
||||
146's address names was given 152's; what was added meanwhile reached 146 when it came back; a removal was
|
||||
carried both ways. Found and fixed there: add-peer on a mirror machine of its own, files without checksums never
|
||||
copied, and `mirror remove` neither recorded nor seen by the running mirror.
|
||||
6. **F5**: large imports limited by the embedding engine's slots. (M)
|
||||
6. **F5**: large imports. Done 2026-10-29: the gateway embeds what `llmgw-kb` sends as text; the slots and
|
||||
halfvec were done before. Left: a larger store machine before a million chunks (the operator's), and the
|
||||
embedding model on a GPU for a real import.
|
||||
|
||||
**From Kong** (section L): L1–L5 and L9 done (v0.15.0); left of L4 is routing by cost between outside
|
||||
providers, of L5 cache-write and audio prices. L6 and L7 are llmgw-80's; L8 (A2A) when asked.
|
||||
@@ -1128,13 +1130,19 @@ four texts per pass, while llmgw-kb sends 32 a request from 6 workers. The
|
||||
usage rows show 4.6 s a request on average, 1.8 s of it in the gateway's
|
||||
queue and 0.04 s in the guard. The GPU sat at 18–50% and the store machine
|
||||
at a third of its CPUs.
|
||||
- An embedding model's profile with many slots by default (`--parallel`
|
||||
- Done for `ctl model serve` (32 slots of 1024 tokens; `ctl model context
|
||||
MODEL 1024 --slots 32` resizes one served before): an embedding model's profile with many slots by default (`--parallel`
|
||||
32–64, `--ctx-size` to match, `--batch-size`/`--ubatch-size` at least the
|
||||
longest chunk), or `ctl model serve` choosing that for a model with
|
||||
`capabilities.embeddings`. With the default 512, one 516-token chunk failed.
|
||||
- Embedding on the gateway's side of a document PUT, so the client sends
|
||||
text only: today an embeddings answer is ~21 KB of JSON a chunk, and
|
||||
the text and vector go up again (~11 KB).
|
||||
- Done (2026-10-29): embedding on the gateway's side of a document PUT.
|
||||
`llmgw-kb` sends text only to a gateway that says `server_embeds`; the
|
||||
gateway embeds through its own `/v1/embeddings` as the caller, 32 a batch,
|
||||
four batches at once, waiting out a busy engine. On 40 DTIC files the
|
||||
upload went from 25.7 to 3.5 MiB, and the ~21 KB a chunk that came down
|
||||
is gone; on 6 files both ways gave the same 32 chunks and the same search
|
||||
answer. Speed is the engine's either way: the PoC's bge-m3 on node02's two
|
||||
CPU cores embeds ~0.2 chunks/s, so a real import still wants it on a GPU.
|
||||
- The store next: HNSW inserts took half of one of pg01's two cores; at a
|
||||
million chunks the vectors alone outgrow its 4 GB. halfvec (pgvector ≥
|
||||
0.7) and a larger store machine.
|
||||
|
||||
@@ -35,6 +35,12 @@ APIs change between minors without a deprecation period.
|
||||
of a vector down and up again, some thirty times more. `llmgw-kb` does so with a gateway that says it can
|
||||
(`server_embeds` in a collection's answer); `--embed-here` keeps the old way.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **A name with a slash in it, asked through a standby control plane.** `llmgw ctl kb delete Demo/hr` on a standby
|
||||
was answered 405: the standby passed the request on to the leader with `Demo%2Fhr` unescaped, two segments instead
|
||||
of one. The path now reaches the leader as it was sent.
|
||||
|
||||
## v0.15.3 — 2026-10-29
|
||||
|
||||
A model stopped and kept, or removed with its files; the second mirror fixed where it was tried.
|
||||
|
||||
@@ -73,8 +73,10 @@ func (f *leaderForward) standby(next http.Handler) http.Handler {
|
||||
p := &httputil.ReverseProxy{
|
||||
Rewrite: func(pr *httputil.ProxyRequest) {
|
||||
pr.Out.URL.Scheme, pr.Out.URL.Host = "https", target.Host
|
||||
// Escaped as it came: a name with a slash in it, Demo/hr
|
||||
// as Demo%2Fhr, is one segment, not two.
|
||||
pr.Out.URL.Path = forwardPrefix + pr.In.URL.Path
|
||||
pr.Out.URL.RawPath = ""
|
||||
pr.Out.URL.RawPath = forwardPrefix + pr.In.URL.EscapedPath()
|
||||
pr.Out.Host = target.Host
|
||||
// The leader's own token is not this one's, and the
|
||||
// certificate is what it believes.
|
||||
@@ -221,7 +223,7 @@ func (f *leaderForward) leaderSide(admin http.Handler) http.Handler {
|
||||
return
|
||||
}
|
||||
r2 := r.Clone(r.Context())
|
||||
r2.URL.Path, r2.URL.RawPath = path, ""
|
||||
r2.URL.Path, r2.URL.RawPath = path, strings.TrimPrefix(r.URL.EscapedPath(), forwardPrefix)
|
||||
w.Header().Set(hdrAnsweredBy, f.self)
|
||||
admin.ServeHTTP(w, r2)
|
||||
})
|
||||
|
||||
@@ -77,7 +77,7 @@ func newForwardCluster(t *testing.T) *forwardCluster {
|
||||
|
||||
// The leader's admin API: says whose it is, and who it was asked by.
|
||||
admin := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
_ = json.NewEncoder(w).Encode(map[string]string{"path": r.URL.Path, "by": r.Header.Get(hdrForwardedBy),
|
||||
_ = json.NewEncoder(w).Encode(map[string]string{"path": r.URL.Path, "raw": r.URL.EscapedPath(), "by": r.Header.Get(hdrForwardedBy),
|
||||
"auth": r.Header.Get("Authorization"), "actor": r.Header.Get("X-Llmgw-Actor")})
|
||||
})
|
||||
mux := http.NewServeMux()
|
||||
@@ -242,3 +242,13 @@ func TestAStandbysPortalIsTheLeaders(t *testing.T) {
|
||||
t.Errorf("a forwarded request was forwarded again: %q", body)
|
||||
}
|
||||
}
|
||||
|
||||
// A name with a slash in it, escaped, stays one segment on the leader: a
|
||||
// collection Demo/hr removed through a standby was a 405 there.
|
||||
func TestAnEscapedSlashReachesTheLeaderEscaped(t *testing.T) {
|
||||
c := newForwardCluster(t)
|
||||
resp, got := c.ask(t, "/api/v1/kb/collections/Demo%2Fhr")
|
||||
if resp.StatusCode != http.StatusOK || got["raw"] != "/api/v1/kb/collections/Demo%2Fhr" {
|
||||
t.Fatalf("%d %v", resp.StatusCode, got)
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user