Commit Graph
4 Commits
Author SHA1 Message Date
deniz 2eb79f3400 agenta memory eklendi
CI / test (pull_request) Successful in 2m31s
2026-09-24 04:04:03 +03:00
Ilker ManapandClaude Opus 5 d26a30471e Let people forget a conversation, and make retention actually expire things
CI / test (push) Successful in 1m38s
Two halves of keeping the promise that storing makes.

DELETE /v1/threads/{id} is a client endpoint, not an admin one: the person who
said the thing should be able to unsay it, and making them ask an operator
would mean the operator reads it first. Deleting a thread in another project
answers exactly as deleting one that never existed does — two answers would
let a caller map which ids are real in projects they cannot read, and this one
would do it by trying to destroy them.

And something now runs the pruner. Without it "we keep conversations for 24
hours" was a comment rather than a promise: the rows would accumulate and the
number a project chose would describe nothing. Hourly, and once at startup,
because a gateway that has been down for a week should not wait another hour
before honouring a window that expired while it was off.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 11:52:06 +02:00
Ilker ManapandClaude Opus 5 9eb321ae3e Let a conversation continue on a different model than it started on
CI / test (push) Has been cancelled
The request path half. A caller sets x-llmgw-thread and sends only the new
turn; llmgw puts the stored history in front of it, sends that to whichever
model is serving, and stores the exchange afterwards.

A header rather than a body field because some SDKs drop fields they do not
know and all of them let a caller set a header.

Capturing the reply is opt-in and bounded. This is the hottest path in the
system, so a request without a thread accumulates nothing at all, and one with
a thread stops accumulating at a cap rather than growing to it — a gateway
that kept every reply in memory would be one long answer away from trouble. A
reply is truncated on a rune boundary rather than dropped: a shortened turn
replays as a shortened turn, while nothing at all replays as the assistant
having said nothing, and invalid UTF-8 would break the next turn's body for
good rather than for one turn.

The streamed path sets the captured reply on every exit, not only the one that
looks like the end. A stream can also finish with the client gone or the
engine stalled, and what the assistant said up to that point is still what it
said.

A thread on a project that keeps no conversations is refused. The e2e test
asserted that and got a cheerful 200: the request was served, nothing was
stored, and the caller would have discovered it only when the model had no
idea what they were referring to — which reads as the model being broken. It
now asks whether storage is on before doing anything else.

History is spliced after the model is finally settled, because a budget
fallback may have changed it and trimming is measured against the context of
the model that will actually serve.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 11:50:40 +02:00
Ilker ManapandClaude Opus 5 cebf9906d2 Splice a stored conversation into a request, trimming only what must go
CI / test (push) Successful in 1m39s
The part that makes a thread work across models: the history goes in front of
what the caller just said, because a client using a thread sends the new turn
rather than the conversation again.

Trimming has three rules, and each is there because the alternative fails
quietly. The oldest stored turns go first, so a conversation that outgrew the
model now serving it stays usable — after a budget fallback that model was not
the caller's choice, and a permanently unusable thread would be a strange way
to punish them for running out of budget. The caller's own messages are never
dropped: answering something they did not ask is worse than refusing. And a
system message is kept wherever it sits, because losing one silently changes
what the model is for while it carries on answering.

A model that declares no context length has nothing trimmed. Guessing a limit
for it would drop turns for no reason and the caller would never learn why the
model forgot.

The body is rebuilt through a map so every field the client sent survives.
Forwarding fields llmgw does not understand is how engine-specific options
reach an engine at all, and a thread must not be what starts dropping them.
An empty history returns the body byte for byte, so a request without a thread
takes exactly the path it took before this existed.

Not yet wired to the request path.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 11:10:07 +02:00