Guard a GPU-less node's problems, and bound thread ids
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
a56ad8e4c4
commit
f9493aadb9
6 files changed
+64
-1
No files matched your search
@@ -1004,6 +1004,10 @@ curl $BASE/v1/chat/completions \
|
||||
-d '{"model": "<model-name>", "messages": [{"role":"user","content":"and then?"}]}'
|
||||
```
|
||||
|
||||
A thread belongs to the project, not to the key: **any key of the project that
|
||||
knows the id reads the conversation.** Use ids nobody can guess — a UUID, not
|
||||
`my-conversation` — of at most 128 letters, digits and `- _ . :`.
|
||||
|
||||
llmgw puts the stored history in front of that turn and sends it to whichever
|
||||
model is serving — **including a different one from the one the conversation
|
||||
started on**. That is the point: a thread begun on a large model can continue
|
||||
|
||||
Reference in new issue
Block a user