A served model's context can be changed, engine and memory with it
CI / test (push) Successful in 12m0s
CI / test (push) Successful in 12m0s
ctl model context MODEL TOKENS [--slots N], POST /api/v1/models/{name}/context
and the console's Context length…: the catalog, the engine's arguments and
the KV memory change together, refused past the trained context or what the
cards hold, then the replicas restart one by one.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
1908bfb4b2
commit
986981ac81
14 files changed
+644
-1
No files matched your search
@@ -213,6 +213,8 @@ var Routes = []Route{
|
||||
{Method: "POST", Path: "/api/v1/models/{name}/update", Summary: "Move a model to its repository's newer weights",
|
||||
Description: "With a quality suite, or the model's standing gate, the new weights are asked it first as a hidden canary.",
|
||||
Request: modelserve.Change{}, OptionalBody: true, Response: modelserve.Job{}, Status: http.StatusAccepted},
|
||||
{Method: "POST", Path: "/api/v1/models/{name}/context", Summary: "Change a served model's context: the catalog, its engines' arguments and memory, then its replicas restarted",
|
||||
Request: modelserve.ContextRequest{}, Response: modelserve.ContextChange{}},
|
||||
{Method: "POST", Path: "/api/v1/models/{name}/rollback", Summary: "Go back to the weights the last update replaced",
|
||||
Response: Result{}},
|
||||
{Method: "POST", Path: "/api/v1/models/{name}/origin", Summary: "Record the repository a model's weights came from",
|
||||
|
||||
Reference in new issue
Block a user