Merge admission-bulk into jobs-3d

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-10 17:35:39 +02:00
commit ec233d3586
20 files changed
+328 -40

No files matched your search

+21
View File
@@ -1038,6 +1038,27 @@ qwen2.5-coder-3b-instruct yes 1/1 ready coder
No restart, and it survives one: models added this way are stored, while
anything in `server.yaml` is declarative and returns on every start.
A profile may load weights of its own. An embedding model, for example, can
be served by vLLM from its repository on a card and by llama.cpp from a GGUF
on CPUs, as one model:
```yaml
name: qwen3-embed-4b
source: Qwen/Qwen3-Embedding-4B # what a profile loads unless it says
capabilities: { embeddings: true }
profiles:
- name: gpu
engine: vllm
gpu_class: 6000-24g
args: ["--runner", "pooling"]
- name: cpu
engine: llamacpp
source: hf:Qwen/Qwen3-Embedding-4B-GGUF/Qwen3-Embedding-4B-Q8_0.gguf
```
Updates and canaries move the model's `source`; a profile with its own keeps
it.
### Models that load when someone asks
A cluster with more models than GPUs can hold every model in its catalog and