## CHANGELOG ### LoRA adapters on vLLM, loaded on demand - **vLLM models can carry adapters.** A PEFT adapter directory (a mirror version ending in `/`, a path on the nodes, or a Hugging Face repository id) is registered at start with `--enable-lora --lora-modules`, and a request names it as its model, as on llama.cpp. `scale` applies on llama.cpp only. SGLang and TensorRT-LLM profiles still refuse adapters. - **`max_loaded_adapters: N` on a model** has each replica hold at most N of its adapters. Replicas start holding none (vLLM with `VLLM_ALLOW_RUNTIME_LORA_UPDATING=True` and `--max-loras N`, unless the profile's args set their own). The router loads an adapter on a replica through `/v1/load_lora_adapter` when a request first wants it there, and unloads the least recently used one no request is running on to make room. vLLM profiles only; llama.cpp keeps loading every adapter at start, since llama-server can rescale an adapter it started with but not load another. - **Adapter affinity:** a request for an adapter goes to a replica that has it, or is loading it, while that replica is not saturated; otherwise to one with room for another adapter; otherwise where load sends it — which is how a busy adapter spreads to a second replica. Requests arriving during a load wait for it instead of loading the adapter again. - **Self-correcting:** each replica's `/v1/models` is read when the router first meets it and every 30 seconds, so adapters loaded before a restart or by another router are known; a 404 for an adapter the router thought loaded loads it again and retries. An adapter vLLM refuses (not found, rank above `--max-lora-rank`) fails the request with vLLM's reason, without trying it on every replica. - **Metrics:** `llmgw_adapter_affinity_total{model,result=hit|joined|loaded}`, `llmgw_adapter_loads_total{model,outcome}`, `llmgw_adapter_evictions_total{model,outcome}`. - The admin API and `ctl model adapters` read the file as GGUF only for a model with llama.cpp profiles. ## BACKLOG **24 LoRA affinity (branch `features-c2`):** built `max_loaded_adapters` (catalog), vLLM adapter support at start and on demand (`engines/lora.go`, `vllm.go`), the agent linking fetched adapters as `llmgw-lora/` in the engines' working directory, the router's adapter tracker (`router/lora.go`: loads/unloads with the engine key, LRU eviction never of an adapter in use, one load per adapter per replica, `/v1/models` sync, 404 reload) and the `adapter-affinity` scheduler filter in the chat, batch and long-context profiles, with metrics and docs; tested against a fake of vLLM's HTTP interface. **Left:** never run against a real vLLM — in particular that vLLM resolves the relative `llmgw-lora/` path against its working directory on every node (multi-node too) and that unloading an adapter mid-request is harmless; the agent still fetches every adapter at replica start, so a new adapter needs a restart (fetching on first load would need an agent command and a proto change); load timeout and resync interval are constants, not config; SGLang's runtime LoRA API is not driven; no per-replica "adapters held" view in the console or admin API; the debug profile has no affinity.