## CHANGELOG ### Engines and the mirror - **TensorRT-LLM as an engine** (`engine: trtllm`), on nodes where you installed it: name its interpreter in the node's `agent.yaml` (`engines.trtllm.python`) and the node offers it while that interpreter has the `tensorrt_llm` package. llmgw does not install it. The agent starts `trtllm-serve serve` with the replica's host, port, `--tp_size`, `--pp_size` and served name, through a few lines of its own that make it demand the gateway's key on every path but `/health` (trtllm-serve has no key option), turn on the Prometheus endpoint and iteration statistics, and keep usage reports to NVIDIA off. The router reads it at `/prometheus/metrics` — `trtllm_num_requests_running`, `trtllm_num_requests_waiting`, `trtllm_kv_cache_utilization`, tokens and successes. Needs TensorRT-LLM 1.3 (rc13 or later) and an Ampere or newer GPU; one node per replica. Profiles may not give it an options file, `--trust_remote_code`, `--grpc`, `--num_serve_frontends` or the flags the agent sets; `--set PATH=VALUE` changes a setting instead. `llmgw ctl engine-check trtllm --bin ` checks an installed one. Written against TensorRT-LLM's documented command line, server and metrics and tested with stand-ins for them; not yet run on a datacenter GPU. - **`llmgw mirror import-ollama [NAME[:TAG]]`** puts a model you already have in Ollama into the mirror for llama.cpp: its GGUF, and a vision model's projector beside it, each checked against the sha256 Ollama named its file by. With no name it lists Ollama's models; it finds Ollama's directory itself (`OLLAMA_MODELS`, `~/.ollama/models`, Ollama's Linux service) or takes `--ollama-dir`. The version is the tag and the start of the digest, since a tag moves in Ollama. It prints a model definition to add, with Ollama's `num_ctx` as `--ctx-size`. The console's **Mirror** view does the same with **I**, for the mirror beside the control plane. - **`llmgw mirror import FILE... --model NAME`** copies files already on the mirror machine into it the way a download is kept: whole or not at all, the checksum beside it, a version never replaced, a `.gguf` checked to be one, and `--sha256` checked when given. ## BACKLOG **Done (branch `features-c2`):** engine `trtllm` (`internal/agent/engines/trtllm.go`): `trtllm-serve` through a launcher of llmgw's that demands the replica's key (trtllm-serve has none) and writes the options file that turns on `/prometheus/metrics` and the iteration statistics; the scrape contract for it (`scrape.TRTLLMMapping`, with a per-engine metrics path); found, not installed — `engines.trtllm.python` in `agent.yaml`, probed from the package's metadata; catalog validation (GPU class, one node, no GGUF source, the flags a profile may not give); `ctl engine-check trtllm`, which asks `trtllm-serve serve --help` through its module. Ollama/GGUF import: `mirror import-ollama` (manifest parsing, blob digest check, projector beside the model, all or nothing, a model definition printed), `mirror import` for any local file, and the console's **I** on the Mirror view. Left: TensorRT-LLM has not run on real hardware — the first node with one should run `engine-check` and serve a model before anyone relies on it; no installer for it; no multi-node TensorRT-LLM replica (it needs an MPI or Slurm launch from one node); Triton Inference Server as an engine is not done; trtllm's queue gauges are set per iteration, so an idle engine shows its last batch until the next request; `--num_serve_frontends` is refused because extra front ends would not demand the key. The Ollama import runs on the mirror machine only — the mirror's API takes no files, so an Ollama elsewhere means copying the blob or running the import there; Ollama models kept as separate tensors or as several weight layers, and its LoRA adapters, are refused or left with a sentence; files an import run with `sudo` writes are root's, as `mirror add`'s are.