Ilker ManapandClaude Opus 5
afb3797e57
The page described nginx and a curl loop, which was accurate when the mirror was a directory somebody kept by hand. It is a subcommand now, so it says so: the two listeners and why they are two, the API and what each endpoint answers, and the certificates coming from the authority the fleet already has. nginx stays, further down. It serves a directory perfectly well and anyone already running it should not be told to install something to replace it. The systemd unit spells out --base in a comment because it is the flag that goes wrong quietly: it ends up in catalog entries, and a wrong one makes `mirror unused` call files unreferenced while they are in use. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
systemd units
Two roles, two units. Copy the one you need to /etc/systemd/system/, put the
matching configuration in /etc/llmgw/, and systemctl enable --now it.
The control plane — llmgw-server.service
Runs unprivileged. It needs to read its own TLS key, so that file must belong to
the llmgw user and nothing else:
useradd --system --home-dir /var/lib/llmgw --shell /usr/sbin/nologin llmgw
chown llmgw:llmgw /etc/llmgw/server-key.pem
chmod 0400 /etc/llmgw/server-key.pem
StateDirectory=llmgw creates /var/lib/llmgw with the right owner, which is
where the store belongs. ProtectSystem=strict makes the rest of the filesystem
read-only, so anything else it writes to has to be named explicitly.
The agent — llmgw-agent.service
Runs as root, because it launches engines and needs device access.
KillMode=process is the line that matters and the reason this file exists
rather than being generated. Without it, systemd kills the whole control group
on stop — including the engines, which hold a loaded model and are serving
traffic. Engines are meant to outlive an agent restart: the router talks to them
directly, and the agent adopts them again when it comes back. Taking them down
with the agent turns a thirty-second upgrade into a full model reload on every
node.
systemd will log Found left-over process ... in control group while starting unit. Ignoring. on the next start. That is the mechanism working, not a fault.