llmgw/deploy/systemd

systemd units

Two roles, two units. Copy the one you need to /etc/systemd/system/, put the matching configuration in /etc/llmgw/, and systemctl enable --now it.

The control plane — llmgw-server.service

Runs unprivileged. It needs to read its own TLS key, so that file must belong to the llmgw user and nothing else:

useradd --system --home-dir /var/lib/llmgw --shell /usr/sbin/nologin llmgw
chown llmgw:llmgw /etc/llmgw/server-key.pem
chmod 0400 /etc/llmgw/server-key.pem

StateDirectory=llmgw creates /var/lib/llmgw with the right owner, which is where the store belongs. ProtectSystem=strict makes the rest of the filesystem read-only, so anything else it writes to has to be named explicitly.

The agent — llmgw-agent.service

Runs as root, because it launches engines and needs device access.

KillMode=process is the line that matters and the reason this file exists rather than being generated. Without it, systemd kills the whole control group on stop — including the engines, which hold a loaded model and are serving traffic. Engines are meant to outlive an agent restart: the router talks to them directly, and the agent adopts them again when it comes back. Taking them down with the agent turns a thirty-second upgrade into a full model reload on every node.

systemd will log Found left-over process ... in control group while starting unit. Ignoring. on the next start. That is the mechanism working, not a fault.