Ilker ManapandClaude Opus 5.5
4d9284ce9f
The agent's defaults said eth0 before normalize looked for the interface holding the management address, so it never looked: a node on ens3 or bond0 had NCCL and Gloo bootstrap over an interface it does not have. It is now found, with eth0 only as the last resort. The agent unit sets LimitMEMLOCK=infinity. RDMA pins what it registers, and under systemd's 8 MB default an engine without the capability that lets root ignore the limit fails its first collective over InfiniBand. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
systemd units
Two roles, two units. Copy the one you need to /etc/systemd/system/, put the
matching configuration in /etc/llmgw/, and systemctl enable --now it.
The control plane — llmgw-server.service
Runs unprivileged. It needs to read its own TLS key, so that file must belong to
the llmgw user and nothing else:
useradd --system --home-dir /var/lib/llmgw --shell /usr/sbin/nologin llmgw
chown llmgw:llmgw /etc/llmgw/server-key.pem
chmod 0400 /etc/llmgw/server-key.pem
StateDirectory=llmgw creates /var/lib/llmgw with the right owner, which is
where the store belongs. ProtectSystem=strict makes the rest of the filesystem
read-only, so anything else it writes to has to be named explicitly.
The agent — llmgw-agent.service
Runs as root, because it launches engines and needs device access.
KillMode=process is the line that matters and the reason this file exists
rather than being generated. Without it, systemd kills the whole control group
on stop — including the engines, which hold a loaded model and are serving
traffic. Engines are meant to outlive an agent restart: the router talks to them
directly, and the agent adopts them again when it comes back. Taking them down
with the agent turns a thirty-second upgrade into a full model reload on every
node.
systemd will log Found left-over process ... in control group while starting unit. Ignoring. on the next start. That is the mechanism working, not a fault.